Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add monitoring #36

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions osfv_cli/monitoring/zabbix/External_Check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# External Check Zabbix

This is a brief guide on setting up external checks in Zabbix.
You can find example scripts in this folder.

## Preparation

1. Make sure your script is in

```
/usr/lib/zabbix/externalscripts
```

> The default location of externalscripts can be changed in `/etc/zabbix/zabbix_server.conf` by editing `ExternalScripts=/usr/lib/zabbix/externalscripts` (make sure to remove #)
Copy link
Contributor

@macpijan macpijan May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


**Ensure that you have only one tab open when creating anything in Zabbix (items/triggers/hosts).**
> It's likely that the last used tab is overwriting session data.

## Create the item in Zabbix
>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>

> Item is a tool that gathers data from device, system or external script
2. Navigate to **Configuration** > **Hosts**.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This numbering makes no sense. Either use 1 everywhere (and let the markdown renderer handle it), or use incremental numbering in each section manually.

2. Go to the **Items** next to the host you want to monitor.
2. Click **Create item** in top right.
- Change Type to **External check**.
- As key use your script name together with desired **Macro**, for example `script.sh[{HOST.IP}]`.
+ You can combine multiple Macros `script.sh[{HOST.DESCRIPTION},{HOST.ID},{HOST.CONN}]`.

[list of macros](https://www.zabbix.com/documentation/current/en/manual/appendix/macros/supported_by_location#host-inventory)

## Create the trigger
>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>

> Trigger is a rule that watches for specific events, like high CPU usage.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add empty line below

3. Navigate to **Configuration** > **Hosts**.
3. Go to the **Triggers** next to the host with the item you want to monitor.
3. Click **Create trigger** in top right.
- Severity: Select as needed.
- Expression: press Add
+ Item: Select the item for which you want to create a trigger.
+ Function: Defines how your trigger will be activated.
+ Result: Actual rule of activation.
> If your script returns numeric values directly and/or you don't won't to calculate the average result, simply use `last()` function
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add empty line above


## Testing

4. Test manually.
- login as **zabbix** user and go to the script location.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please indent

- run your script as zabbix.
> You can directly incorporate information from Zabbix macros into your command, such as ./script.sh '192.168.10.0'.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add empty line above


4. Test in zabbix web interface.
- Go to your item location.
- If you are using external checks, **Test** option should be available.
- Check if you Macros give proper data to your script.
- Press **Get value and test**.

4. Common problems.
- Timeout: you can change (not VM)Timeout(and TrapperTimeout if needed) settings in /etc/zabbix/zabbix_server.conf
- Make sure all necesery files and commands are accessible for zabbix user.
- If you find something else, please update the file.

## Templates

5. Navigate to **Configuration** > **Templates**.
5. Click **Create template** in top right.
- Specify the name, group and other information if needed.
5. Add **items**, **trigger** and **macros** if needed.
5. Link the template to **Hosts**
5. Mass update
- Go to **Hosts**.
- Check the boxes next to the desired hosts.
- Scroll to the bottom and click **Mass update**.
+ Select **Link templates**.
+ Choose **Link**.
+ Enter your template name.
+ Press **Update**.

5. Single update
+ Open desired **Host**.
+ Add your Template under Templates.
+ Press **Update**.
59 changes: 59 additions & 0 deletions osfv_cli/monitoring/zabbix/external_flash_probe/flash_probe.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/bin/bash

cd /usr/lib/zabbix/externalscripts/

source /usr/lib/zabbix/externalscripts/venv/bin/activate

input_file="/usr/lib/zabbix/externalscripts/macros.txt"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this file, and what it contains?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$input_file in this case macros.txt
contains list of IPs to use
I explained it in new commit

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


output_pass="/usr/lib/zabbix/externalscripts/output_pass.txt"
output_fail="/usr/lib/zabbix/externalscripts/output_fail.txt"
output_used="/usr/lib/zabbix/externalscripts/output_used.txt"
stolen="/usr/lib/zabbix/externalscripts/stolen.txt"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the purpose of this file? Can you briefly document each of them as a comment above?


> "$output_used"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When and where do you clear those files? Why not do it here at the beginning of the script?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i used redirects to clear the file out of habbit, already changed them to rm
> "$output_used"
> "$output_pass"
> "$output_fail"
> "$stolen"

> "$output_pass"
> "$output_fail"
> "$stolen"

check_snipeit() {
ip=$1
if [[ "$ip" =~ [0-9] ]]; then
if osfv_cli snipeit list_used | grep -qw "$ip"; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably not needed, see below.

return 0 # IP is checked out
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic seems reversed. When device is taken, I would treat it as some form of error. And we return 0 on sucess, and non-zero on error.

else
osfv_cli snipeit check_out --rte_ip "$ip"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simply try to check out. If platform is used, you will simply get an arror:

Error checking out asset 52
Response data: {'status': 'error', 'messages': 'That asset is not available for checkout!', 'payload': {'asset': '00052'}}

echo "$ip" >> "$stolen"
return 1 # IP not checked out
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return 0 on success

fi
else
return 1 # IP contains no numbers
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return non-zero on error. ideally, different error code for different type of error (not 1 in two places).

fi
}

probe_flash() {
ip=$1
output_flash=$(osfv_cli rte --rte_ip "$ip" flash probe)
if echo "$output_flash" | grep -qE 'Found .* flash chip ".*" \(.*\) on .*\.'; then
echo "$ip" >> "$output_pass"
else
echo "$ip" >> "$output_fail"
fi
}

while IFS= read -r ip || [ -n "$ip" ]; do
if check_snipeit "$ip"; then
echo "$ip found in snipeit list_used"
# echo "$ip" >> "$output_used"
else
echo "$ip not found in snipeit list_used, probing flash..."
probe_flash "$ip"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about simply invoking check_in below? After we call probe_flash, we no longer need the device to be checked out, right? This makes another loop and stolen.txt not needed.

fi
done < "$input_file"

while IFS= read -r ip || [ -n "$ip" ]; do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was it added just recently, and should fix the devices being reserved on robot for too long?

osfv_cli snipeit check_in --rte_ip "$ip"
done < "$stolen"

deactivate

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove empty line

15 changes: 15 additions & 0 deletions osfv_cli/monitoring/zabbix/external_flash_probe/flash_read.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we actually use flash_read.sh and flash_write.sh files in monitoring process?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, flash_read.sh and flash_write.sh are executed by Zabbix and are separated into two files for ease of use with zabbix. Meanwhile flash_probe.sh is executed by a crontab or another automation tool due to Zabbix's timeout limitations.


fail="/usr/lib/zabbix/externalscripts/output_fail.txt"
used="/usr/lib/zabbix/externalscripts/output_used.txt"
pass="/usr/lib/zabbix/externalscripts/output_pass.txt"

ip=$1

if grep -qw "$ip" "$fail"; then
echo 0
elif grep -qw "$ip" "$used"; then
echo 1
Copy link
Contributor

@macpijan macpijan May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use consistent indentation

elif grep -qw "$ip" "$pass"; then
echo 2
fi
24 changes: 24 additions & 0 deletions osfv_cli/monitoring/zabbix/external_flash_probe/flash_write.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash

output_file="/usr/lib/zabbix/externalscripts/macros.txt"

touch "$output_file"

if [ -z $1 ]; then
# echo "No macro provided."
exit 1
fi

macro_exists() {
grep -qF "$1" "$output_file"
}

for macro in "$@"; do
if ! macro_exists "$macro"; then
echo "$macro" >> "$output_file"
echo 1
else
# echo "Macro '$macro' already exists in the file."
echo 0
fi
done
1 change: 1 addition & 0 deletions osfv_cli/osfv_cli/osfv_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ def list_my_assets(snipeit_api, args):
for asset in my_assets:
print_asset_details(asset)


# List unused assets
def list_unused_assets(snipeit_api, args):
all_assets = snipeit_api.get_all_assets()
Expand Down