cheap_system_monitor/README.md

# Cheap System Monitor

A small Linux system monitor for Ubuntu and Raspberry Pi OS (including the
Raspberry Pi 5). It stores metrics in a fixed-size RRD database and produces
PNG graphs for the last 3 hours, 1 day, 2 weeks, and 1 year.

It collects:

- processor load averages (1, 5, and 15 minutes)
- CPU user, system, and nice usage
- total network receive/transmit throughput
- disk usage for `/` and up to two additional mountpoints
- memory usage
- system uptime in days
- CPU temperature and active Raspberry Pi throttling
- failed systemd services and kernel out-of-memory kills
- available filesystem space, inode usage, and read-only filesystem state
- CPU I/O wait
- a combined storage operational-health score from 0 to 10

## Install

Python 3 is already included in supported distributions. Install RRDTool,
clone/copy this directory, and run the installer:

```sh
sudo apt update
sudo apt install rrdtool
sudo ./install.sh
```

The executable application files are installed under
`/opt/cheap_system_monitor`. Configuration remains in
`/opt/cheap_system_monitor/monitor.conf`, and the RRD database remains under
`/var/lib/cheap-system-monitor` so code, configuration, and mutable data stay
separate.

Edit `/opt/cheap_system_monitor/monitor.conf` to choose network interfaces,
mountpoints, graphs, and the graph directory. For example, use
`mountpoints = /, /boot/firmware, /mnt/data` to monitor two additional disks;
every configured path must be mounted and accessible.
Restarting is unnecessary after configuration changes—the next timer runs use
the new values. If the RRD has already been created, do not change the order of
mountpoints unless you intentionally want the historical graph labels to refer
to different disks.

Use the `graphs` setting to choose which graphs are generated and shown, and
to set their dashboard order. Available names are `load`, `cpu`, `thermal`,
`trouble`, `network`, `disk`, `storage-health`, and `uptime`. For a virtual
machine where temperature is not meaningful, for example:

```ini
graphs = load, cpu, trouble, network, disk, storage-health, uptime
```

Omitting a graph does not disable collection of its underlying metrics. This
keeps the RRD schema consistent if the graph is enabled later.

The generated PNG files and four responsive, auto-refreshing dashboard pages
are placed in `/var/www/html/system-monitor` by default. `index.html` shows
the last 3 hours and links to the 1-day, 2-week, and 1-year pages.
With a web server installed, such as nginx or Apache, they can be served
directly; otherwise they can simply be opened or copied as PNG files.

## Check operation

```sh
systemctl list-timers 'cheap-system-monitor*'
journalctl -u cheap-system-monitor.service -n 20
rrdtool info /var/lib/cheap-system-monitor/system.rrd
```

Run either job manually with:

```sh
sudo systemctl start cheap-system-monitor.service
sudo systemctl start cheap-system-monitor-graphs.service
```

The first meaningful network rate appears after two collection samples. A new
installation naturally needs time to accumulate history; RRD archives are
consolidated to five-minute samples for two weeks and hourly samples for a year.

The RRD schema is fixed when the database is created. When upgrading from a
version without the latest monitoring data sources, move or remove the
existing `system.rrd` before the next collection if preserving its old history
is not required. The collector will then create a database with the new data
sources. Temperature is read from Linux thermal zones. Active Raspberry Pi
throttling is read with `vcgencmd`; it appears as unknown on other systems.
Failed services are counted with `systemctl`. OOM kills come from the kernel's
cumulative `/proc/vmstat` counter and are graphed as events per minute.
Available space, inode consumption, and read-only state are retained as inputs
to the per-mountpoint health score rather than displayed as separate graphs.
I/O wait is included in the CPU graph. These portable indicators work with SD
cards and USB drives without requiring SMART support.

The storage operational-health graph shows one score for each configured
mountpoint. A score of 10 means no current problem is visible, while a read-only
filesystem scores 0. Space and inode usage begin reducing that filesystem's
score above 75%; either reaches 0 when exhausted. Host-wide I/O wait above 10%
can subtract up to three points from every score because Linux's CPU wait metric
cannot attribute the delay to one mountpoint. This is an operational-risk score,
not a measurement of flash wear or remaining media life; SD cards and many USB
bridges do not expose the hardware data needed to estimate those reliably.

## Run from the source tree

For development or a non-system installation, copy the example configuration,
change its paths to writable locations, then use:

```sh
cp cheap-system-monitor.conf.example cheap-system-monitor.conf
python3 system_monitor.py --config cheap-system-monitor.conf
python3 generate_graphs.py --config cheap-system-monitor.conf
```