wlc-monitor/README.md

# WLC Monitor

WLC Monitor polls a Cisco Catalyst 9800 wireless LAN controller over SNMPv3
and sends email when controller, access-point, or configuration state changes.
It can run continuously in Docker or as a one-shot systemd timer.

Polling makes silence observable: a failed poll is itself a signal, whereas
traps and syslog stop when a controller becomes unreachable.

## Alerts

| Severity | Condition | Detection |
|---|---|---|
| CRITICAL | Controller unreachable | Configured number of consecutive failed polls |
| CRITICAL | Controller rebooted | `sysUpTime` reset |
| CRITICAL | AP down | AP disappears from `cLApTable` |
| WARNING | AP rebooted | AP uptime reset |
| WARNING | AP rejoined | Association uptime reset while AP uptime did not |
| NOTICE | Running configuration changed | `ccmHistoryRunningLastChanged` advanced |
| NOTICE | Configuration remains unsaved | Running change is newer than the saved change beyond the threshold |
| RECOVERY | Controller or AP returns | Transition back to the available state |

TimeTicks wrap at about 497 days. WLC Monitor compares the apparent counter
movement with elapsed wall time so a normal wrap is not reported as a reboot.

## Requirements and compatibility

- Docker Engine with Compose v2; or Python 3.10 or newer.
- A Cisco Catalyst 9800 reachable over SNMPv3 authPriv.
- Access to the CISCO-LWAPP-AP-MIB and CISCO-CONFIG-MAN-MIB objects used by the
  controller.
- A reachable SMTP server.

The implementation was developed against a Catalyst 9800-CL running IOS XE
17.12. Other 9800 models and releases may expose different MIB behavior;
compatibility reports are welcome.

SMTP supports STARTTLS, implicit TLS, optional username/password
authentication, and plaintext delivery to a trusted relay. Do not select
`none` across an untrusted network: alert bodies contain infrastructure names
and operational state. Authentication is rejected in plaintext mode so SMTP
credentials cannot be sent over an unencrypted connection.

## Quick start with Docker

```bash
cp .env.example .env
chmod 600 .env
# Edit .env and set the controller, SNMP credentials, mail server,
# sender, and recipients.
docker compose pull
docker compose up -d --no-build
```

Tagged releases are published to
`git.snakedesert.se/lundberg/wlc-monitor`. To build the image locally instead,
run `docker compose up -d --build`.

Verify the controller and mail paths:

```bash
docker compose exec wlc-monitor python /app/wlc_monitor.py --show
docker compose exec wlc-monitor python /app/wlc_monitor.py --test-email
docker compose logs -f wlc-monitor
docker inspect --format '{{.State.Health.Status}}' wlc-monitor
```

The container runs unprivileged with a read-only root filesystem. Its named
`/data` volume holds comparison state, failure counts, and any email awaiting
retry. Preserve this volume across upgrades. Losing it creates a blank
baseline: the next successful poll is deliberately silent, and changes during
the gap cannot be reconstructed.

The health check reports whether the poll loop is progressing, not whether the
controller is reachable. Controller failure is a condition the monitor is
expected to observe, not a reason to restart it.

## Configuration

Environment variables override values in `config.ini`. In containers, prefer
environment variables or mounted secret files. For a host install, begin with
`config.ini.example`.

| Variable | Required/default | Meaning |
|---|---|---|
| `WLC_HOST` | required | Controller hostname or address |
| `WLC_PORT` | `161` | SNMP port |
| `WLC_SNMP_USER` | required | SNMPv3 username |
| `WLC_SNMP_AUTH` | required | Authentication passphrase |
| `WLC_SNMP_PRIV` | required | AES privacy passphrase |
| `WLC_SNMP_AUTH_PROTOCOL` | `sha` | `sha`, `sha224`, `sha256`, `sha384`, or `sha512` |
| `WLC_TIMEOUT` | `5` | Per-request timeout in seconds |
| `WLC_RETRIES` | `1` | SNMP retries |
| `MAIL_SMTP_HOST` | required | SMTP server |
| `MAIL_SMTP_PORT` | `25` | SMTP port |
| `MAIL_SMTP_SECURITY` | `none` | `none`, `starttls`, or `ssl` |
| `MAIL_SMTP_USER` | optional | SMTP username; password must also be set |
| `MAIL_SMTP_PASSWORD` | optional | SMTP password; username must also be set |
| `MAIL_FROM` | required | Envelope/header sender |
| `MAIL_TO` | required | Comma-separated recipients |
| `MAIL_SUBJECT_PREFIX` | `[WLC]` | Subject prefix |
| `MONITOR_INTERVAL` | `300` | Seconds between continuous polls |
| `MONITOR_FAIL_THRESHOLD` | `2` | Consecutive failures before alerting |
| `MONITOR_UNSAVED_MINUTES` | `60` | Time before unsaved configuration alerts |
| `MONITOR_STATE_FILE` | `/data/state.json` | Persistent state path |
| `TZ` | `UTC` in Compose | Time zone used in alert timestamps |

For secrets, `WLC_SNMP_AUTH_FILE`, `WLC_SNMP_PRIV_FILE`, and
`MAIL_SMTP_PASSWORD_FILE` may point to mounted files instead of placing values
directly in the process environment. Set either the direct variable or its
`_FILE` form, never both.

The SNMP authentication protocol must match the controller user. SHA-2 is
preferred where the IOS XE release supports it; `sha` remains available for
older configurations.

## Install with systemd

Create a dedicated account and install the files:

```bash
sudo useradd --system --no-create-home --shell /usr/sbin/nologin wlcmon
sudo install -d -o root -g wlcmon -m 0750 /opt/wlc-monitor
sudo install -o root -g wlcmon -m 0755 wlc_monitor.py /opt/wlc-monitor/
sudo install -o root -g wlcmon -m 0644 requirements.txt README.md /opt/wlc-monitor/
sudo cp config.ini.example /opt/wlc-monitor/config.ini
sudo chown root:wlcmon /opt/wlc-monitor/config.ini
sudo chmod 0640 /opt/wlc-monitor/config.ini
sudo python3 -m venv /opt/wlc-monitor/venv
sudo /opt/wlc-monitor/venv/bin/pip install -r /opt/wlc-monitor/requirements.txt
```

Edit `/opt/wlc-monitor/config.ini`, keeping
`state_file = /var/lib/wlc-monitor/state.json`, then install and start the
timer:

```bash
sudo install -m 0644 wlc-monitor.service wlc-monitor.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now wlc-monitor.timer
```

Verify:

```bash
sudo systemctl start wlc-monitor.service
journalctl -u wlc-monitor.service -n 20
systemctl list-timers wlc-monitor.timer
```

The unit creates `/var/lib/wlc-monitor` privately and applies a restrictive
umask. The configuration is readable only by root and the dedicated service
group.

## Local command-line use

```bash
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt
cp config.ini.example config.ini
chmod 600 config.ini
# Edit config.ini and use a writable state path such as ./state.json.

python wlc_monitor.py --show
python wlc_monitor.py --dry-run --no-save
python wlc_monitor.py --test-email
python wlc_monitor.py --loop --interval 60
python wlc_monitor.py --healthcheck
```

`--dry-run` prints alerts instead of sending them. It still updates state
unless combined with `--no-save`. `--show` returns a nonzero exit status when
the poll fails, making it suitable for scripts.

The first successful poll is deliberately silent because there is no previous
state to compare.

## Controller configuration

Use a read-only SNMPv3 authPriv user and restrict its ACL to the single
monitoring host. Exact SHA-2 syntax varies by IOS XE release; consult the
controller documentation and make `WLC_SNMP_AUTH_PROTOCOL` match.

```text
ip access-list standard SNMP-MON
 permit host <monitor-host-ip>
snmp-server group WLCMON v3 priv read v1default access SNMP-MON
snmp-server user wlcmon WLCMON v3 auth <sha-options> <auth-passphrase> \
 priv aes 128 <privacy-passphrase>
```

Do not paste real controller configuration, credentials, addresses, or device
output into issues.

## Delivery and state behavior

State is written atomically with mode `0600`. It can still contain controller
and AP names, software details, and queued alert bodies, so treat it as
operationally sensitive and do not commit or publish it.

Transition alerts that fail SMTP delivery are stored in a small durable FIFO
outbox and retried on the next poll. Delivery stops at the first failure so a
recovery cannot arrive before its outage. A continuously unsaved configuration
alerts once when it first exceeds the threshold, then becomes eligible again
after it is saved. A crash between the SMTP server accepting a message and the
state write can still cause a duplicate; recipients should tolerate
at-least-once delivery.

## Operational limitations

- SNMP reveals that a configuration changed, not what changed. Inspect the
  controller's configuration history for the actual diff.
- The poll interval is the detection resolution. An AP that disconnects and
  recovers entirely between polls is invisible.
- AP renames look like a down and join pair because APs are keyed by name.
- The monitoring host remains a single point of failure. Use an external
  dead-man/heartbeat service if silent monitor failure must be detected.
- A missing or partial AP table is treated as a failed poll to avoid false
  reboot/rejoin alerts.

## Development

Run the same checks used by CI:

```bash
python -m py_compile wlc_monitor.py
python -m unittest discover -v
python -m pip check
docker compose config --quiet
docker build -t wlc-monitor:test .
```

Forgejo Actions runs these checks for pushes and pull requests. Pushes to
`main` publish `edge` and commit-SHA container tags; version tags matching
`v*` publish the version and `latest` tags. Registry publication requires the
repository secret `REGISTRY_TOKEN`, following the same runner setup as the
other projects on this Forgejo instance.

## Contributing and security

Bug reports and pull requests are welcome; see
[CONTRIBUTING.md](CONTRIBUTING.md). Report vulnerabilities privately as
described in [SECURITY.md](SECURITY.md), not in a public issue.

## License

WLC Monitor is available under the [MIT License](LICENSE).