237 lines
9.1 KiB
Markdown
237 lines
9.1 KiB
Markdown
# WLC Monitor
|
|
|
|
WLC Monitor polls a Cisco Catalyst 9800 wireless LAN controller over SNMPv3
|
|
and sends email when controller, access-point, or configuration state changes.
|
|
It can run continuously in Docker or as a one-shot systemd timer.
|
|
|
|
Polling makes silence observable: a failed poll is itself a signal, whereas
|
|
traps and syslog stop when a controller becomes unreachable.
|
|
|
|
## Alerts
|
|
|
|
| Severity | Condition | Detection |
|
|
|---|---|---|
|
|
| CRITICAL | Controller unreachable | Configured number of consecutive failed polls |
|
|
| CRITICAL | Controller rebooted | `sysUpTime` reset |
|
|
| CRITICAL | AP down | AP disappears from `cLApTable` |
|
|
| WARNING | AP rebooted | AP uptime reset |
|
|
| WARNING | AP rejoined | Association uptime reset while AP uptime did not |
|
|
| NOTICE | Running configuration changed | `ccmHistoryRunningLastChanged` advanced |
|
|
| NOTICE | Configuration remains unsaved | Running change is newer than the saved change beyond the threshold |
|
|
| RECOVERY | Controller or AP returns | Transition back to the available state |
|
|
|
|
TimeTicks wrap at about 497 days. WLC Monitor compares the apparent counter
|
|
movement with elapsed wall time so a normal wrap is not reported as a reboot.
|
|
|
|
## Requirements and compatibility
|
|
|
|
- Docker Engine with Compose v2; or Python 3.10 or newer.
|
|
- A Cisco Catalyst 9800 reachable over SNMPv3 authPriv.
|
|
- Access to the CISCO-LWAPP-AP-MIB and CISCO-CONFIG-MAN-MIB objects used by the
|
|
controller.
|
|
- A reachable SMTP server.
|
|
|
|
The implementation was developed against a Catalyst 9800-CL running IOS XE
|
|
17.12. Other 9800 models and releases may expose different MIB behavior;
|
|
compatibility reports are welcome.
|
|
|
|
SMTP supports STARTTLS, implicit TLS, optional username/password
|
|
authentication, and plaintext delivery to a trusted relay. Do not select
|
|
`none` across an untrusted network: alert bodies contain infrastructure names
|
|
and operational state. Authentication is rejected in plaintext mode so SMTP
|
|
credentials cannot be sent over an unencrypted connection.
|
|
|
|
## Quick start with Docker
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
chmod 600 .env
|
|
# Edit .env and set the controller, SNMP credentials, mail server,
|
|
# sender, and recipients.
|
|
docker compose up -d --build
|
|
```
|
|
|
|
Verify the controller and mail paths:
|
|
|
|
```bash
|
|
docker compose exec wlc-monitor python /app/wlc_monitor.py --show
|
|
docker compose exec wlc-monitor python /app/wlc_monitor.py --test-email
|
|
docker compose logs -f wlc-monitor
|
|
docker inspect --format '{{.State.Health.Status}}' wlc-monitor
|
|
```
|
|
|
|
The container runs unprivileged with a read-only root filesystem. Its named
|
|
`/data` volume holds comparison state, failure counts, and any email awaiting
|
|
retry. Preserve this volume across upgrades. Losing it creates a blank
|
|
baseline: the next successful poll is deliberately silent, and changes during
|
|
the gap cannot be reconstructed.
|
|
|
|
The health check reports whether the poll loop is progressing, not whether the
|
|
controller is reachable. Controller failure is a condition the monitor is
|
|
expected to observe, not a reason to restart it.
|
|
|
|
## Configuration
|
|
|
|
Environment variables override values in `config.ini`. In containers, prefer
|
|
environment variables or mounted secret files. For a host install, begin with
|
|
`config.ini.example`.
|
|
|
|
| Variable | Required/default | Meaning |
|
|
|---|---|---|
|
|
| `WLC_HOST` | required | Controller hostname or address |
|
|
| `WLC_PORT` | `161` | SNMP port |
|
|
| `WLC_SNMP_USER` | required | SNMPv3 username |
|
|
| `WLC_SNMP_AUTH` | required | Authentication passphrase |
|
|
| `WLC_SNMP_PRIV` | required | AES privacy passphrase |
|
|
| `WLC_SNMP_AUTH_PROTOCOL` | `sha` | `sha`, `sha224`, `sha256`, `sha384`, or `sha512` |
|
|
| `WLC_TIMEOUT` | `5` | Per-request timeout in seconds |
|
|
| `WLC_RETRIES` | `1` | SNMP retries |
|
|
| `MAIL_SMTP_HOST` | required | SMTP server |
|
|
| `MAIL_SMTP_PORT` | `25` | SMTP port |
|
|
| `MAIL_SMTP_SECURITY` | `none` | `none`, `starttls`, or `ssl` |
|
|
| `MAIL_SMTP_USER` | optional | SMTP username; password must also be set |
|
|
| `MAIL_SMTP_PASSWORD` | optional | SMTP password; username must also be set |
|
|
| `MAIL_FROM` | required | Envelope/header sender |
|
|
| `MAIL_TO` | required | Comma-separated recipients |
|
|
| `MAIL_SUBJECT_PREFIX` | `[WLC]` | Subject prefix |
|
|
| `MONITOR_INTERVAL` | `300` | Seconds between continuous polls |
|
|
| `MONITOR_FAIL_THRESHOLD` | `2` | Consecutive failures before alerting |
|
|
| `MONITOR_UNSAVED_MINUTES` | `60` | Time before unsaved configuration alerts |
|
|
| `MONITOR_STATE_FILE` | `/data/state.json` | Persistent state path |
|
|
| `TZ` | `UTC` in Compose | Time zone used in alert timestamps |
|
|
|
|
For secrets, `WLC_SNMP_AUTH_FILE`, `WLC_SNMP_PRIV_FILE`, and
|
|
`MAIL_SMTP_PASSWORD_FILE` may point to mounted files instead of placing values
|
|
directly in the process environment. Set either the direct variable or its
|
|
`_FILE` form, never both.
|
|
|
|
The SNMP authentication protocol must match the controller user. SHA-2 is
|
|
preferred where the IOS XE release supports it; `sha` remains available for
|
|
older configurations.
|
|
|
|
## Install with systemd
|
|
|
|
Create a dedicated account and install the files:
|
|
|
|
```bash
|
|
sudo useradd --system --no-create-home --shell /usr/sbin/nologin wlcmon
|
|
sudo install -d -o root -g wlcmon -m 0750 /opt/wlc-monitor
|
|
sudo install -o root -g wlcmon -m 0755 wlc_monitor.py /opt/wlc-monitor/
|
|
sudo install -o root -g wlcmon -m 0644 requirements.txt README.md /opt/wlc-monitor/
|
|
sudo cp config.ini.example /opt/wlc-monitor/config.ini
|
|
sudo chown root:wlcmon /opt/wlc-monitor/config.ini
|
|
sudo chmod 0640 /opt/wlc-monitor/config.ini
|
|
sudo python3 -m venv /opt/wlc-monitor/venv
|
|
sudo /opt/wlc-monitor/venv/bin/pip install -r /opt/wlc-monitor/requirements.txt
|
|
```
|
|
|
|
Edit `/opt/wlc-monitor/config.ini`, keeping
|
|
`state_file = /var/lib/wlc-monitor/state.json`, then install and start the
|
|
timer:
|
|
|
|
```bash
|
|
sudo install -m 0644 wlc-monitor.service wlc-monitor.timer /etc/systemd/system/
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl enable --now wlc-monitor.timer
|
|
```
|
|
|
|
Verify:
|
|
|
|
```bash
|
|
sudo systemctl start wlc-monitor.service
|
|
journalctl -u wlc-monitor.service -n 20
|
|
systemctl list-timers wlc-monitor.timer
|
|
```
|
|
|
|
The unit creates `/var/lib/wlc-monitor` privately and applies a restrictive
|
|
umask. The configuration is readable only by root and the dedicated service
|
|
group.
|
|
|
|
## Local command-line use
|
|
|
|
```bash
|
|
python3 -m venv .venv
|
|
. .venv/bin/activate
|
|
python -m pip install -r requirements.txt
|
|
cp config.ini.example config.ini
|
|
chmod 600 config.ini
|
|
# Edit config.ini and use a writable state path such as ./state.json.
|
|
|
|
python wlc_monitor.py --show
|
|
python wlc_monitor.py --dry-run --no-save
|
|
python wlc_monitor.py --test-email
|
|
python wlc_monitor.py --loop --interval 60
|
|
python wlc_monitor.py --healthcheck
|
|
```
|
|
|
|
`--dry-run` prints alerts instead of sending them. It still updates state
|
|
unless combined with `--no-save`. `--show` returns a nonzero exit status when
|
|
the poll fails, making it suitable for scripts.
|
|
|
|
The first successful poll is deliberately silent because there is no previous
|
|
state to compare.
|
|
|
|
## Controller configuration
|
|
|
|
Use a read-only SNMPv3 authPriv user and restrict its ACL to the single
|
|
monitoring host. Exact SHA-2 syntax varies by IOS XE release; consult the
|
|
controller documentation and make `WLC_SNMP_AUTH_PROTOCOL` match.
|
|
|
|
```text
|
|
ip access-list standard SNMP-MON
|
|
permit host <monitor-host-ip>
|
|
snmp-server group WLCMON v3 priv read v1default access SNMP-MON
|
|
snmp-server user wlcmon WLCMON v3 auth <sha-options> <auth-passphrase> \
|
|
priv aes 128 <privacy-passphrase>
|
|
```
|
|
|
|
Do not paste real controller configuration, credentials, addresses, or device
|
|
output into issues.
|
|
|
|
## Delivery and state behavior
|
|
|
|
State is written atomically with mode `0600`. It can still contain controller
|
|
and AP names, software details, and queued alert bodies, so treat it as
|
|
operationally sensitive and do not commit or publish it.
|
|
|
|
Transition alerts that fail SMTP delivery are stored in a small durable FIFO
|
|
outbox and retried on the next poll. Delivery stops at the first failure so a
|
|
recovery cannot arrive before its outage. A continuously unsaved configuration
|
|
alerts once when it first exceeds the threshold, then becomes eligible again
|
|
after it is saved. A crash between the SMTP server accepting a message and the
|
|
state write can still cause a duplicate; recipients should tolerate
|
|
at-least-once delivery.
|
|
|
|
## Operational limitations
|
|
|
|
- SNMP reveals that a configuration changed, not what changed. Inspect the
|
|
controller's configuration history for the actual diff.
|
|
- The poll interval is the detection resolution. An AP that disconnects and
|
|
recovers entirely between polls is invisible.
|
|
- AP renames look like a down and join pair because APs are keyed by name.
|
|
- The monitoring host remains a single point of failure. Use an external
|
|
dead-man/heartbeat service if silent monitor failure must be detected.
|
|
- A missing or partial AP table is treated as a failed poll to avoid false
|
|
reboot/rejoin alerts.
|
|
|
|
## Development
|
|
|
|
Run the same checks used by CI:
|
|
|
|
```bash
|
|
python -m py_compile wlc_monitor.py
|
|
python -m unittest discover -v
|
|
python -m pip check
|
|
docker compose config --quiet
|
|
docker build -t wlc-monitor:test .
|
|
```
|
|
|
|
## Contributing and security
|
|
|
|
Bug reports and pull requests are welcome; see
|
|
[CONTRIBUTING.md](CONTRIBUTING.md). Report vulnerabilities privately as
|
|
described in [SECURITY.md](SECURITY.md), not in a public issue.
|
|
|
|
## License
|
|
|
|
WLC Monitor is available under the [MIT License](LICENSE).
|