44 lines
2.7 KiB
Markdown
44 lines
2.7 KiB
Markdown
# contrib: getting the domains CDB onto the service host
|
|
|
|
migadu_counters reads limits from the rspamd domains CDB that the agent builds
|
|
(`extract.sh` → `/usr/local/etc/agent/build/rspamd/rspamd_domains_db`). The
|
|
files here fetch it the way rcptd fetches its maps (`ansible-freebsd3`
|
|
`roles/rcptd/templates/{build.sh.j2,rcptd-maps.cron.j2}`); the future Ansible
|
|
role templates them.
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `pull-domains.sh` | rsync the CDB from the agent, `cmp` against the installed `.orig`, and only on change verify (`cdb -s`) and install it with an atomic `mv`; `.pull-lock` with a 120 s max age; log to `/var/log/migadu_counters/pull.log` via `tee -a`. |
|
|
| `migadu_counters-pull.cron` | FreeBSD `@15` crontab fragment (15 s after the previous run completes) wrapped in `timeout -k 30 300`. |
|
|
| `pull-domains_test.sh` | Test against a local source directory (`RSYNC_SRC` override): unchanged → no `mv`, no reload; changed → atomic install, `.orig` updated, reload; corrupt or empty CDB → not installed, non-zero exit, old file untouched; lock handling. Run with `make test-contrib` (needs tinycdb's `cdb` and `rsync`). |
|
|
|
|
## What the Ansible role must template
|
|
|
|
Environment of `pull-domains.sh` (every variable has the production default
|
|
shown in the script):
|
|
|
|
- `AGENT_HOST` (`agent@<agent host>`) and `AGENT_PATH`; ssh is forced to IPv4
|
|
(`RSYNC_RSH="ssh -4"`), the agent's host key must be accepted beforehand
|
|
(rcptd has `accept-agent-hostkey.sh.j2` for that).
|
|
- `MAPS_DIR` (pulled copy), `DOMAINS_DIR` / `DOMAINS_CDB` (what `domains_cdb`
|
|
in `config.toml` points at), `LOG_FILE`, `LOCK_FILE`.
|
|
- `RELOAD_CMD` (default `service migadu_counters reload`, i.e. SIGHUP). The
|
|
daemon's fsnotify watcher already reloads on the `mv`; the reload is belt
|
|
and braces.
|
|
- The cron user (`migadu_counters`) must own `MAPS_DIR`, `DOMAINS_DIR` and the
|
|
log directory, and have the ssh key for the agent host. `timeout`, `rsync`
|
|
and `cdb` (tinycdb) must be installed.
|
|
|
|
## Rules the daemon relies on
|
|
|
|
- **Never rewrite the CDB in place.** The daemon keeps the file open; an
|
|
in-place rewrite truncates the inode under it and breaks every lookup until
|
|
the next reload. The script only ever `mv`s a verified dotfile over the
|
|
target, which is atomic on the same filesystem (`DOMAINS_DIR` must hold both).
|
|
- A file that does not open as a CDB or holds no records is refused and the
|
|
installed file kept; the daemon applies the same rule on reload.
|
|
- Memory stays flat across the 15-second cycles: a reload replaces the whole
|
|
in-memory DB object (2 KB index plus a bounded parse cache,
|
|
`max_cached_domains`, default 20 000 records) and the old one is closed after
|
|
a 30 s grace period. Unchanged pulls are a `cmp` and nothing else.
|