# Plan: `migadu_counters` — real-time mail counting service (Go) ## Context Daily send/receive limits used to be counted and enforced by the rspamd Lua plugin `ansible-freebsd2/roles/rspamd-postfix/templates/rspamd/plugins.d/migadu_counters.lua` (Redis counters per address + org, soft-reject/reject on breach, web app notified once). Mail now flows mizu → mailqueuer and bypasses rspamd, so nothing counts in real time. `stats-merger/internal/quota` recomputes usage from the delivery ledger after each merge, but it is reporting-only today: only fernrohr reads `/api/quota/{status,stats,overquota}`; nothing calls `/check`. `migadu_counters` is a new standalone Go service that becomes the single counting source and the enforcement trigger: - mailqueuer calls it **fire-and-forget**, once per ingested recipient, from its publish loop; - one route per flow: `POST /in`, `POST /out` (`/rel` later, not planned here); - it increments Redis counters at **Address / Domain / Organization** level, in the key format the web app already reads. The domain level is turned back on (Lua defined it but never tracked it); - when an increment crosses a limit it **notifies the web app** (`migadu_web` `GET /api/limits`), which decides what to do (notify the user, remove sending/receiving); - a small CLI, `counterctl`, shows an address's incoming/outgoing counters for today or a given date (see "CLI: counterctl" below). The service returns no verdicts and enforces nothing itself. Out of scope: `/rel`, the mailqueuer client change, retiring stats-merger quota, the web app's `D` branch, deployment (see prerequisites at the end). ## How it fits together ``` sender ──SMTP──▶ mizu ──POST /ingest──▶ mailqueuer X-Trace-Id, X-Mail-From, │ one queue message per recipient; X-Mail-To, X-Auth-User, │ after each successful publish: X-Junk │ fire-and-forget HTTP call, reply ignored ▼ POST /out {"trace_id","auth_user","mail_from","recipient"} ← X-Auth-User set POST /in {"trace_id","mail_from","recipient","junk"} ← X-Auth-User empty Authorization: Bearer (mailqueuer sends it, migadu_counters checks it; 401 if wrong) │ ▼ ┌─────────────────────────────────────┐ │ migadu_counters (daemon :8095) │ │ ┌─────────────────────────────────┐ │ rejected here, nothing counted: │ │ HTTP validation layer │ │ 405 method · 413 body > 64 KB │ │ method → size → bearer token → │ │ 401 bad/missing token · 403 source IP │ │ source IP → content-type → JSON │ │ 415 content-type · 400 json / fields / │ │ → fields → e-mail syntax │ │ e-mail syntax · 503 handler timeout │ └───────────────┬─────────────────┘ │ │ ▼ validated request │ rspamd_domains_db ───▶│ counting: 1. domain in CDB? → org_id, limits │ (CDB: domain → │ 2. MULTI: INCR A/D/O keys, EXPIRE 31 d │ org_id | limits) │ 3. did this INCR cross a limit? → notify │ └───┬─────────────────────────┬───────┘ admin :9095 /health /metrics │ │ Redis Sentinel ◀──┘ └──────▶ GET https://admin.migadu.com/api/limits writes → master ?level=&id=&flow=&period=d&by=&count= reads → replicas (basic auth) → migadu_web notifies the user DB 0: cnt///d// or disables sending/receiving counterctl --address x@y [--date YYYY-MM-DD] ──GET /counters (Bearer)──▶ migadu_counters ──▶ Redis replicas ``` ## Existing pieces reused or matched - **Limits**: the CDB the Lua plugin reads, `/usr/local/etc/rspamd/db/rspamd_domains_db`, built by `ansible-freebsd2/roles/agent/templates/extract.sh.j2:478` with tinycdb `cdb -m` (no NUL terminators, exact-byte lookups like rcptd). Key = `domain`, value = `|O/i=..,O/o=..,D/i=..,A/o=..,alice@/o=..`. Parsing and limit resolution follow `stats-merger/internal/quota/quota.go` (`ParseLimits` L66, `DomainLimits.limit` L116): keys lowercased, malformed pairs skipped, `@/` overrides `A/`, missing key = unlimited. A domain present in the CDB is "Migadu-hosted". - **Redis keys** — must match the web app's key builder (`migadu_web/apps/core/lib/core/redis_sentinel.ex:73-103`): `cnt///d//` with id = address / domain / org_id, **date in UTC** (web app uses `Date.utc_today()`), **Redis DB 0** (web app reads `:redis_sentinel_db0`; rspamd uses `redis_rspamd_db: 0`). TTL 31 days. DB 1 and the `/s` storage keys belong to the web app; never touched. The web app may DEL today's keys to reset a customer (`redis_sentinel.ex:145-153`); counting must keep working after that, and a second crossing must notify again. - **Web app notification**: `GET https://admin.migadu.com/api/limits?level=&id=&flow=&period=&by=&count=` with basic auth (`apps/admin_web/lib/admin_web/controllers/internal_api/notification_controller.ex:33`, `router.ex:99` `Plug.BasicAuth`). All six params are pattern-matched, so all must be present. `id` = address for A, domain name for D, org id for O. Credentials: `vault_migadu_web.internal_api_basic_auth_*` (see `rspamd-postfix/templates/rspamd/local.d/migadu_counters.conf.j2`). - **Redis topology**: `ansible-freebsd2/roles/redis-sentinel`, `inventory/hosts.ini:160`: hosts `redisSentinel0-2`, Redis :6379 + Sentinel :26379, master name `mymaster`, 1 master + 2 replicas, one password (`vault_redis_sentinel_password`) for Redis; Sentinel itself has no `requirepass`. - **Service conventions** from `~/Work/rcptd` (`cmd/rcptd/main.go`): TOML config via `-config` + default paths, bearer `auth_token` with env override, slog, CDB behind `atomic.Pointer` with a 30 s grace close, SIGHUP + fsnotify reload (`internal/watcher/watcher.go` table mode), admin port with `/health` + `/metrics`, graceful shutdown. No dependency on rcptd's `shared` module (not present locally). - **Libraries** (in the module cache): `github.com/colinmarc/cdb`, `github.com/BurntSushi/toml` v1.6.0, `github.com/fsnotify/fsnotify` v1.9.0, `github.com/redis/go-redis/v9` v9.7.3, `github.com/prometheus/client_golang`; tests: `github.com/alicebob/miniredis/v2` v2.34.0. ## API (called by mailqueuer, fire-and-forget, one request per recipient) > **Revised 2026-10-14 (review round 1, `review/README.md` #1–#6):** `/in` and `/out` now validate, enqueue > and answer `202 {"accepted":true}`; counting runs in a worker pool (`queue_size`, `workers`) under its own > time budget, and the counted values appear only in the counting log and `GET /counters`. A full queue answers > `503 {"error":"queue full"}`. Only the counted sides are validated (`mail_from` never rejects; `auth_user` and > `recipient` independently), IDN domains are converted to punycode, `user@dom@SUFFIX` logins count under the > base address. `INCR` replaces `INCRBY`; per-key EXEC failures and lost replies are handled individually > (re-read from the master). Parsed CDB records are cached (`max_cached_domains`). `auth_token` is mandatory, > unknown config keys are fatal, the default API port is `:5555`. The `200 {"counted":…}` contract below is > the original design and is superseded by the README. mailqueuer's ingest handler is already per recipient: it loops over `toPublish` (`mailqueuer/internal/handler/ingest.go:235-306`), publishes one queue message per recipient (own uuid `id`, L237) and marks dedup only after a successful publish (L263-272). Both routes are called from inside that loop, right after a successful `Publish`, with exactly one recipient. The route is chosen by `X-Auth-User`: non-empty → `/out`, empty → `/in`. mailqueuer never waits for or acts on the response. **Authentication**: every request carries `Authorization: Bearer ` — mailqueuer sends it (the same header it already sends to rcptd, `internal/routing/routing.go:256-258`), migadu_counters validates it (constant-time compare against `auth_token`; missing/wrong → `401`, nothing counted). One shared token, set in both configs; no other auth. All routes take a JSON body ≤ 64 KB. Processing is **synchronous**: one Redis round trip, then the response. Only web-app notifications run in the background. - `200` `{"counted":[{"level":"A","id":"x@example.com","flow":"o","count":12,"limit":100,"reached":false},…]}` (`reached: true` on the increment that crossed the limit → notification triggered). Skipped requests return `{"counted":[],"skipped":"exempt|unknown_domain|junk"}`. - `400` invalid body/address, `401` bad token, `503` Redis error (nothing counted). ### `POST /out` — authenticated submission ```json { "trace_id": "…", "auth_user": "x@example.com", "mail_from": "x@example.com", "recipient": "y@example.com" } ``` 1. **Sender outgoing +1** on A(`auth_user`) / D(its domain) / O(its org) — only if `auth_user`'s domain is in the CDB. 2. **Recipient incoming +1** on A / D / O of the recipient's domain row — only if that domain is in the CDB (any Migadu-hosted domain, any org). That mail never passes `/in`. Example: `x@example.com` → `y@example.com`, `s@gmail.com` = two `/out` calls. Call 1: out A x +1, D example.com +1, O +1; in A y +1, D example.com +1, O +1. Call 2: out +1 on each; gmail not in CDB → no incoming. Totals: outgoing +2 per level, incoming for y +1 per level. ### `POST /in` — inbound mail from outside ```json { "trace_id": "…", "mail_from": "s@gmail.com", "recipient": "y@example.com", "junk": false } ``` Recipient incoming +1 on A / D / O if its domain is in the CDB. The sender is never counted. `junk = true` is skipped unless `count_junk = true` (Lua only counted `no action` mail). ### `GET /counters?address=…` or `?domain=…` `[&date=YYYY-MM-DD]` Read-only view (used by `counterctl`), served from the **read client**. `date` is an ISO 8601 calendar date (`YYYY-MM-DD`, interpreted as a UTC day); default = today UTC; a future date is `400`. Keys live 31 days, so older dates return zeros. Exactly one of `address`/`domain` is required. ```json { "query": "me@example.com", "mode": "address", "date": "2026-10-13", "managed": true, "exempt": false, "domain": "example.com", "org_id": 1234, "levels": [ {"level": "A", "id": "me@example.com", "in": 12, "in_limit": 500, "out": 3, "out_limit": 200}, {"level": "D", "id": "example.com", "in": 40, "in_limit": 1000, "out": 9, "out_limit": 400}, {"level": "O", "id": "1234", "in": 120, "in_limit": 5000, "out": 30, "out_limit": 1000} ] } ``` `in_limit`/`out_limit` are `null` when unlimited. `mode: "domain"` returns the D and O rows only. `managed: false` (domain not in the CDB) still returns the raw counters with `null` limits — Lua-era keys may exist. Values are read with one `MGET` of the ≤ 6 keys. ### Shared rules - Addresses are trimmed and lowercased; validation = one `@`, non-empty parts, ≤ 254 chars. - **Exempt list** (default `support@migadu.com`, `contact@migadu.com`): if `auth_user`, `mail_from` or `recipient` is exempt, nothing is counted for that request (Lua: mail to/from support never costs quota). - `trace_id` (mizu's `X-Trace-Id`, shared by all recipients of one message) is logged for correlation only; mailqueuer's internal per-recipient queue id is **not** sent. No Redis idempotency keys: mailqueuer already guarantees at-most-once per (trace, recipient) publish; the residual case (mailqueuer restart + mizu retry) also double-delivers the mail, so the count stays truthful. - `O` and `D` limits come from the domain row that triggered the increment (sender's row for outgoing, recipient's row for incoming) — same caveat as Lua if domains of one org carry different `O/*` values. ## HTTP validation layer (mailqueuer → migadu_counters) Everything mailqueuer sends passes this layer **before** any CDB or Redis access; a request that fails any step is answered with a JSON error `{"error":"…"}` and nothing is counted. Two parts: **1. Middleware chain** (`internal/server/middleware.go`, applied in this order to `/in`, `/out`; `/counters` gets the same chain with method GET and no body steps): | # | Step | Rule | Failure | |---|------|------|---------| | 1 | recover | panic in a handler → log with stack, keep the daemon alive | `500` | | 2 | method | `POST` only (`GET` for `/counters`) | `405` + `Allow` header | | 3 | body size | `http.MaxBytesReader` 64 KB | `413` | | 4 | bearer token | `Authorization: Bearer `, constant-time compare with `auth_token` | `401` + `WWW-Authenticate: Bearer` | | 5 | source IP | optional `allowed_cidrs` (config); matched on `RemoteAddr` only — `X-Forwarded-For` is never trusted (no proxy in front) | `403` | | 6 | content type | `Content-Type: application/json` (parameters such as `charset` ignored) | `415` | | 7 | JSON decode | exactly one JSON object; syntax errors rejected; **unknown fields ignored** so mailqueuer can add fields without breaking counting | `400` | | 8 | request validation | per-route field rules below | `400` | | 9 | timeout | `http.TimeoutHandler` (`handler_timeout`, default 4 s) around the counting work | `503` | Every rejection is logged at `warn` with route, remote address, status, reason and `trace_id` (if parsable) and counted in `requests_total{route,status}` — a burst of `400`/`401` from mailqueuer means contract drift or a token mismatch and should alert. **2. Request validators** (`internal/api/validate.go`, pure functions, table-tested): | Field | `/out` | `/in` | Rule | |-------|--------|-------|------| | `trace_id` | optional | optional | ≤ 128 chars, printable ASCII; control characters/newlines stripped before it reaches any log line (log-injection guard) | | `auth_user` | **required** | — (ignored) | e-mail syntax | | `recipient` | **required** | **required** | e-mail syntax | | `mail_from` | optional | optional (`""` = bounce sender `<>`) | e-mail syntax when non-empty | | `junk` | — (ignored) | optional, default `false` | boolean | E-mail syntax = trimmed, lowercased, ≤ 254 chars, exactly one `@`, non-empty local part and domain, no whitespace/control characters; the domain is ASCII (punycode accepted). Deliberately permissive: the CDB lookup decides whether an address is counted, the validator only guards against garbage and log injection. Errors name the field: `{"error":"recipient: missing"}`, `{"error":"auth_user: invalid e-mail address"}`. Config: `allowed_cidrs = ["10.0.0.0/8"]` (optional; empty = any source), `handler_timeout` (see daemon section). The response codes mailqueuer may see are therefore: `200` counted/skipped, `4xx` rejected by this layer, `503` Redis unavailable or timeout — it ignores all of them (fire-and-forget), they exist for operators and tests. ## Redis (`internal/store`) Config has a **write** side and a **read** side: - **Sentinel mode (production)**: `sentinels`, `master_name`, `password` (+ optional `sentinel_password`). Write = `redis.NewFailoverClient` (follows failover). Read = `NewFailoverClient{ReplicaOnly: true}`; go-redis picks a random replica and falls back to the master when none is known (`sentinel.go:526`). - **Static mode (dev/tests, when `sentinels` is empty)**: `write_addr` + `read_addrs` (round-robin; defaults to `write_addr`). - Both pooled (`pool_size`); one `timeout_ms` for dial/read/write. `db` must be `0` (validated, startup error otherwise). Counting = one `TxPipeline` (MULTI/EXEC) per request: `INCRBY key 1` + `EXPIRE key 2678400` for every affected key (≤ 6 keys). EXEC returns the new values, so no read-after-write and replica lag is irrelevant. Increments are atomic (fixes the Lua GET-then-INCR race); counting continues past the limit. **Limit crossing**: for each incremented key with a configured limit `L`, notify iff `new − amount < L ≤ new` (with amount 1: `new == L`). Exactly one increment can cross, across all workers/instances, with no marker key. After a web-app reset (DEL) the counter crosses again and notifies again. Accepted gap (same as Lua): a limit lowered below the current count mid-day produces no crossing. Redis errors → `503`, log, `redis_errors_total{op}`. ## Notification (`internal/notify`) - One URL per level in `[notify]`: `org_url` (O), `address_url` (A), `domain_url` (D). Empty URL = that level is counted and `limit_reached_total` is incremented, but no HTTP call (useful while the web app's `D` branch is a TODO). Basic auth and timeout are shared. - `GET ` with template vars `$LEVEL $ID $FLOW $PERIOD(=d) $ADDRESS $COUNT $LIMIT`, each `url.QueryEscape`d. `ADDRESS` = the address that triggered it (sender for outgoing, recipient for incoming). `COUNT` = new value. - Runs in a background goroutine (semaphore of 8 to not flood the web app): up to 5 attempts, exponential backoff 1 s → 16 s. Final failure → error log + `notifications_total{level,result="failed"}` (alertable). Shutdown waits ≤ 10 s for in-flight notifications. ## CLI: `counterctl` ``` counterctl --address me@dimitrijedimitrijevic.com # today (UTC) counterctl --address me@dimitrijedimitrijevic.com --date 2026-10-12 counterctl --domain dimitrijedimitrijevic.com --json ``` - Flags (Go `flag`, so `-x` and `--x` both work): `--address` | `--domain` (exactly one), `--date YYYY-MM-DD` (ISO 8601 calendar date; default today UTC; parsed with `time.Parse("2006-01-02")`; future date = error; older than 31 days = warning on stderr that keys have expired), `--json` (raw API response), `--config PATH`, `--url` (override), `--version`. - Talks to the running service's `GET /counters` instead of Redis directly — one code path for key building and limit resolution, no Redis/CDB access needed from the operator's shell, same approach as `rcptctl`. Config discovery mirrors the service and `rcptctl` (`cmd/rcptctl/main.go:65-75`): `--config`, else `/etc/migadu_counters/config.toml`, `/usr/local/etc/migadu_counters/config.toml`; reads `addr` and `auth_token` only (`MIGADU_COUNTERS_AUTH_TOKEN` overrides); `addr` like `:8095` becomes `http://localhost:8095` (`rcptctl/api.go:52-60`); warns when the host is not local (token over plaintext). - Output (`text/tabwriter`), limits shown as `-` when unlimited, `!` appended when value ≥ limit: ``` me@dimitrijedimitrijevic.com date 2026-10-13 (UTC) domain dimitrijedimitrijevic.com org 1234 LEVEL ID IN LIMIT OUT LIMIT A me@dimitrijedimitrijevic.com 12 500 3 200 D dimitrijedimitrijevic.com 40 1000 9 400 O 1234 120 5000 30 1000 ``` `managed: false` prints a `domain is not in rspamd_domains_db (not counted)` note after the table; `exempt: true` prints `address is exempt from counting`. - Exit codes: `0` success, `1` usage error / bad date / service unreachable / non-200 (message on stderr). ## Counting transaction log (`internal/txlog`) A second logger, separate from the slog application log, that writes the Lua plugin's step-by-step lines for **every** counting transaction (the Lua `infox` lines from `migadu_counters.lua:217/235/266/268/271/283`), so existing grep habits keep working. rspamd's task id `` is replaced by `` (shared with mizu/mailqueuer logs via `X-Trace-Id`), and the route is added. `breach=` becomes `reached=` because the meaning changed (Lua: already over → rejected; here: this increment crossed the limit). Text format (default), timestamps UTC with milliseconds, one block per request: ``` 2026-10-06 01:12:06.412 out; request auth_user=noah@wearemiraclesnow.com recipient=y@example.org mail_from=noah@wearemiraclesnow.com 2026-10-06 01:12:06.412 out; local domain: wearemiraclesnow.com account_id=116681 2026-10-06 01:12:06.412 out; domain: wearemiraclesnow.com limits_map={A/i=500, A/o=100, D/i=10000, D/o=2200, O/i=10000, O/o=2200} 2026-10-06 01:12:06.412 out; determined limit (level=[A] flow=[o] key=[cnt/A/noah@wearemiraclesnow.com/d/2026-10-06/o]) => 100 2026-10-06 01:12:06.412 out; determined limit (level=[D] flow=[o] key=[cnt/D/wearemiraclesnow.com/d/2026-10-06/o]) => 2200 2026-10-06 01:12:06.412 out; determined limit (level=[O] flow=[o] key=[cnt/O/116681/d/2026-10-06/o]) => 2200 2026-10-06 01:12:06.412 out; recipient domain not in domains db: example.org (not counted) 2026-10-06 01:12:06.414 out; tracked address counters for noah@wearemiraclesnow.com => count=12 limit=100 reached=false 2026-10-06 01:12:06.414 out; tracked domain counters for wearemiraclesnow.com => count=1830 limit=2200 reached=false 2026-10-06 01:12:06.414 out; tracked account counters for 116681 => count=2200 limit=2200 reached=true 2026-10-06 01:12:06.414 out; limit reached (level=[O] id=[116681] flow=[o] by=[noah@wearemiraclesnow.com]) => notifying org_url 2026-10-06 01:12:06.903 out; sent limit exceeded notification to url "https://admin.migadu.com/api/limits?level=O&id=116681&flow=o&period=d&by=noah%40wearemiraclesnow.com&count=2200" ``` - A local recipient produces the same `local domain / limits_map / determined limit / tracked …` block for `flow=[i]` under the recipient's domain row. `/in` requests log the recipient block only. - `determined limit … => unlimited` when no limit is configured. Skips are logged as one line: `skipped: exempt address `, `skipped: junk`, `recipient domain not in domains db: (not counted)`, `sender domain not in domains db: (not counted)`. - Notification outcome lines come from the background sender, tagged with the same ``: `sent limit exceeded notification to url "…"` or `failed limit exceeded notification to url "…" after 5 attempts: `. - Redis failure: `redis error: (nothing counted)`. - `format = "json"` switches the same lines to one JSON object per line (`ts, trace_id, route, event, level, flow, key, id, count, limit, reached, domain, org_id, limits, url, error`) for later ingestion. - The "tracked" lines are written after the pipeline EXEC (counts are the returned values); the earlier lines are written before the Redis call, so a Redis failure still leaves the lookup trail in the log. - Writer: the same `LogWriter` type as the app log (mutex, unbuffered for `tail -f`, `Reopen` on SIGHUP). Config: `[logging] counting_output` (default: same destination as `output`; `"off"` disables), `counting_format` (`text` | `json`). The per-request details live only here; the app log keeps startup, reloads, warnings and errors, to avoid duplication. ## Process model: standalone daemon `cmd/migadu_counters` is a long-running **foreground** process: - Two `http.Server`s: the API (`/in`, `/out`, `/counters`, bearer auth, `http.TimeoutHandler`) on `addr`, and the admin server (`/health`, `/metrics`, no auth) on `admin_addr`. Server timeouts as in rcptd: `read_header_timeout=5`, `write_timeout=30`, `idle_timeout=60`, `handler_timeout=4` (seconds). - Startup: load config → logger → open CDB (fatal if unreadable) → create Redis clients + PING the write side (warn, not fatal: the service must come up while Redis is briefly away) → start fsnotify watcher → listen. - Signals: `SIGHUP` → reopen both log files (app + counting) + reload CDB (`atomic.Pointer` swap, old file closed after 30 s); `SIGTERM`/`SIGINT` → stop accepting, drain in-flight requests (≤ 10 s), wait for background notifications (≤ 10 s), close Redis clients, exit 0. - Supervision on FreeBSD mirrors rcptd (`ansible-freebsd3/roles/rcptd/templates/rcptd.rc.j2`): an rc.d script runs the binary under `/usr/sbin/daemon -f -H -p -o `; daemon(8) owns the pidfile, restarts on exit and captures stderr; `service migadu_counters reload` = `kill -HUP $(cat pidfile)` (the child's pid). Therefore **no `daemon`/`pid_file` options** (rcptd's config template warns that writing the same pidfile from the process breaks the supervisor). Logging defaults to stderr for daemon(8) `-o`; a file path is supported with `Reopen` on SIGHUP for newsyslog. - `[logging]` is rcptd's `LoggingConfig` verbatim (`output` = stderr|stdout|path, `format` = console|json, `level`); copy `rcptd/internal/config/logger.go` (self-contained). - Binaries carry `version/commit/date` via ldflags (`-version` flag); Makefile mirrors rcptd's: `build`, `test`, `freebsd` (CGO_ENABLED=0, `migadu_counters-freebsd-amd64`, `counterctl-freebsd-amd64`), `build-linux`. ## Layout ``` migadu_counters/ ├── go.mod # module migadu_counters (exists) ├── Makefile # build / test / freebsd / build-linux, ldflags version info ├── cmd/migadu_counters/main.go # config, logger, CDB load/reload, redis clients, HTTP + admin servers, signals ├── cmd/counterctl/main.go # CLI: flags, config discovery, GET /counters, table/json output ├── internal/api/types.go # JSON types shared by the server (/in, /out, /counters) and counterctl ├── internal/api/validate.go # per-route request validators, e-mail syntax, trace_id sanitizing ├── internal/config/config.go # TOML structs, defaults, validation (sentinel vs static, db==0, urls) ├── internal/config/logger.go # LoggingConfig + LogWriter.Reopen (from rcptd), counting_* fields ├── internal/txlog/txlog.go # counting transaction logger: text (Lua-like) / json, per-request block ├── internal/domains/domains.go # CDB open/lookup, ParseRecord/ParseLimits, Limit(level, flow, local) ├── internal/watcher/watcher.go # fsnotify reload of the CDB (rcptd table mode, trimmed) ├── internal/store/redis.go # write/read clients, Incr pipeline, Get for /counters ├── internal/counting/counting.go # /in, /out request → increments (level, id, flow, limit), exempt/junk/unknown ├── internal/notify/notify.go # per-level URL, templating/escaping, background sender with retries ├── internal/server/middleware.go # recover, method, body size, bearer token, allowed_cidrs, content-type, decode, timeout ├── internal/server/server.go # /in, /out, /counters handlers; admin /health, /metrics ├── internal/metrics/metrics.go # prometheus definitions ├── config.toml.example └── README.md # API contract for mailqueuer, key format, counting rules, ops ``` Example config (`/usr/local/etc/migadu_counters/config.toml`; discovery order `-config`, `/etc/migadu_counters/config.toml`, `/usr/local/etc/migadu_counters/config.toml`): ```toml addr = ":5555" # was :8095 before review round 1 admin_addr = ":9095" auth_token = "…" # or MIGADU_COUNTERS_AUTH_TOKEN; same value in mailqueuer's config allowed_cidrs = [] # optional source-IP allowlist for /in, /out, /counters, e.g. ["10.0.0.0/8"] domains_cdb = "/usr/local/etc/rspamd/db/rspamd_domains_db" exempt = ["support@migadu.com", "contact@migadu.com"] count_junk = false read_header_timeout = 5 write_timeout = 30 idle_timeout = 60 handler_timeout = 4 [logging] output = "stderr" # daemon(8) -o captures it; or a file path (reopened on SIGHUP) format = "console" # or "json" level = "info" counting_output = "/var/log/migadu_counters/counting.log" # default: same as output; "off" disables counting_format = "text" # Lua-like lines (default) or "json" [redis] db = 0 # must stay 0: migadu_web reads message counters from DB 0 password = "…" # or MIGADU_COUNTERS_REDIS_PASSWORD pool_size = 32 timeout_ms = 500 # Sentinel mode (production): writes → master, reads → replicas master_name = "mymaster" sentinels = ["10.0.0.10:26379", "10.0.0.11:26379", "10.0.0.12:26379"] sentinel_password = "" # Static mode (used when sentinels is empty) # write_addr = "127.0.0.1:6379" # read_addrs = ["127.0.0.1:6379"] [notify] # One URL per level; leave empty to disable notifications for that level. org_url = "https://admin.migadu.com/api/limits?level=$LEVEL&id=$ID&flow=$FLOW&period=$PERIOD&by=$ADDRESS&count=$COUNT" address_url = "https://admin.migadu.com/api/limits?level=$LEVEL&id=$ID&flow=$FLOW&period=$PERIOD&by=$ADDRESS&count=$COUNT" domain_url = "" user = "…" password = "…" # or MIGADU_COUNTERS_NOTIFY_PASSWORD timeout_seconds = 5 ``` Metrics: `migadu_counters_requests_total{route,status}`, `_request_duration_seconds{route}`, `_increments_total{level,flow}`, `_limit_reached_total{level,flow}`, `_notifications_total{level,result}`, `_redis_errors_total{op}`, `_cdb_domains` (gauge), `_cdb_reloads_total{result}`. `/health`: 200 when the CDB is readable and the write client answers PING within 200 ms; else 503 with reason. ## Behaviour changes vs the Lua plugin (intentional) - Separate `/in` and `/out` routes, one request per recipient, instead of rspamd's `MIGADU_FLOW` symbol. - Outgoing counted **per recipient** (Lua: once per message, skipped when all recipients were same-domain). Existing `*/o` limits therefore become stricter — a 50-recipient message costs 50; limits may need re-tuning before the web app's `outgoing_limit_exceeded!` is relied upon. - Local recipients of authenticated mail get incoming counted via `/out`. - Domain (`D`) level counted; notified only when `domain_url` is set. - Atomic increments; counters don't stop at the limit; no reject/soft-reject anywhere. - Notification on the exact crossing increment; re-fires after a web-app reset. - UTC day keys (matches the web app; Lua used server local time). Malformed CDB rows are skipped, not fatal. ## Deployment prerequisites (not part of this repo) - An Ansible role in `ansible-freebsd3` cloned from `roles/rcptd`: user/group, `config.toml.j2`, the rc.d script (daemon(8) with pidfile `/var/run/migadu_counters/migadu_counters.pid`, log `/var/log/migadu_counters/…`), newsyslog entry, binary `migadu_counters-freebsd-amd64` + `counterctl-freebsd-amd64` in `/usr/local/bin`. - The CDB must exist on the host running the service (the agent delivers it to rspamd hosts today; stats-merger pulls the text variant) — same file, so a copy/pull job is enough. - Firewall: mailqueuer hosts → service port; service → Sentinel hosts :26379/:6379 and admin.migadu.com:443. - Secrets: bearer token shared with mailqueuer, Redis password, web app basic-auth credentials. ## Implementation order 0. Copy this plan into the repo as `plans/2026-10-13-migadu-counters.md` (requested; kept as the design record). 1. `go.mod` deps, `Makefile`, `config.toml.example`, `internal/config` (structs, defaults, validation, logger from rcptd). 2. `internal/domains` (+ tests with a temp CDB), `internal/watcher` (trimmed from rcptd). 3. `internal/store` (sentinel/static clients, Incr pipeline, MGET) + miniredis tests, incl. crossing detection. 4. `internal/counting` (request → increments, exempt/junk/unknown) + tests for the `/out` and `/in` examples. 5. `internal/txlog` (text + json, golden tests), `internal/notify` (per-level URLs, retries) + httptest tests. 6. `internal/api` types, `internal/server` (`/in`, `/out`, `/counters`, admin), `internal/metrics`. 7. `cmd/migadu_counters/main.go` (wiring, signals, shutdown), `cmd/counterctl/main.go` (+ tests). 8. `README.md` (API contract for mailqueuer, key format, counting rules, ops), manual verification below. ## Verification 1. `go vet ./... && go test -race ./...` - `domains`: parse/override/unlimited/malformed rows via a temp CDB (`cdb.Create`); unknown domain. - `counting`: the `/out` example (two calls → out +2 per level, y in +1 per level); external-only recipient; sender domain not in CDB (only recipient counted); exempt sender/recipient → nothing; `/in` local/unknown/junk. - `store` + miniredis: key format, TTL set, DB 0, crossing fires exactly once (limit 3: 3rd increment `reached`, 4th not; after `DEL` the next crossing fires again); concurrent increments → one crossing; read-side round-robin / fallback; Redis down → 503. - `notify`: each level hits its own URL, empty URL skips, all vars escaped (`@` → `%40`), retries then failure metric. - `txlog`: golden-file tests with an injected clock for the `/out` example above (local + external recipient, one level reached), an `/in` request, each skip reason, a Redis failure, and the `json` format; `Reopen` after the file is moved writes to the new path. - `api/validate`: table tests per field and route (missing, empty, too long, two `@`, whitespace, control chars, uppercase → lowercased, punycode accepted, `mail_from` empty on `/in`, `trace_id` sanitized). - `server/middleware`: one test per step in chain order — 405, 413 (65 KB body), 401 (missing/wrong/`Basic`), 403 (`allowed_cidrs` miss) vs pass, 415, 400 (syntax error, two objects, unknown field *accepted*), 503 (handler timeout with a blocked Redis), 500 (panic recovered, daemon still serving); each rejection leaves Redis untouched (miniredis key count unchanged) and increments `requests_total{status}`. - `server`: 401, 400, 200 with `counted` payload; `/counters` with address/domain, default date = today UTC, explicit `date`, future date → 400, malformed date → 400, unmanaged domain → `managed:false` + null limits; notify targets are per-level `httptest.Server`s. - `counterctl`: date parsing/validation (valid ISO, malformed, future, > 31 days warning), exactly-one-of `--address`/`--domain`, `addr` → URL normalization, table rendering against a fixed API response (`httptest.Server`), `--json` passthrough, exit codes. 2. Manual: local Redis in static mode, CDB built from a sample line (`cdb -m -c`, or the test helper), `go run ./cmd/migadu_counters -config config.toml.example` with `org_url`/`address_url` pointed at a local listener. `curl -XPOST localhost:8095/out -H 'Authorization: Bearer …' -d '{…}'` repeatedly; check `redis-cli -n 0 KEYS 'cnt/*'`, `GET /counters?address=…`, exactly one notification per level on crossing, `redis-cli DEL` the A key → crossing notifies again; edit the CDB + `kill -HUP` → limits reload. `go run ./cmd/counterctl --config config.toml.example --address x@example.com` shows the same numbers as `redis-cli`; `--date ` shows zeros/older data; `--date 2099-01-01` is rejected. `tail -f counting.log` while sending the curl requests shows the full block per request, and `grep '' counting.log` returns one transaction. Daemon behaviour: `kill -TERM` exits 0 after draining; with `logging.output`/`counting_output` set to files, `kill -HUP` reopens them (rotate with `mv` first and confirm new lines land in the new files); `make freebsd` produces the two static binaries. Optionally a local Sentinel (1 master + 1 replica) to confirm reads hit the replica.