migadu_counters

Real-time mail counting for Migadu. mailqueuer tells it about every ingested recipient; it increments per-address / per-domain / per-organisation daily counters in Redis (the keys the web app already charts) and, when an increment reaches a configured limit, notifies the web app, which decides what happens to the customer. It replaces the rspamd Lua plugin migadu_counters.lua and the ledger-derived quota in stats-merger. It returns no verdicts and enforces nothing itself.

Design record: plans/2026-10-13-migadu-counters.md.

 sender ──SMTP──▶ mizu ──POST /ingest──▶ mailqueuer
                                              │ per recipient, after a successful publish:
                                              │ fire-and-forget HTTP call, reply ignored
                                              ▼
            POST /out  {"trace_id","auth_user","mail_from","recipient"}   ← X-Auth-User set
            POST /in   {"trace_id","mail_from","recipient","junk"}        ← X-Auth-User empty
            Authorization: Bearer <token>
                                              ▼
                               ┌─────────────────────────────────────┐
         rspamd_domains_db ───▶│ migadu_counters   (:5555, admin :9095)│
         (domain → org|limits) │ validate → 202 → queue → workers:     │
                               │ lookup domain → MULTI/INCR → notify   │──▶ GET admin.migadu.com/api/limits
                               └──────────────────┬──────────────────┘    (on the increment that crosses a limit)
                                                  ▼
                                 Redis Sentinel: writes → master, reads → replicas
                                 DB 0: cnt/<A|D|O>/<id>/d/<YYYY-MM-DD>/<i|o>  (TTL 31 d)

 counterctl --address x@y [--date YYYY-MM-DD] ──GET /counters──▶ migadu_counters ──▶ replicas

API (for mailqueuer)

Every request: Authorization: Bearer <auth_token> (same value in mailqueuer's config; the daemon refuses to start without one), Content-Type: application/json, body ≤ 64 KB, one JSON object. Unknown fields are ignored. mailqueuer never waits for or acts on the reply.

Counting is decoupled from the request: /in and /out validate the body, enqueue a job and answer 202 at once; a worker pool does the CDB lookup, the Redis transaction and the notification under its own time budget, so a client that hangs up or times out can no longer cancel a count. The counted values appear in the counting log and in GET /counters, not in the reply.

POST /out	{"trace_id":"…","auth_user":"x@example.com","mail_from":"x@example.com","recipient":"y@example.com"} — one call per recipient of an authenticated submission. Sender A/D/O outgoing +1 (if the sender's domain is hosted), recipient A/D/O incoming +1 (if the recipient's domain is hosted). auth_user and recipient are validated independently: whichever side is a usable address is counted, the other is logged as not usable; 400 only when neither is. A support/token login user@dom@SUFFIX counts under user@dom (the suffix is logged).
POST /in	{"trace_id":"…","mail_from":"s@gmail.com","recipient":"y@example.com","junk":false} — one call per recipient of inbound mail. Recipient A/D/O incoming +1 if hosted. junk:true is skipped unless count_junk = true. mail_from may be "" (bounce) or anything mizu accepted — it only feeds the exempt check and never rejects the request.
GET /counters?address=… or ?domain=… [&date=YYYY-MM-DD]	Today's (UTC) or a given day's values and limits, read from the replicas.

Responses: 202 {"accepted":true} (validated and queued; skips such as exempt/junk/unknown domain are decided by the worker and appear only in the counting log), 4xx {"error":"…"} from the validation layer (405 method, 413 size, 401 token, 403 source IP, 415 content type, 400 JSON / required fields / e-mail syntax), 503 {"error":"queue full"} when the counting queue (queue_size) is full — the drop is counted in dropped_total{route} and logged — or 503 on a handler timeout of the synchronous part.

Rules: addresses are trimmed and lowercased, Unicode domains are converted to punycode (the CDB key), and a plus-address tag is dropped so b+tag@… counts against the mailbox b@… (the counter the web app reads); mail to or from an exempt address (support@migadu.com, contact@migadu.com) is never counted; a domain that is not in rspamd_domains_db is not hosted here and not counted; the sender of /in mail is never counted; there is no reject/soft-reject — counting continues past the limit.

Limits and keys

Limits come from the rspamd domains CDB (tinycdb, built by the agent): example.com → 1049|O/i=200,O/o=20,D/i=200,D/o=20,A/i=200,A/o=20,info@/o=5. <local>@/<flow> overrides the A/<flow> default for one mailbox; a missing key means unlimited. The file is reloaded on SIGHUP and automatically when it changes on disk (atomic mv is detected); a file that cannot be read or is empty is rejected and the current table kept. A row that exists but cannot be parsed (non-numeric org id) is treated as "not hosted" for that domain and warned about in the app log; the other side of the request is still counted. Always replace the file with mv (as the agent's cdb -c + mv do) — rewriting it in place truncates the inode the daemon holds open and breaks lookups until the next reload.

Redis keys are exactly the web app's (migadu_web core/redis_sentinel.ex): cnt/<A|D|O>/<id>/d/<YYYY-MM-DD>/<i|o> in DB 0, UTC day, TTL 31 days, INCR+EXPIRE in one MULTI/EXEC. The web app may DEL a day key to reset a customer; the next crossing notifies again.

Redis failures are handled per outcome: a key whose own INCR fails (it holds a non-number) is logged redis error on key … while the other keys are counted and checked for crossings; when the EXEC reply is lost after the write (timeout, reset) the keys are re-read once from the master and the block says redis outcome unknown … re-read from master; a transaction Redis rejected as a whole (READONLY or EXECABORT during a failover) applied nothing and is retried once after 200 ms; a definite failure (connection refused, or a rejection that repeats) logs redis error: … (nothing counted).

The CDB is pulled from the agent host by contrib/pull-domains.sh (see contrib/README.md): rsync, compare, verify, atomic mv, every 15 s.

A limit is "reached" by the increment that moves the counter from below the limit to at or above it (old < limit ≤ new). Because INCR is atomic exactly one increment crosses, so the web app is notified once per level/id/flow/day — GET <org_url|address_url|domain_url> with $LEVEL $ID $FLOW $PERIOD(=d) $ADDRESS $COUNT $LIMIT substituted (URL-escaped), basic auth, in the background. An empty URL disables notifications for that level.

Early warning. The same rule at warn_percent of the limit (default 80, rounded up, 0 = off) calls warn_org_url / warn_address_url / warn_domain_url, which also accept $PERCENT. No warning URL means the feature is off. A limit too small to have a threshold below it (1 or 2 at 80%) has no warning. The warning URLs must not use the breach URL's path, because the web app treats every call there as a breach; the daemon refuses to start if they do. The web app endpoint is GET /api/limits/warning.

No repeats. A breach or warning is sent once:

• A confirmed crossing writes a marker, ntf/<level>/<id>/d/<day>/<flow>/<limit|warn> (TTL 48 h). After a lost Redis reply a notification goes out only if SET NX on that marker succeeds, so a counter that already notified stays silent. If Redis cannot answer, nothing is sent and _redis_errors_total{op="marker"} rises.
• An attempt is retried only when the request never left or the web app answered 429 or 5xx, up to 5 attempts with exponential backoff. A request that was sent but got no answer may have been acted on, so it is not retried; it is logged as outcome unknown.
• The web app may DEL a day key to reset a customer; the next confirmed crossing notifies again, and the web app's once-per-day guard decides whether an email goes out.

Delivery is therefore at most once. A notification that fails, or is in flight when the daemon stops past its 30 s deadline, is not sent later that day; look for failed limit in the counting log and _notifications_total{result="failed"}.

Counting transaction log

Every request writes one block in the shape of the old Lua plugin's log lines (rspamd's task id replaced by the trace id) to logging.counting_output:

2026-10-06 01:12:06.412 <ad8525> out; request auth_user=noah@example.com recipient=y@example.org mail_from=noah@example.com
2026-10-06 01:12:06.412 <ad8525> out; local domain: example.com account_id=116681
2026-10-06 01:12:06.412 <ad8525> out; domain: example.com limits_map={A/i=500, A/o=100, D/i=10000, D/o=2200, O/i=10000, O/o=2200}
2026-10-06 01:12:06.412 <ad8525> out; determined limit (level=[A] flow=[o] key=[cnt/A/noah@example.com/d/2026-10-06/o]) => 100
2026-10-06 01:12:06.412 <ad8525> out; recipient domain not in domains db: example.org (not counted)
2026-10-06 01:12:06.414 <ad8525> out; tracked address counters for noah@example.com => count=12 limit=100 reached=false
2026-10-06 01:12:06.414 <ad8525> out; tracked account counters for 116681 => count=2200 limit=2200 reached=true
2026-10-06 01:12:06.414 <ad8525> out; limit reached (level=[O] id=[116681] flow=[o] by=[noah@example.com]) => notifying org_url
2026-10-06 01:12:06.903 <ad8525> out; sent limit exceeded notification to url "https://admin.migadu.com/api/limits?…"

grep '<trace_id>' returns one transaction; grep 'skipped:' every request that counted nothing; grep 'not usable' every side that failed validation (sender address not usable: nobody (invalid e-mail address, not counted)); grep 'dropped:' every request refused by a full queue; grep 'rejected:' every /in or /out body refused by validation. A user@dom@SUFFIX login shows as … token_login=true on the request line; the suffix may be a credential and is never logged. counting_format = "json" emits the same events as one JSON object per line. Domains with thousands of per-mailbox overrides print the base keys plus the address's own overrides and a count of the rest.

Running

migadu_counters -config /usr/local/etc/migadu_counters/config.toml   # default paths: /etc/…, /usr/local/etc/…
migadu_counters -version

Foreground process; two listeners: addr (API, default :5555) and admin_addr (/health, /metrics, no auth). Signals: SIGHUP reopens both log files and reloads the CDB; SIGTERM/SIGINT stop accepting, drain the counting queue, wait for in-flight notifications and their retries (≤ 30 s) and exit 0. If the deadline passes first, the remaining jobs are lost and the exit code is 1. On FreeBSD run it like rcptd: an rc.d script with /usr/sbin/daemon -f -H -p <pidfile> -o <logfile>; service migadu_counters reload sends SIGHUP. There is deliberately no self-daemonising or pidfile option (daemon(8) owns both). Startup fails without a readable, non-empty CDB; an unreachable Redis is only warned about (/health answers 503 until it is back).

See config.toml.example for every option. auth_token is mandatory, the example's "…" placeholders are refused, admin_addr defaults to 127.0.0.1:9095, redis.password is required with Sentinel (unless redis.password_optional = true) and unknown keys (typos such as auth-token) are fatal at startup. Secrets may come from the environment: MIGADU_COUNTERS_AUTH_TOKEN, MIGADU_COUNTERS_REDIS_PASSWORD, MIGADU_COUNTERS_NOTIFY_PASSWORD. redis.db must stay 0. Production uses Sentinel (sentinels, master_name = "mymaster"): writes go to the master, reads to a replica (falling back to the master when none is known); without sentinels the static write_addr/read_addrs are used.

Metrics (/metrics on the admin port): migadu_counters_requests_total{route,status}, _request_duration_seconds{route}, _increments_total{level,flow}, _limit_reached_total{level,flow}, _warning_reached_total{level,flow}, _notifications_total{kind,level,result} (kind: limit, warning), _redis_errors_total{op} (incr, incr_unknown, mget, ping, marker), _dropped_total{route}, _queue_depth, _queue_wait_seconds, _cdb_domains, _cdb_reloads_total{result}.

counterctl

counterctl --address me@example.com                 # today, UTC
counterctl --address me@example.com --date 2026-10-12
counterctl --address me@example.com --out            # outgoing columns only (--in: incoming)
counterctl --domain example.com --json

Reads addr/auth_token from the daemon's config (--config, default paths, MIGADU_COUNTERS_AUTH_TOKEN overrides the token; --url overrides the address) and calls GET /counters. Output:

me@example.com   date 2026-10-13 (UTC)   domain example.com   org 1234
LEVEL  ID              IN   LIMIT  OUT   LIMIT
A      me@example.com  12   500    200!  200
D      example.com     40   1000   9     -
O      1234            120  5000   30    1000

! marks a value at or above its limit, - an unlimited level. --in / --out restrict the table to one flow (LEVEL ID IN LIMIT or LEVEL ID OUT LIMIT; both or neither show the full table; --json ignores them). --date takes an ISO 8601 calendar date (YYYY-MM-DD); future dates are refused and dates older than 31 days are warned about (keys have expired). Exit 0 on success, 1 on any error.

Development

make build      # bin/migadu_counters, bin/counterctl
make test       # go test -race ./...  (tests use miniredis and test_files/rspamd_domains_db.cdb)
make vet        # go vet + gofmt
make test-contrib  # shell test for contrib/pull-domains.sh (needs tinycdb cdb + rsync)
make freebsd    # static FreeBSD amd64 binaries for the Ansible role

test_files/ is gitignored because the CDB holds real customer domains, so on a fresh clone the tests fail until you copy rspamd_domains_db from an agent host (/usr/local/etc/agent/build/rspamd/) to test_files/rspamd_domains_db.cdb.

Manual check with a local Redis: redis-server --port 6379, a config in static mode (write_addr = "127.0.0.1:6379") pointing domains_cdb at test_files/rspamd_domains_db.cdb, then

curl -s -XPOST localhost:5555/out -H 'Authorization: Bearer <token>' -H 'Content-Type: application/json' \
  -d '{"trace_id":"t1","auth_user":"a@yagolivas.com","recipient":"x@gmail.com"}'   # -> {"accepted":true}
redis-cli -n 0 KEYS 'cnt/*'
go run ./cmd/counterctl -config config.toml --address a@yagolivas.com --out