counters/todo.md

TODO — migadu_counters

State as of 2026-10-15. Review findings refer to review/README.md; the proof tests live in review/_proof_tests/ and should become regression tests as each item is fixed (round 1 and items 7–8 are mapped at the end of review/_proof_tests/README.md).

Done

• Review round 1: #1–#6 (queue + 202, lenient validation + IDN, token logins, INCR with per-key results and master re-read, CDB parse cache, mandatory token / port 5555), plus contrib/pull-domains.sh and counterctl --in/--out.
• #7 plus-addressing counts against the mailbox (domains.Mailbox).
• #8 malformed CDB row is "not hosted" + warning (domains.ErrMalformed).

Review round 2 — remaining findings

Suggested order: #10, #15, #9, then the rest in one sweep.

Can stop the daemon or lose counts

• #10 SIGHUP during startup kills the daemon. The signal handler is installed after the Redis ping (up to 2 s). Install signal.Notify before anything slow in cmd/migadu_counters/main.go (run), and queue a reload that arrives before the app is ready. Evidence: review/_proof_tests/a/run/.
• #15 Sentinel mode starts without a Redis password. Every write fails; only /health and metrics show it. Make an empty redis.password fatal in config.Validate when sentinels is set (allow an explicit opt-out such as password_optional = true for dev). Proof: b/internal/config/review_b_test.go, case "sentinel without password".
• Persistent / recoverable counting on Redis outage (deferred from round 1, #4). A Redis stall longer than a few seconds fills the queue and new requests are dropped (503, dropped_total). Options: marker key per transaction for exact lost-reply recovery; a disk-backed queue. Decide whether the drop-and-alert behaviour is acceptable before building anything.

Correctness

• #9 Notification retries can send duplicate customer emails. The web app's mailbox-level handler (core/email.ex:3572-3584) emails on every call; the org handler is guarded once per day. In internal/notify: treat a slow-but-2xx attempt as success (do not retry after the client timeout if the request was sent — or make the retry budget shorter than the web app's handling time), do not keep the concurrency slot while backing off, let Shutdown finish pending retries up to its deadline, and fix the "after N attempts" count in the failure line.
• #12 Override keys are not lowercased. Info@/o=5 would be ignored and the domain default used instead. Lowercase the key in domains.ParseLimits; add a test row. No such keys in today's data.
• #16 counterctl warns one day too early. A date exactly 31 days back is reported as expired. Compare against the key TTL in whole days (now.UTC day minus date > 31). Proof: b/cmd/counterctl/review_b_test.go TestReviewB_31DayWarningOffByOne.
• #18 Work keeps running after a handler timeout. Not reproduced by the reviewer and largely covered by the queue change (the synchronous part only validates). Re-check that http.TimeoutHandler cannot leave a goroutine writing after the 503, then close.

Operability / security hygiene

• #11 Rejected requests leave no trace. 4xx rejections write nothing to the counting log and no trace_id to the app log. Decode trace_id best-effort before rejecting; add a rejected: <status> <reason> line to the counting log and trace_id to the warn line. Proof: b/internal/server/review_b_test.go TestReviewB_RejectLogHasNoTraceID.
• #13 Credentials in the notify URL are written to the log. Redact user:pass@ from URLs in the txlog sent/failed notification lines; warn at startup when a notify URL is plain http:// with basic auth. Proof: b/internal/server/review_b3_test.go.
• #14 Error responses expose internals. 503 bodies carry Redis addresses and file paths; keep the detail in the log and answer a generic message. Default admin_addr to 127.0.0.1:9095 (currently :9095, all interfaces, no auth). Proof: TestReviewB_ErrorLeak.
• #17 A recovered panic is not counted in requests_total. The recoverer sits outside observe; swap the order or record the 500 explicitly. Proof: TestReviewB_PanicNotInMetrics.
• Test gaps (B F13). Watcher tests can occasionally pass on a stale signal — make them wait for the specific reload they triggered. Add a test for handler_timeout vs queue timeout interplay.

Outside this repo (deployment prerequisites)

• mailqueuer client: [counters] section and a fire-and-forget POST /in|/out per recipient inside the publish loop of mailqueuer/internal/handler/ingest.go (after a successful publish; /out when X-Auth-User is set). Reply is 202 {"accepted":true}; counted values only in the counting log and GET /counters.
• Ansible role in ansible-freebsd3, cloned from roles/rcptd: user/group, config.toml.j2 (auth_token is mandatory, port 5555), rc.d script under daemon(8), newsyslog entries for app.log, counting.log, pull.log, templated contrib/pull-domains.sh + contrib/migadu_counters-pull.cron (see contrib/README.md), agent host key acceptance, firewall: mailqueuer hosts → :5555, service → Sentinel :26379/:6379 and admin.migadu.com:443.
• migadu_web /api/limits D branch (domain level); until then notify.domain_url stays empty.
• Limits re-tuning: outgoing is counted per recipient (Lua counted per message), so existing */o limits are stricter. Review before relying on outgoing_limit_exceeded!.
• Retire stats-merger's quota module once this service is the source (only fernrohr reads it).
• Dependencies: golang.org/x/net is pinned to v0.43.0 because the offline module cache had it; run go get -u golang.org/x/net && go mod tidy with network access.
• git init the repo (never asked for so far).