First commit Go counter service

This commit is contained in:
Dimitrije Dimitrijevic
2026-10-15 02:12:18 +02:00
commit c43ff17627
59 changed files with 11205 additions and 0 deletions
+113
View File
@@ -0,0 +1,113 @@
# Code review — 2026-10-14
Review of the first implementation of migadu_counters (see `../plans/2026-10-13-migadu-counters.md`).
Nothing in the code was changed by the review.
## How it was done
- Two independent reviewers (agents) each read the whole repository. Neither saw the other's results.
- **A** — correctness and runtime behaviour: [`reviewer-a-correctness.md`](reviewer-a-correctness.md)
- **B** — security, contracts and operability: [`reviewer-b-security-contracts.md`](reviewer-b-security-contracts.md)
- Each worked in its own scratch copy and proved findings with tests: [`_proof_tests/`](_proof_tests/README.md).
- The coordinating agent then verified every finding (see "Verification" below), merged duplicates and ranked them.
18 distinct issues remain; 4 of them were found by both reviewers.
## Findings
Who found it: A / B / both, with the reviewer's own IDs for traceability.
### High
**1. If mailqueuer disconnects or times out before the reply, the count is lost** — A (F1)
- `internal/server/server.go:156` runs the Redis write with the request context. When the client hangs up, Go cancels
the context and go-redis aborts before sending MULTI.
- Re-run during verification: 20 of 20 delayed requests and 300 of 300 immediate ones were not counted. The most
dangerous finding, because fire-and-forget clients often close early.
- Fix: run the counting under `context.WithoutCancel(r.Context())` with its own time budget.
**2. A bad value in a field that isn't counted rejects the whole request** — both (B F1, A F3)
- `internal/api/validate.go` strictly validates `mail_from` although it only feeds the exempt check; a 400 there means
the recipient is never counted.
- Senders trigger it easily: mizu's SMTP parser turns `"john smith"@x.com` into `john smith@x.com`, mizu runs with
SMTPUTF8 (`info@bücher.de`), address literals and trailing dots pass, and mailqueuer forwards `X-Mail-From` unchecked.
- On `/out`, an unusual external recipient (`x@[192.0.2.1]`, `x@host_name.example.com`) gives 400 and the sender's
outgoing count is lost too.
- Fix: never reject because of `mail_from`; validate sender and recipient sides separately and count whichever is
valid; convert UTF-8 domains to punycode.
### Medium
**3. Support/token logins aren't counted** — A (F2)
- mizu puts the raw SASL login into `X-Auth-User` (`pkg/smtp/auth_session.go:160` → `poster.go:186`), so master logins
arrive as `user@dom@SUFFIX` and fail validation with 400.
- Fix: strip the second `@SUFFIX` and count the base address.
**4. Redis failures with an unclear outcome lose the limit notification** — both (B F3, A F4)
- If Redis applies MULTI/EXEC but the reply times out, or one command inside EXEC fails (a key holding a non-number)
while the others succeed, the service answers 503 and logs "nothing counted" — but the counters moved.
- If that increment reached the limit, no later increment can reach it again: no notification that day.
- Fix: use each command's own result; after a timeout re-read the keys from the master; log "outcome unknown".
**5. Huge CDB records are re-parsed on every request** — A (F5)
- `boxmailhere.com` (193 KB, 9,798 overrides): about 0.9 ms and 1.4 MB per lookup, 2.2 ms and 1.6 MB per request
(re-measured).
- Fix: cache parsed records per loaded CDB.
**6. An empty `auth_token` leaves the API open with only a warning** — B (F4)
- A typo such as `auth-token =` starts the daemon unauthenticated; anyone who can reach it can inflate any customer's
counters, which triggers the web app's limit actions.
- Fix: make an empty token fatal unless explicitly opted out; make unknown config keys errors.
**7. Plus-addressed recipients get their own counters** — B (F2); A notes the Lua plugin did the same
- Mail to `b+1@…` creates `cnt/A/b+1@…`; the mailbox's own counter (the one the web app charts) never moves. If such
a counter reaches its limit, the web app answers "mailbox not found".
- rcptd strips the `+tag`, so stripping it here too seems right. **Needs a product decision.**
**8. One malformed CDB row breaks counting for anything touching that domain** — both (B F5, A F6)
- A non-numeric org id gives 503 and drops the other side's valid count; the plan says such rows are skipped.
- Fix: treat the row as not hosted and log a warning.
### Low
9. **Notification retries can send customers duplicate emails** — B (F11), A (F9). Resolved disagreement: the
organisation handler has a once-per-day guard (`migadu_web core/accounts.ex:1928-1956`), the mailbox handler
(`core/email.ex:3572-3584`) emails on every call, so a retry after a slow-but-successful call means a second
email. Also: shutdown cuts off pending retries, retries keep their concurrency slot while waiting, and the
"after 5 attempts" log line can be wrong.
10. **A SIGHUP during startup kills the daemon** — A (F8). The signal handler is installed only after the Redis
ping, which can take up to 2 s.
11. **Rejected requests leave no trace** — both (B F9, A F2 note). Nothing in the counting log, no trace_id in the app
log; together with #2 the lost counts are invisible.
12. **Override keys aren't lowercased** — A (F7). `Info@/o=5` would be ignored; none in today's data.
13. **Credentials in the notify URL are written to the log** — B (F7); plain `http://` with basic auth isn't warned about.
14. **Error responses expose internals** — B (F10): internal Redis addresses and paths; `admin_addr` defaults to all
interfaces without auth.
15. **Sentinel mode starts without a Redis password** — B (F12): every write then fails; only `/health` and metrics show it.
16. **counterctl warns one day too early** — B (F6): a date exactly 31 days back is reported as expired.
17. **A recovered panic isn't counted in `requests_total`** — B (F8).
18. **Work keeps running after a handler timeout** — A (F10, not reproduced): not tracked by shutdown, a late panic is
silently lost. Largely covered by the fix for #1.
**Test gaps** — B (F13): the suite misses most of the above; `TestValidation_JSONAndFields` asserts the 400 behind #2
as correct; watcher tests can occasionally pass on a stale signal.
## Verification
Done by the coordinating agent after both reviews:
- Both scratch copies were checked to have source files identical to the repo, so the proof tests ran against the
reviewed code. B's tests fail as described; A's demonstration tests reproduce their evidence (#1 was worse on
re-run: 0 counted instead of 21 of 300).
- External facts checked in the sibling repos:
- mailqueuer reads `X-Mail-From` without validating it (`internal/handler/ingest.go`).
- mizu's go-smtp fork unquotes quoted local parts and copies the domain verbatim (`parse.go` `parseMailbox`,
`parseLocalPart`); mizu sets `EnableSMTPUTF8 = true` (`cmd/mizu-server/main.go:1363`).
- mizu stores the raw SASL username as the authenticated user (`pkg/smtp/auth_session.go:160`) and sends it as
`X-Auth-User`; `user@domain@SUFFIX` logins are documented in mizu's `CLAUDE.md`.
- Web app: org-level limit handlers are guarded once per day, the mailbox-level ones are not (see #9).
- Reviewer disagreements: #7 (A: Lua parity, B: defect) is left as a product decision; #9 settled as above.
## Suggested order
Fix #1–4, #6, #8 and #10 first (small, they protect the counts), then #5 and #7 (#7 after the decision), then the
low items. Turn each proof test into a regression test as its finding is fixed.
+71
View File
@@ -0,0 +1,71 @@
# Proof tests from the 2026-10-14 code review
These are the tests the two reviewers wrote to demonstrate their findings. They live under a
directory whose name starts with `_`, so `go build ./...`, `go test ./...` and `go vet ./...`
ignore them. They are evidence, not part of the suite. When a finding is fixed, turn its proof
test into a real regression test in the package it belongs to.
| Dir | Reviewer | Style |
|---|---|---|
| `a/` | A — correctness and runtime | Demonstrations: the tests **pass** and print the evidence with `t.Log` (run with `-v`). |
| `b/` | B — security, contracts, ops | Assertions: the tests **fail** while the defect exists (a few `PASS`es only log evidence). |
## Running
Copy the files into a scratch copy of the repo, so the real packages stay clean:
```sh
rm -rf /tmp/mc_proof && cp -R /Users/dice/Work/migadu_counters /tmp/mc_proof && cd /tmp/mc_proof
cp -R review/_proof_tests/a/internal/. internal/ # reviewer A
cp -R review/_proof_tests/b/internal/. internal/ # reviewer B
cp -R review/_proof_tests/b/cmd/. cmd/
go test -count=1 -v -run 'TestReview' ./internal/... ./cmd/...
go test -run '^$' -bench Review -benchmem ./internal/txlog/ # A's CDB parsing benchmarks
```
Requirements and notes:
- `a/internal/store/review_test.go` (`TestReview_TimeoutAfterWriteStillCounts`) needs a real Redis on
port 16401: `redis-server --port 16401 --save '' --appendonly no`. The test uses `DEBUG SLEEP` to make the
reply arrive after the client timeout. It fails to connect if no Redis is running.
- `b/bad.cdb` is a three-row fixture with one malformed row (`bad.com x12|A/i=5`). It is used by
`b/internal/server/review_b4_test.go`, whose path was updated to `../../review/_proof_tests/b/bad.cdb`.
- `a/run/` holds the config and the app log from reviewer A's manual reproduction of "SIGHUP during startup
kills the daemon" (merged item #10). The paths in `config.toml` point at the reviewer's former scratch copy,
so edit them before reuse. The log stops after `domains db loaded`, which is the evidence.
## Map to the merged report (`../README.md`)
| Merged # | Proof |
|---|---|
| 1 | `a/internal/server/review_test.go` `TestReview_ClientDisconnectLosesCount`, `TestReview_ClientDisconnectNoDelay` |
| 2 | `b/internal/server/review_b_test.go` `TestReviewB_InMailFromDropsRecipientCount`, `TestReviewB_OutRecipientDropsSenderCount`; `a/…` `TestReview_MasterLoginAndOddSenders` |
| 3 | `a/internal/server/review_test.go` `TestReview_MasterLoginAndOddSenders` (first case) |
| 4 | `b/internal/server/review_b2_test.go` `TestReviewB_LostCrossingOnReplyTimeout`; `a/internal/server/review_test.go` `TestReview_PartialExecFailure`; `a/internal/store/review_test.go` (real Redis) |
| 5 | `a/internal/txlog/review_bench_test.go` |
| 6 | `b/internal/config/review_b_test.go` `TestReviewB_MisconfigStarts`; `b/…/review_b_test.go` `TestReviewB_EmptyTokenOpen` |
| 7 | `b/internal/server/review_b_test.go` `TestReviewB_PlusTagSeparateCounters` |
| 8 | `b/internal/server/review_b4_test.go`; `a/internal/counting/review_test.go` |
| 10 | `a/run/` (manual reproduction) |
| 11 | `b/internal/server/review_b_test.go` `TestReviewB_RejectLogHasNoTraceID` |
| 13 | `b/internal/server/review_b3_test.go` |
| 14 | `b/internal/server/review_b_test.go` `TestReviewB_ErrorLeak` |
| 15 | `b/internal/config/review_b_test.go` (case "sentinel without password") |
| 16 | `b/cmd/counterctl/review_b_test.go` `TestReviewB_31DayWarningOffByOne` |
| 17 | `b/internal/server/review_b_test.go` `TestReviewB_PanicNotInMetrics` |
## Fixed in round 1 (2026-10-14) — regression tests now in the suite
| Merged # | Regression test |
|---|---|
| 1 | `internal/server` `TestClientDisconnectStillCounts`, `TestAcceptedBeforeCounting`, `TestQueueFull`, `TestDrainCountsEverything`; `internal/counting/queue_test.go` |
| 2 | `internal/server` `TestIn_OddMailFromStillCountsRecipient`, `TestOut_OddRecipientStillCountsSender`; `internal/api` `TestValidateOut`, `TestValidateIn`, `TestNormalizeEmail` |
| 3 | `internal/server` `TestOut_LoginSuffixCountsBaseAddress`; `internal/api` `TestNormalizeAuthUser` |
| 4 | `internal/server` `TestPartialExecFailureStillNotifies`, `TestLostReplyStillNotifies`; `internal/counting` `TestCounter_PartialExecFailure`, `TestCounter_LostReplyReReadsMaster`; `internal/store` `TestIncr_PartialFailure`, `TestOutcomeUnknown` |
| 5 | `internal/domains` `TestLookup_CachesParsedRecord`, `TestLookup_CacheBounded`, `BenchmarkLookupSmall`/`BenchmarkLookupHuge` |
| 7 | `internal/server` `TestIn_PlusTagCountsMailbox`; `internal/counting` `TestPlan_PlusTagCountsMailbox`; `internal/domains` `TestMailbox` (decision 2026-10-15: strip the tag, as rcptd does) |
| 8 | `internal/server` `TestMalformedRowCountsOtherSide`; `internal/counting` `TestPlan_MalformedRowIsNotHosted`; `internal/domains` `TestLookup_MalformedRow`; `internal/txlog` `TestTrail_MalformedRow` |
| 6 | `internal/config` `TestValidate_Errors` (auth_token / unknown keys), `TestLoad_TokenFromEnvOnly`, `TestLoad_Defaults` (`:5555`); `cmd/migadu_counters` `TestRun_FlagsAndExitCodes` |
The proof files stay here as the review record; they no longer compile against the current API
(`CountResponse`, `store.Incr`) and are not meant to.
@@ -0,0 +1,34 @@
package counting
import (
"path/filepath"
"testing"
"time"
"github.com/colinmarc/cdb"
"migadu_counters/internal/domains"
)
func TestReview_MalformedRowFailsWholeRequest(t *testing.T) {
path := filepath.Join(t.TempDir(), "d.cdb")
w, err := cdb.Create(path)
if err != nil {
t.Fatal(err)
}
w.Put([]byte("good.example"), []byte("7|A/o=5,A/i=5"))
w.Put([]byte("bad.example"), []byte("|A/i=5")) // org id missing (malformed row)
if _, err := w.Freeze(); err != nil {
t.Fatal(err)
}
db, err := domains.Open(path)
if err != nil {
t.Fatal(err)
}
defer db.Close()
p := &Planner{Lookup: db, Now: time.Now}
plan, err := p.PlanOut("t", "x@good.example", "x@good.example", "y@bad.example")
t.Logf("PlanOut(good sender -> malformed-row recipient): plan=%v err=%v", plan, err)
plan, err = p.PlanIn("t", "s@gmail.com", "y@bad.example", false)
t.Logf("PlanIn(malformed-row recipient): plan=%v err=%v", plan, err)
}
@@ -0,0 +1,96 @@
package server
import (
"fmt"
"net"
"strings"
"testing"
"time"
"migadu_counters/internal/counting"
"migadu_counters/internal/domains"
)
func rawPostAndClose(t *testing.T, addr, path, body string) {
c, err := net.Dial("tcp", addr)
if err != nil {
t.Fatal(err)
}
req := fmt.Sprintf("POST %s HTTP/1.1\r\nHost: x\r\nAuthorization: Bearer %s\r\nContent-Type: application/json\r\nContent-Length: %d\r\n\r\n%s", path, token, len(body), body)
if _, err := c.Write([]byte(req)); err != nil {
t.Fatal(err)
}
c.Close() // fire-and-forget client that does not wait for the reply
}
func TestReview_ClientDisconnectLosesCount(t *testing.T) {
h := newHarness(t, options{})
real := *h.lookup.Load()
slow := counting.LookupFunc(func(d string) (*domains.Domain, error) {
time.Sleep(50 * time.Millisecond) // any latency before the Redis call
return real(d)
})
h.lookup.Store(&slow)
addr := strings.TrimPrefix(h.api.URL, "http://")
const n = 20
for i := 0; i < n; i++ {
rawPostAndClose(t, addr, "/in", `{"trace_id":"t","mail_from":"s@gmail.com","recipient":"a@yagolivas.com"}`)
}
time.Sleep(1500 * time.Millisecond)
v, _ := h.write.Get("cnt/A/a@yagolivas.com/d/" + day + "/i")
t.Logf("sent %d /in requests, counter = %q", n, v)
t.Logf("txlog tail:\n%s", lastLines(h.tx.String(), 3))
}
func TestReview_ClientDisconnectNoDelay(t *testing.T) {
h := newHarness(t, options{})
addr := strings.TrimPrefix(h.api.URL, "http://")
const n = 300
for i := 0; i < n; i++ {
rawPostAndClose(t, addr, "/in", `{"trace_id":"t","mail_from":"s@gmail.com","recipient":"b@yagolivas.com"}`)
}
time.Sleep(2 * time.Second)
v, _ := h.write.Get("cnt/A/b@yagolivas.com/d/" + day + "/i")
t.Logf("sent %d /in requests without delay, counter = %q", n, v)
}
func lastLines(s string, n int) string {
l := strings.Split(strings.TrimRight(s, "\n"), "\n")
if len(l) > n {
l = l[len(l)-n:]
}
return strings.Join(l, "\n")
}
func TestReview_MasterLoginAndOddSenders(t *testing.T) {
h := newHarness(t, options{})
cases := []struct{ path, body string }{
{"/out", `{"trace_id":"m1","auth_user":"x@yagolivas.com@SUPPORTTOKEN","mail_from":"x@yagolivas.com","recipient":"y@yagolivas.com"}`},
{"/in", `{"trace_id":"i1","mail_from":"a b@spammer.example","recipient":"y@yagolivas.com"}`},
{"/in", `{"trace_id":"i2","mail_from":"u@[192.0.2.1]","recipient":"y@yagolivas.com"}`},
{"/in", `{"trace_id":"i3","mail_from":"u@münchen.de","recipient":"y@yagolivas.com"}`},
{"/in", `{"trace_id":"i4","mail_from":"u@example.com.","recipient":"y@yagolivas.com"}`},
{"/in", `{"trace_id":"i5","mail_from":"a@b@example.com","recipient":"y@yagolivas.com"}`},
}
for _, c := range cases {
st, body := h.post(c.path, c.body)
t.Logf("%s %s -> %d %s", c.path, c.body, st, strings.TrimSpace(body))
}
t.Logf("keys in redis: %v", h.write.Keys())
}
func TestReview_PartialExecFailure(t *testing.T) {
h := newHarness(t, options{})
// A counter key holding a non-integer (or wrong type) value.
h.write.Set("cnt/O/1049/d/"+day+"/o", "garbage")
// A/o limit is 20 for yagolivas.com; preset A/o to 19 so this request crosses it.
h.write.Set("cnt/A/x@yagolivas.com/d/"+day+"/o", "19")
st, body := h.post("/out", `{"trace_id":"p1","auth_user":"x@yagolivas.com","recipient":"z@gmail.com"}`)
t.Logf("status %d body %s", st, strings.TrimSpace(body))
a, _ := h.write.Get("cnt/A/x@yagolivas.com/d/" + day + "/o")
d, _ := h.write.Get("cnt/D/yagolivas.com/d/" + day + "/o")
t.Logf("after: A/o=%s (limit 20 crossed) D/o=%s", a, d)
time.Sleep(300 * time.Millisecond)
t.Logf("address notifications sent: %d", h.addrURL.count())
t.Logf("txlog:\n%s", h.tx.String())
}
@@ -0,0 +1,36 @@
package store
import (
"context"
"testing"
"time"
"github.com/redis/go-redis/v9"
"migadu_counters/internal/config"
)
func TestReview_TimeoutAfterWriteStillCounts(t *testing.T) {
st, err := New(config.RedisConfig{WriteAddr: "127.0.0.1:16401", PoolSize: 4, TimeoutMS: 500})
if err != nil {
t.Fatal(err)
}
defer st.Close()
admin := redis.NewClient(&redis.Options{Addr: "127.0.0.1:16401"})
defer admin.Close()
key := "cnt/A/review@example.com/d/2026-10-14/o"
admin.Del(context.Background(), key)
admin.Set(context.Background(), key, "99", 0) // limit 100: next increment crosses
// warm the pool
if err := st.Ping(context.Background()); err != nil {
t.Fatal(err)
}
go admin.Do(context.Background(), "DEBUG", "SLEEP", "1") // server stall > timeout_ms
time.Sleep(50 * time.Millisecond)
_, err = st.Incr(context.Background(), []string{key}, 1)
t.Logf("Incr error: %v", err)
time.Sleep(1500 * time.Millisecond)
v, _ := admin.Get(context.Background(), key).Result()
t.Logf("key after the 'failed' Incr: %s", v)
admin.Del(context.Background(), key)
}
@@ -0,0 +1,63 @@
package txlog
import (
"io"
"testing"
"time"
"migadu_counters/internal/counting"
"migadu_counters/internal/domains"
)
const cdbPath = "../../test_files/rspamd_domains_db.cdb"
func BenchmarkReview_Open(b *testing.B) {
for i := 0; i < b.N; i++ {
db, err := domains.Open(cdbPath)
if err != nil {
b.Fatal(err)
}
db.Close()
}
}
func benchLookup(b *testing.B, dom string) {
db, err := domains.Open(cdbPath)
if err != nil {
b.Fatal(err)
}
defer db.Close()
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if _, err := db.Lookup(dom); err != nil {
b.Fatal(err)
}
}
}
func BenchmarkReview_LookupSmall(b *testing.B) { benchLookup(b, "yagolivas.com") }
func BenchmarkReview_LookupHuge(b *testing.B) { benchLookup(b, "boxmailhere.com") }
func BenchmarkReview_InRequestHuge(b *testing.B) {
db, err := domains.Open(cdbPath)
if err != nil {
b.Fatal(err)
}
defer db.Close()
p := &counting.Planner{Lookup: db, Now: time.Now}
l := New(io.Discard, "text", nil)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
plan, err := p.PlanIn("t", "s@gmail.com", "someone@boxmailhere.com", false)
if err != nil {
b.Fatal(err)
}
out := &counting.Outcome{Plan: plan}
for _, inc := range plan.Increments {
out.Results = append(out.Results, counting.Result{Increment: inc, Count: 1})
}
l.Outcome(out, nil)
}
}
+1
View File
@@ -0,0 +1 @@
time=2026-10-14T08:24:13.995+02:00 level=INFO msg="domains db loaded" path=/tmp/review_a/test_files/rspamd_domains_db.cdb domains=93549
+13
View File
@@ -0,0 +1,13 @@
addr = "127.0.0.1:18095"
admin_addr = "127.0.0.1:19095"
auth_token = "tok"
domains_cdb = "/tmp/review_a/test_files/rspamd_domains_db.cdb"
[logging]
output = "/tmp/review_a/run/app.log"
counting_output = "/tmp/review_a/run/counting.log"
[redis]
write_addr = "127.0.0.1:16401"
timeout_ms = 500
[notify]
org_url = ""
address_url = ""
Binary file not shown.
@@ -0,0 +1,60 @@
package main
import (
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/alicebob/miniredis/v2"
"migadu_counters/internal/config"
"migadu_counters/internal/domains"
"migadu_counters/internal/server"
"migadu_counters/internal/store"
)
// Date exactly 31 days back: its keys are still alive (TTL refreshed by the
// day's last increment, so they live until ~23:59 today) but counterctl claims
// it is "more than 31 days ago" and "will read as 0".
func TestReviewB_31DayWarningOffByOne(t *testing.T) {
f := &fakeAPI{body: sampleResponse()}
s := httptest.NewServer(f)
defer s.Close()
cfg := writeConfig(t, "auth_token = \"t\"\n")
code, _, errOut := runCLI(t, noEnv, "--config", cfg, "--url", s.URL, "--address", "me@example.com", "--date", "2026-09-12")
t.Logf("now=%s --date 2026-09-12 (31 days back) exit=%d stderr=%q", now, code, errOut)
lastIncr := time.Date(2026, 9, 12, 23, 59, 0, 0, time.UTC)
t.Logf("a key last incremented at %s expires at %s (still alive now)", lastIncr, lastIncr.Add(store.TTL))
if strings.Contains(errOut, "will read as 0") {
t.Errorf("warning fires for a date whose keys still exist")
}
}
// --url with a trailing slash, against the real API handler.
func TestReviewB_URLTrailingSlash(t *testing.T) {
mr := miniredis.RunT(t)
st, err := store.New(config.RedisConfig{WriteAddr: mr.Addr(), PoolSize: 2, TimeoutMS: 1000})
if err != nil {
t.Fatal(err)
}
db, err := domains.Open("../../test_files/rspamd_domains_db.cdb")
if err != nil {
t.Fatal(err)
}
srv := server.New(server.Deps{AuthToken: "t", Store: st, Domains: func() *domains.DB { return db }, Now: func() time.Time { return now }})
s := httptest.NewServer(srv.Handler())
defer s.Close()
cfg := writeConfig(t, "auth_token = \"t\"\n")
for _, u := range []string{s.URL, s.URL + "/", s.URL + "/api"} {
code, out, errOut := runCLI(t, noEnv, "--config", cfg, "--url", u, "--address", "a@yagolivas.com")
t.Logf("--url %q -> exit %d stdout=%q stderr=%q", u, code, firstLine(out), strings.TrimSpace(errOut))
}
}
func firstLine(s string) string {
if i := strings.IndexByte(s, '\n'); i >= 0 {
return s[:i]
}
return s
}
@@ -0,0 +1,45 @@
package config
import (
"os"
"path/filepath"
"testing"
)
func TestReviewB_MisconfigStarts(t *testing.T) {
dir := t.TempDir()
for name, body := range map[string]string{
"typo token, open API": `auth-token = "s3cret"
domains_cdb = "x.cdb"
allowed_cidr = ["10.0.0.0/8"]
[redis]
write_addr = "127.0.0.1:6379"`,
"sentinel without password": `auth_token = "t"
domains_cdb = "x.cdb"
[redis]
master_name = "mymaster"
sentinels = ["10.0.0.10:26379"]`,
"notify creds in URL / http basic": `auth_token = "t"
domains_cdb = "x.cdb"
[redis]
write_addr = "127.0.0.1:6379"
[notify]
org_url = "http://admin:hunter2@admin.migadu.com/api/limits?level=$LEVEL&id=$ID&flow=$FLOW&period=$PERIOD&by=$ADDRESS&count=$COUNT"
user = "u"
password = "p"`,
"public admin": `auth_token = "t"
domains_cdb = "x.cdb"
admin_addr = "0.0.0.0:9095"
[redis]
write_addr = "127.0.0.1:6379"`,
} {
p := filepath.Join(dir, "c.toml")
os.WriteFile(p, []byte(body), 0o600)
cfg, err := Load(p)
if err != nil {
t.Logf("%s: Load error: %v", name, err)
continue
}
t.Logf("%s: Load OK; AuthToken=%q prefixes=%v redis.password=%q warnings=%q", name, cfg.AuthToken, cfg.AllowedPrefixes(), cfg.Redis.Password, cfg.Warnings())
}
}
@@ -0,0 +1,87 @@
package server
import (
"io"
"net"
"strings"
"sync/atomic"
"testing"
"time"
"github.com/alicebob/miniredis/v2"
"migadu_counters/internal/store"
)
// delayProxy forwards to backend; replies are held back by *delay.
func delayProxy(t *testing.T, backend string, delay *atomic.Int64) string {
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { ln.Close() })
go func() {
for {
c, err := ln.Accept()
if err != nil {
return
}
b, err := net.Dial("tcp", backend)
if err != nil {
c.Close()
continue
}
go func() { io.Copy(b, c); b.Close() }()
go func() {
buf := make([]byte, 32*1024)
for {
n, err := b.Read(buf)
if n > 0 {
time.Sleep(time.Duration(delay.Load()))
if _, werr := c.Write(buf[:n]); werr != nil {
return
}
}
if err != nil {
c.Close()
return
}
}
}()
}
}()
return ln.Addr().String()
}
// F: the EXEC reply is lost (read timeout) after Redis applied the increments:
// the service answers 503 "nothing counted", the crossing increment is never
// reported, and the web app is never notified for that day.
func TestReviewB_LostCrossingOnReplyTimeout(t *testing.T) {
backend := miniredis.RunT(t)
var delay atomic.Int64
proxy := delayProxy(t, backend.Addr(), &delay)
h := newHarness(t, options{writeAddr: proxy, handlerTimeout: 10 * time.Second})
key := store.Key("A", "a@yagolivas.com", day, "o") // A/o = 20
backend.Set(key, "19")
// warm the pool so the handshake is done before replies are delayed
if st, body := h.post("/in", `{"recipient":"w@sttml.ru"}`); st != 200 {
t.Fatalf("warmup %d %s", st, body)
}
delay.Store(int64(2500 * time.Millisecond)) // > timeout_ms 2000 of the harness
st, body := h.post("/out", `{"trace_id":"t1","auth_user":"a@yagolivas.com","recipient":"x@gmail.com"}`)
t.Logf("request 1: %d %s", st, strings.TrimSpace(body))
v, _ := backend.Get(key)
t.Logf("redis value after 503: %s", v)
t.Logf("txlog:\n%s", h.tx.String())
delay.Store(0)
time.Sleep(3 * time.Second)
st, body = h.post("/out", `{"trace_id":"t2","auth_user":"a@yagolivas.com","recipient":"x@gmail.com"}`)
t.Logf("request 2: %d %s", st, strings.TrimSpace(body))
time.Sleep(200 * time.Millisecond)
if h.addrURL.count() == 0 {
t.Errorf("A/o limit 20 was crossed (value %s -> now >20) but no notification was ever sent", v)
}
}
@@ -0,0 +1,32 @@
package server
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"migadu_counters/internal/notify"
"migadu_counters/internal/txlog"
)
func TestReviewB_NotifyURLCredentialsLogged(t *testing.T) {
var gotAuth string
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotAuth = r.Header.Get("Authorization")
}))
defer ts.Close()
u := strings.Replace(ts.URL, "http://", "http://admin:hunter2@", 1) + "/api/limits?level=$LEVEL&id=$ID&flow=$FLOW&period=$PERIOD&by=$ADDRESS&count=$COUNT"
buf := &syncBuffer{}
tx := txlog.New(buf, "text", nil)
n := notify.New(notify.Config{OrgURL: u, Timeout: time.Second}, NotifyHooks(tx, nil))
n.Notify(notify.Event{TraceID: "t", Route: "out", Level: "O", ID: "1049", Flow: "o", Address: "a@yagolivas.com", Count: 20, Limit: 20})
n.Shutdown(context.Background())
t.Logf("server saw Authorization=%q", gotAuth)
t.Logf("counting log: %s", buf.String())
if strings.Contains(buf.String(), "hunter2") {
t.Errorf("notify credentials written to the counting log")
}
}
@@ -0,0 +1,27 @@
package server
import (
"strings"
"testing"
"migadu_counters/internal/counting"
"migadu_counters/internal/domains"
)
func TestReviewB_MalformedOrgIDDropsOtherParty(t *testing.T) {
h := newHarness(t, options{})
db, err := domains.Open("../../review/_proof_tests/b/bad.cdb")
if err != nil {
t.Fatal(err)
}
defer db.Close()
lf := counting.LookupFunc(db.Lookup)
h.lookup.Store(&lf)
st, body := h.post("/out", `{"trace_id":"t","auth_user":"a@good.com","recipient":"b@bad.com"}`)
t.Logf("/out good.com -> bad.com: %d %s keys=%v", st, strings.TrimSpace(body), h.write.Keys())
st, body = h.post("/in", `{"trace_id":"t","recipient":"b@bad.com"}`)
t.Logf("/in bad.com: %d %s", st, strings.TrimSpace(body))
if len(h.write.Keys()) == 0 {
t.Errorf("sender a@good.com outgoing not counted because the recipient's CDB row is malformed")
}
}
@@ -0,0 +1,142 @@
package server
import (
"io"
"net"
"net/http"
"strings"
"testing"
"time"
"migadu_counters/internal/counting"
"migadu_counters/internal/domains"
"migadu_counters/internal/store"
)
// F1: an /in request whose (never counted) mail_from is unusual-but-SMTP-valid
// is rejected wholesale, so the recipient's incoming counters are not bumped.
func TestReviewB_InMailFromDropsRecipientCount(t *testing.T) {
h := newHarness(t, options{})
for _, mf := range []string{
"john smith@example.com", // go-smtp unquotes MAIL FROM:<"john smith"@example.com>
"a@b@example.com", // MAIL FROM:<"a@b"@example.com>
"info@bücher.de", // SMTPUTF8 U-label sender domain
"postmaster@[192.0.2.1]", // address literal
"bounce@mail_relay.example.com", // underscore host
strings.Repeat("v", 250) + "@example.com", // > 254 chars
} {
st, body := h.post("/in", `{"trace_id":"t","mail_from":"`+mf+`","recipient":"b@yagolivas.com"}`)
t.Logf("mail_from=%.40q -> %d %s", mf, st, strings.TrimSpace(body))
}
t.Logf("keys written: %v", h.write.Keys())
if !h.write.Exists(store.Key("A", "b@yagolivas.com", day, "i")) {
t.Errorf("recipient b@yagolivas.com never counted although every message was accepted by mizu")
}
}
// F1b: /out where one recipient has a domain the validator dislikes drops the
// sender's outgoing increment too.
func TestReviewB_OutRecipientDropsSenderCount(t *testing.T) {
h := newHarness(t, options{})
for _, rc := range []string{"x@[192.0.2.1]", "x@bücher.de", "x@host_name.example.com"} {
st, body := h.post("/out", `{"trace_id":"t","auth_user":"a@yagolivas.com","mail_from":"a@yagolivas.com","recipient":"`+rc+`"}`)
t.Logf("recipient=%q -> %d %s", rc, st, strings.TrimSpace(body))
}
if !h.write.Exists(store.Key("A", "a@yagolivas.com", day, "o")) {
t.Errorf("sender a@yagolivas.com outgoing never counted")
}
}
// F2: plus-addressing creates a separate A counter per tag.
func TestReviewB_PlusTagSeparateCounters(t *testing.T) {
h := newHarness(t, options{})
for _, r := range []string{"b+1@yagolivas.com", "b+2@yagolivas.com", "B+3@yagolivas.com"} {
if st, body := h.post("/in", `{"recipient":"`+r+`"}`); st != 200 {
t.Fatalf("%d %s", st, body)
}
}
t.Logf("keys: %v", h.write.Keys())
if !h.write.Exists(store.Key("A", "b@yagolivas.com", day, "i")) {
t.Errorf("mailbox b@yagolivas.com A/i counter (what the web app reads) never incremented")
}
}
// F5: a recovered panic is answered 500 but not counted in requests_total.
func TestReviewB_PanicNotInMetrics(t *testing.T) {
h := newHarness(t, options{})
boom := counting.LookupFunc(func(string) (*domains.Domain, error) { panic("boom") })
h.lookup.Store(&boom)
st, _ := h.post("/in", `{"recipient":"b@yagolivas.com"}`)
got := h.metric(h.m.Requests.WithLabelValues("/in", "500"))
t.Logf("status %d, requests_total{/in,500}=%v", st, got)
if got != 1 {
t.Errorf("500 from recovered panic not counted in requests_total")
}
}
// F6: rejection log has no trace_id.
func TestReviewB_RejectLogHasNoTraceID(t *testing.T) {
h := newHarness(t, options{})
st, _ := h.post("/in", `{"trace_id":"TRACE-XYZ","recipient":"nobody"}`)
if st != 400 {
t.Fatal(st)
}
if !strings.Contains(h.log.String(), "TRACE-XYZ") {
t.Errorf("rejection log lacks trace_id:\n%s", h.log.String())
}
}
// Leakage: 503 body / health carry raw redis errors with internal addresses.
func TestReviewB_ErrorLeak(t *testing.T) {
h := newHarness(t, options{})
addr := h.write.Addr()
h.write.Close()
st, body := h.post("/in", `{"recipient":"b@yagolivas.com"}`)
t.Logf("api: %d %s", st, strings.TrimSpace(body))
resp, err := http.Get(h.admin.URL + "/health")
if err != nil {
t.Fatal(err)
}
b, _ := io.ReadAll(resp.Body)
resp.Body.Close()
t.Logf("health: %d %s", resp.StatusCode, strings.TrimSpace(string(b)))
if strings.Contains(body, addr) {
t.Logf("API 503 body leaks redis address %s", addr)
}
}
// Chunked body over 64 KB is rejected 413.
func TestReviewB_ChunkedTooLarge(t *testing.T) {
h := newHarness(t, options{})
pr, pw := io.Pipe()
go func() {
pw.Write([]byte(`{"recipient":"b@yagolivas.com","pad":"`))
for i := 0; i < 100; i++ {
pw.Write([]byte(strings.Repeat("x", 1024)))
}
pw.Write([]byte(`"}`))
pw.Close()
}()
req, _ := http.NewRequest("POST", h.api.URL+"/in", pr)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatal(err)
}
b, _ := io.ReadAll(resp.Body)
resp.Body.Close()
t.Logf("chunked 100KB: %d %s (cl=%d)", resp.StatusCode, b, req.ContentLength)
if resp.StatusCode != 413 {
t.Errorf("want 413")
}
}
// Empty configured token = no auth at all.
func TestReviewB_EmptyTokenOpen(t *testing.T) {
h := newHarness(t, options{token: "x"})
_ = h
}
var _ = net.Listen
var _ = time.Second
+142
View File
@@ -0,0 +1,142 @@
# Review A — correctness and runtime behaviour
- Date: 2026-10-13/14 (interrupted by an API usage limit, resumed)
- Scope: the whole repository at the state of the first implementation; read-only
- Lens: counting rules vs plan/Lua, Redis store, UTC day, concurrency and lifecycle, resource use, error paths
- Proof tests: `_proof_tests/a/` (see `_proof_tests/README.md`)
- Verification by the coordinating agent and the merged ranking: `README.md`
The report below is the reviewer's text as delivered. Paths under `/tmp/review_a` refer to its scratch copy, which no longer exists.
---
Findings, most severe first. The repo was not modified. All scratch work is in /tmp/review_a: test files `internal/server/review_test.go`, `internal/store/review_test.go`, `internal/counting/review_test.go`, `internal/txlog/review_bench_test.go`, and a daemon build in `run/`. I started Redis on port 16401 and have killed it, along with any daemon processes I started.
---
**F1 — high — correctness/lifecycle: a client disconnect cancels the Redis write**
- location: `internal/server/server.go:156` (`s.d.Counter.Execute(r.Context(), plan)`)
- summary: Counting uses the request context. When mailqueuer closes the connection or its client times out before the reply, Go cancels that context and go-redis aborts before sending the MULTI (pool `waitTurn`, dial and retry backoff all check `ctx.Done()`). The increment is lost.
- failure scenario: A fire-and-forget client writes POST /in and closes the socket, or uses a short timeout while Redis or Sentinel is slow. The handler then logs "redis error: context canceled (nothing counted)" and the counter is never incremented. Because mailqueuer never retries, the loss is silent and permanent.
- evidence: `TestReview_ClientDisconnectNoDelay` sends 300 raw POST /in requests that close right after writing; the counter ends at "21", so 93% were lost. `TestReview_ClientDisconnectLosesCount`, with 50 ms of latency before Redis, loses 20 of 20. The txlog shows `redis error: redis incr: context canceled (nothing counted)`.
- verdict: CONFIRMED. How often it bites depends on how the not-yet-written mailqueuer client behaves.
- fix: `ctx, cancel := context.WithTimeout(context.WithoutCancel(r.Context()), budget)` for `Execute`, so the work does not depend on the caller staying connected.
**F2 — medium — counting rules / contract: master/token logins on /out get a 400, so nothing is counted**
- location: `internal/api/validate.go:29,84` (`requiredEmail("auth_user")`, exactly one `@`)
- summary: mizu puts the raw SASL login in X-Auth-User (`mizu/pkg/smtp/server.go:2065` passes `s.authenticatedUser`). For master/token logins (Sora support access, remotelookup tokens) that login is `user@dom@SUFFIX`, which NormalizeEmail rejects.
- failure scenario: `{"auth_user":"x@yagolivas.com@SUPPORTTOKEN","recipient":"y@yagolivas.com"}` returns 400 `auth_user: invalid e-mail address`. Neither the sender's outgoing A/D/O nor the hosted recipient's incoming A/D/O is counted. The counting log has no line for it, and the warn log omits trace_id, which the plan requires.
- evidence: `TestReview_MasterLoginAndOddSenders` prints `-> 400`, and the Redis keys are `[]`. mizu `CLAUDE.md:471-501` documents the `user@domain@SUFFIX` logins.
- verdict: CONFIRMED (rejection). How frequent these logins are is PLAUSIBLE, not measured.
- fix: Normalise auth_user by stripping a second `@SUFFIX` and counting the base address (`local@domain`). Log rejections to the counting log with the trace_id.
**F3 — medium — counting rules: a `mail_from` the sender controls can suppress counting of the recipient**
- location: `internal/api/validate.go:35,49` (`optionalEmail("mail_from")` fails the whole request)
- summary: mail_from is used only for the exempt check, yet a syntactically "invalid" envelope sender returns 400 for the whole request. go-smtp (the migadu fork used by mizu, `parse.go:173-238`) accepts these senders: it unquotes quoted local parts (`"a b"@x` becomes `a b@x`, `"a@b"@x` becomes `a@b@x`), and it accepts address literals, UTF-8 domains and trailing dots. mailqueuer passes X-Mail-From through unchanged.
- failure scenario: Inbound spam with `MAIL FROM:<"a b"@spammer.example>` (or `<u@[192.0.2.1]>`, `<u@münchen.de>`, `<u@example.com.>`) gets a 400 on /in, so the hosted recipient's incoming limits are never counted. The same applies to /out for allowed_from regex or wildcard senders.
- evidence: the same test: all five /in variants return 400 `mail_from: invalid e-mail address`, and no keys are written.
- verdict: CONFIRMED (rejection). That mizu accepts such senders end-to-end is PLAUSIBLE (shown in the parser only).
- fix: Treat an unparseable mail_from as unknown: log it, don't match it against the exempt list, and count anyway. Validate strictly only the fields that are counted.
**F4 — medium — Redis store: ambiguous Redis failures are logged as "nothing counted" and the crossing notification is lost**
- location: `internal/store/store.go:128-137`, `internal/server/server.go:156-166`, `internal/txlog/txlog.go:111`
- summary: `TxPipelined` returns an error in two cases where the increments may already be committed:
- (a) A read timeout after MULTI/EXEC was written. go-redis does not retry (`txPipelineProcessCmds` returns canRetry=false), but Redis executes the transaction.
- (b) A per-command error inside EXEC (non-integer or WRONGTYPE key). The other INCRBYs are applied.
In both cases the service answers 503, logs "nothing counted" and skips crossing detection. The one increment that crossed is consumed, so that level/id/flow is not notified again that day.
- failure scenario:
- (a) A/o=99 with limit 100 and the master stalls for more than `timeout_ms` (500 ms). Redis applies 100, the service returns 503, and no notification is sent.
- (b) The O key holds "garbage" and A/o=19 with limit 20. A becomes 20 and D becomes 1, but the response is 503 "nothing counted" and 0 notifications are sent.
- evidence: `TestReview_TimeoutAfterWriteStillCounts` against real Redis 8.6 with `DEBUG SLEEP 1`: `Incr error: ... i/o timeout`, then `key after the 'failed' Incr: 100`. `TestReview_PartialExecFailure` (miniredis): `status 503 ... ERR value is not an integer`, `A/o=20 (limit 20 crossed) D/o=1`, `address notifications sent: 0`.
- verdict: CONFIRMED
- fix:
- Use the per-command results: go-redis sets each cmd's error, so commands that succeeded can still be evaluated for crossing.
- On a timeout after the write, re-read the keys from the master and run crossing detection on any key now at or above its limit. Alternatively, raise the read timeout for EXEC.
- Log "outcome unknown" instead of "nothing counted".
**F5 — medium — resource use: huge CDB values are re-parsed (and their keys sorted) on every request**
- location: `internal/domains/domains.go:154-167` (`ParseRecord` per lookup), `internal/txlog/txlog.go:215-229` (`summarize` sorts every key per request)
- summary: boxmailhere.com (193 KB, 9,798 overrides) is read, split into a map of about 9.8k entries, then key-sorted for every /in or /out request that touches it.
- failure scenario: Sustained inbound mail to boxmailhere.com costs about 2.6 ms of CPU and 1.6 MB of garbage per request on an M2, so 100 req/s comes to roughly 160 MB/s of allocations and a quarter of a core. An /out between two such domains doubles it.
- evidence: `go test ./internal/txlog -bench Review` gives:
| benchmark | time/op | memory/op | allocs/op |
|---|---|---|---|
| LookupSmall | 1.6 µs | 544 B | 9 |
| LookupHuge | 908 µs | 1.43 MB | 88 |
| InRequestHuge (plan + txlog) | 2.64 ms | 1.60 MB | 266 |
| Open (count iteration) | 85 ms | – | – |
- verdict: CONFIRMED
- fix: Cache parsed records per DB generation, dropping the cache on swap. Or resolve limits lazily: extract only `O/,D/,A/` and `<local>@/<flow>` from the raw value. Pre-sort the base keys once.
**F6 — low — counting rules / spec deviation: a malformed CDB row fails the whole request instead of being skipped**
- location: `internal/domains/domains.go:116-123`, `internal/counting/counting.go:172-179`
- summary: The README and plan say "Malformed CDB rows are skipped, not fatal" (the Lua plugin and stats-merger both treat them as not hosted). Here a bad org id surfaces as a lookup error and the request is answered 503.
- failure scenario: A recipient row `|A/i=5` (empty org id), with sender x@good.example. PlanOut returns an error, the server answers 503 "domain lookup failed", and the valid sender's outgoing counters are not counted either. Every request touching that domain fails.
- evidence: `TestReview_MalformedRowFailsWholeRequest`: `plan=<nil> err=lookup recipient domain bad.example: domain bad.example: invalid org id ""`.
- verdict: CONFIRMED. There are no malformed rows in the current fixture.
- fix: In `Lookup`, map a ParseRecord error to ErrNotFound (or a distinct "malformed, not counted" result) plus a warning, and continue with the other party.
**F7 — low — spec deviation: override keys are not lowercased**
- location: `internal/domains/domains.go:54,74`
- summary: The plan says parsing follows stats-merger, where keys are lowercased (stats-merger `quota.go:103,118`). The code only trims keys, while the local part is lowercased by `normalize`.
- failure scenario: A CDB value with `Info@/o=5` and mail to or from `info@domain` gets A/o (for example 100) instead of 5. The per-mailbox limit is silently ignored.
- evidence: code trace. ParseLimits stores `k` as given, and Limit looks up `localPart+"@/"+flow` with a lowercased localPart. The fixture has 0 mixed-case overrides out of 22,995.
- verdict: CONFIRMED (code path). Impact with today's data is nil.
- fix: `strings.ToLower(strings.TrimSpace(k))` in ParseLimits.
**F8 — low — lifecycle: SIGHUP during startup kills the daemon**
- location: `cmd/migadu_counters/main.go:76-101` (`signal.Notify` runs only after newApp, the up-to-2 s Redis ping, Listen and Serve)
- summary: Go's default action for SIGHUP is to exit, so a reload or newsyslog HUP during startup terminates the process.
- failure scenario: The daemon starts while Redis is slow or unreachable (the ping can take up to 2 s), and `service reload` or newsyslog sends HUP. The process exits with status 129, and requests already being served are dropped.
- evidence: Redis on 16401 was held with `DEBUG SLEEP 3`, the daemon started, and HUP was sent after 0.3 s: `daemon exited with status 129`. The app log stops after "domains db loaded".
- verdict: CONFIRMED
- fix: Call `signal.Notify` (or `signal.Ignore(SIGHUP)`) first thing in `run`, before newApp.
**F9 — low — notifications: in-memory notifications are dropped on restart or short outages; log is misleading**
- location: `internal/notify/notify.go:148-185,217-223`
- summary:
- `Shutdown` closes `quit`, which aborts every pending retry immediately instead of using the 30 s drain.
- Retries hold a semaphore slot during backoff (1+2+4+8 s), so at most 8 events are delivered every 15 s or more while the web app fails.
- The total retry window is about 15 s plus attempt timeouts.
- When a retry is cut off by shutdown, the Failed hook still reports "after 5 attempts".
- failure scenario: A web app deploy lasting about 30 s, or a migadu_counters restart, while limits are crossed. Those crossings never notify, because the counter has already passed the limit and only a reset re-arms it.
- evidence: code trace. `case <-n.quit: ... attempt = n.cfg.Attempts`, then `Failed(ev, u, n.cfg.Attempts, ...)`.
- verdict: PLAUSIBLE
- fix: Let the shutdown deadline, not `quit`, bound retries. Release the semaphore while sleeping. Report the real attempt count. Consider a persistent or replayable record of reached-but-undelivered events, for example a Redis list.
**F10 — low — TimeoutHandler/shutdown interplay: lingering handlers misreport**
- location: `internal/server/server.go:95,181-183`, `cmd/migadu_counters/main.go:322-332`
- summary:
- After `http.TimeoutHandler` sends its 503, the handler goroutine keeps running, so the request is usually counted and notified despite the 503.
- These goroutines are not tracked by `http.Server.Shutdown`. If one finishes after `notifier.Shutdown`, its crossing is dropped and logged as "no address_url configured, not notifying", with `notifications_total{result="disabled"}`.
- A panic after the timeout is swallowed with no log, because TimeoutHandler's recover sends it into a channel nobody reads.
- verdict: PLAUSIBLE (code trace)
- fix: Give `Execute` its own context and budget (see F1). Track in-flight executions with a WaitGroup that Shutdown waits on before stopping the notifier. Distinguish "notifier closed" from "disabled".
---
**Checked, no issue found**
- **A/D/O planning:** /out counts sender A/D/O outgoing on auth_user's row and hosted recipient A/D/O incoming on the recipient's row. /in counts the recipient only. No duplicate keys in one MULTI, including self-sends and same-org cases.
- **Exempt and junk:** auth_user, mail_from and recipient (lowercased) are checked per the plan. Junk is skipped unless `count_junk`. Unknown domains are skipped. The behaviour changes against Lua are the ones the plan documents.
- **`<local>@/<flow>` override:** the override wins over `A/<flow>`. A duplicate key resolves last-wins, as in Lua. Empty or non-numeric values are skipped.
- **Limit 0 and negative limits:** limit 0 never crosses. This matches the web app, which strips `=0` overrides as "unlimited" (`config.ex` cdb_domain_limits). There are no zero or negative limits in the fixture.
- **Crossing detection:** old < limit ≤ new on atomic INCRBY, exactly once under concurrency. After the web app's DEL it re-fires, and the web app's own once-per-day guard (`limit_exceeded_at`) suppresses the duplicate email.
- **Key format:** matches `redis_sentinel.ex` (`cnt/<L>/<id>/d/YYYY-MM-DD/<i|o>`, DB 0, lowercased ids). TTL is refreshed in the same MULTI.
- **UTC day:** the date is computed once per plan, so all keys in a request share the same day. `/counters`, counterctl and the txlog all use UTC.
- **go-redis behaviour:**
- TxPipeline is not retried after the write, so there is no double counting.
- READONLY, OOM or NOREPLICAS during queueing turn into EXECABORT, which is all-or-nothing.
- ReplicaOnly falls back to the master. Sentinel address slices are copied.
- MGET parsing treats nil as 0.
- **MULTI per-command error semantics:** handled as reported in F4, nothing beyond that.
- **CDB swap:** the parsed Domain is a copy. Lookups happen right after `Load`, so the 30 s grace is enough. Reloads are serialised by reloadMu.
- **fsnotify on kqueue:** a rename-over produces Remove followed by Create, so the reload fires. The `.tmp` file is ignored.
- **File handles on reload:** a failed `cdb.Open` leaks the *os.File only until the GC finalizer closes it. Open with its count iteration takes about 85 ms.
- **Watcher shutdown:** Stop and debounce races are harmless; a late reload only affects a process that is already exiting.
- **Logging:** LogWriter Reopen and Write are mutex-safe. A SIGHUP during shutdown is ignored. The fatal channel is buffered for both servers.
- **Notifier:** wg.Add happens under the closed-check lock, so there is no Add-after-Wait race. URL escaping and `$LEVEL`/`$LIMIT` replacement order are correct, and all six web-app params are present in the example config.
- **Panics:** a panic before the timeout is recovered with a 500. Indexing in `/counters` and the address split is guarded by validation.
- **Plus-addressing:** counted as a separate address, same as Lua.
- **Tooling:** `go vet ./...` is clean, and all existing tests pass in the scratch copy.
+136
View File
@@ -0,0 +1,136 @@
# Review B — security, contracts and operability
- Date: 2026-10-13/14 (interrupted by an API usage limit, resumed)
- Scope: the whole repository at the state of the first implementation; read-only
- Lens: HTTP validation layer, input normalisation and keys, web-app notification contract, config and secrets, counterctl, metrics and /health, docs vs code, test quality
- Proof tests: `_proof_tests/b/` (see `_proof_tests/README.md`)
- Verification by the coordinating agent and the merged ranking: `README.md`
The report below is the reviewer's text as delivered. Paths under `/tmp/review_b` refer to its scratch copy, which no longer exists; the fixture `bad.cdb` now lives at `_proof_tests/b/bad.cdb`.
---
## Findings (most severe first)
Scratch copy: `/tmp/review_b`. My proof tests are `internal/server/review_b*_test.go`, `cmd/counterctl/review_b_test.go`, `internal/config/review_b_test.go` and the fixture `/tmp/review_b/bad.cdb`. Nothing under /Users/dice/Work was modified. No Redis server was started; every Redis was miniredis, so there are no processes to kill.
---
**F1 — high — contract / security (limit bypass)**
- location: `internal/api/validate.go:35,49` (optional `mail_from` goes through `requiredEmail`), `:32`, `:78-103` (`NormalizeEmail`: ASCII-only domain, ≤254, no whitespace); `internal/server/server.go:115-118,133-136`
- summary: If any field fails the strict syntax check, the whole request is rejected with 400. That includes `mail_from`, which is never counted. So the sender alone decides whether the recipient's increments happen, and on /out a recipient address can drop the sender's outgoing increments.
- failure scenario: An external sender floods `b@yagolivas.com` with `MAIL FROM:<"john smith"@x.com>`. mizu's go-smtp fork (`migadu/go-smtp parse.go:208-224`) unquotes this to `john smith@x.com`. mailqueuer never validates X-Mail-From. Other spellings work the same way: `<"a@b"@x.com>`, `<info@bücher.de>` (mizu sets `EnableSMTPUTF8=true`), `<postmaster@[192.0.2.1]>`, `<x@mail_relay.example.com>`, or any path over 254 chars. Every /in call then returns 400 and the recipient's A/D/O incoming counters never move, so the incoming limit never fires. Legitimate SMTPUTF8 senders on IDN domains are not counted either. On /out, a recipient like `x@[192.0.2.1]`, `x@bücher.de` or `x@host_name.example.com` makes the call 400, so the authenticated sender's A/D/O outgoing increments are lost. A 400 also writes nothing to the counting log (see F9), so the loss is invisible there.
- evidence: `TestReviewB_InMailFromDropsRecipientCount`: all 6 mail_from variants return `400 {"error":"mail_from: invalid e-mail address"}` and leave `keys written: []`. `TestReviewB_OutRecipientDropsSenderCount`: all 3 return `400 recipient: invalid e-mail address`, and `cnt/A/a@yagolivas.com/.../o` is never created.
- verdict: CONFIRMED for the service behaviour. The end-to-end path is PLAUSIBLE, based on reading the go-smtp parser and confirming that mailqueuer does not validate the from header.
- fix: On /in, and on /out for `mail_from`, never reject because of `mail_from`: sanitise it for logging and treat it as non-exempt. Convert U-label domains with `idna.Lookup.ToASCII`. On /out, validate sender and recipient independently and count whichever side is valid. Keep 400 only for a missing or garbage *counted* address.
**F2 — medium — input normalisation (two spellings, two counters)**
- location: `internal/counting/counting.go:164-198` (A id = raw `address`; the override lookup uses the raw local part); `internal/api/validate.go:78`
- summary: Plus-addressed recipients get their own A key per tag. The mailbox's own A counter, the one the web app reads and charts, is never incremented, and the per-mailbox `local@/i` override is never applied.
- failure scenario: rcptd strips `+tag` when validating (see `rcptd/internal/tables/regex_plustag_test.go`), so mizu accepts `b+1@…`, `b+2@…` and so on. Mail sent to `b+N@yagolivas.com` creates `cnt/A/b+N@yagolivas.com/...` keys, each starting at 0. The mailbox `b@yagolivas.com` never reaches A/i. When a tag key does cross, the web app gets `id=b+N@…`, `get_active_mailbox_by_address` returns nil, and it only sends a "mailbox not found" Telegram canary. An attacker can create unbounded keys, each with a 31-day TTL. The Lua plugin had the same behaviour, but the plan's goal is keys the web app reads.
- evidence: `TestReviewB_PlusTagSeparateCounters`: keys = `cnt/A/b+1@…/i, cnt/A/b+2@…/i, cnt/A/b+3@…/i, cnt/D/…, cnt/O/1049/…`; `cnt/A/b@yagolivas.com/.../i` is missing.
- verdict: CONFIRMED
- fix: Canonicalise the A-level id and the override local part by stripping `+…` from the local part (rcptd's recipient_delimiter), and apply the same rule in `/counters`.
**F3 — medium — correctness / notification contract**
- location: `internal/store/store.go:123-143`, `internal/server/server.go:155-167`, `internal/txlog/txlog.go:111`, `internal/counting/counting.go:204-206`
- summary: If Redis applies MULTI/EXEC but the reply is lost (read timeout, connection reset, handler timeout), the service answers 503 and logs "nothing counted". The counters were incremented, though. If that increment was the crossing one, no later increment can cross again, so the web app is never notified for that level/id/flow/day.
- failure scenario: A/o limit 20, counter at 19. The master stalls longer than `timeout_ms` (fork, failover). Redis applies the INCR (value 20), go-redis returns `i/o timeout` (txPipeline reads are not retried), and the client gets 503. The counting log says `redis error … (nothing counted)`. The next mail gives count 21 with `reached:false`, and no notification is ever sent.
- evidence: `TestReviewB_LostCrossingOnReplyTimeout` uses a delaying TCP proxy in front of miniredis. Request 1 returns `503 redis incr: read tcp …: i/o timeout`, and Redis then holds `20`. Request 2 returns `"count":21,"limit":20,"reached":false`. Zero notifications.
- verdict: CONFIRMED
- fix: For a failed or ambiguous EXEC, do not claim "nothing counted". Make crossing detection recoverable: on the next successful increment, notify when `old >= limit` and a per-day marker (`SET NX`) is absent; or use a marker key set together with the INCR. Log "outcome unknown" instead of "nothing counted".
**F4 — medium — config / auth**
- location: `internal/server/middleware.go:140-142`; `internal/config/config.go:296-300` (empty token is only a warning), `:151-153` (unknown keys are only warnings)
- summary: An empty `auth_token` turns off authentication on /in, /out and /counters, and a typo in the key name produces that state with only a log warning. The plan's step 4 makes the bearer token mandatory.
- failure scenario: `auth-token = "s3cret"` and `allowed_cidr = ["10.0.0.0/8"]` (typos), or `MIGADU_COUNTERS_AUTH_TOKEN` missing from the rc environment. The daemon starts, anyone who can reach :8095 can inflate any customer's counters (which triggers the web app's limit actions) or read counters, and the allowlist is silently empty.
- evidence: `TestReviewB_MisconfigStarts` "typo token": Load OK with `AuthToken="" prefixes=[]` and three warnings only.
- verdict: CONFIRMED
- fix: Make an empty `auth_token` a validation error, or require an explicit `insecure_no_auth = true`. Make unknown keys fatal, or at least fatal for security-relevant near-misses.
**F5 — medium/low — CDB robustness / spec deviation**
- location: `internal/domains/domains.go:116-124` (`ParseRecord` errors on a bad org id), `internal/counting/counting.go:172-180`, `internal/server/server.go:145-151`
- summary: One CDB row with a non-numeric org id causes 503 for every request that touches that domain. On /out it also drops the other party's valid increments. The plan says malformed rows are skipped, not fatal; Lua treated such a row as not hosted.
- failure scenario: The row `bad.com x12|A/i=5` is deployed. `/out a@good.com → b@bad.com` returns 503, and `a@good.com`'s outgoing is not counted. Every `/in` to bad.com returns 503.
- evidence: `TestReviewB_MalformedOrgIDDropsOtherParty` (fixture `/tmp/review_b/bad.cdb`): `503 {"error":"domain lookup failed: … invalid org id \"x12\""} keys=[]`.
- verdict: CONFIRMED (the real fixture has no such rows today)
- fix: Have `Lookup` return `ErrNotFound`, or a dedicated skip error, for unparsable records, log once, and count the other party.
**F6 — low — counterctl date warning**
- location: `cmd/counterctl/main.go:88-90`
- summary: The comparison is `now − midnight(D) > 31d`, so the warning fires for a date exactly 31 days back and claims its keys "will read as 0". The plan says "older than 31 days".
- failure scenario: The current time is 2026-10-13 12:00 UTC and the operator runs `--date 2026-09-12`. That day's keys were last refreshed around 23:59 and live until about 2026-10-13 23:59, so real non-zero numbers print alongside "more than 31 days ago … will read as 0".
- evidence: `TestReviewB_31DayWarningOffByOne`: the warning is printed; a key last incremented at 09-12 23:59 expires 10-13 23:59. The existing test covers only 12 and 42 days back.
- verdict: CONFIRMED
- fix: Warn only when `today − D ≥ 32 days` (always expired). At exactly 31 days, say the counts may be partial.
**F7 — low — secrets in logs**
- location: `internal/config/config.go:279-287` (userinfo in notify URLs accepted, `http://` with basic auth accepted without warning), `internal/notify/notify.go:147`, `internal/txlog/txlog.go:125,133`, `internal/config/logger.go:110` (mode 0644)
- summary: Credentials given as URL userinfo work, because Go sends them as basic auth. The counting log then prints the full URL, password included, on every sent or failed notification, into a world-readable file.
- evidence: `TestReviewB_NotifyURLCredentialsLogged`: the server sees `Basic YWRtaW46aHVudGVyMg==`, and the log line is `sent limit exceeded notification to url "http://admin:hunter2@127.0.0.1:…/api/limits?…"`.
- verdict: CONFIRMED
- fix: Reject userinfo in the notify URLs, or redact with `url.Redacted()` before logging. Warn or fail when `user` is set with an `http://` URL.
**F8 — low — metrics**
- location: `internal/server/middleware.go:73-104`, `server.go:96` (`recoverer(observe(timed))`; `observe` has no defer)
- summary: A recovered panic is answered 500 but never counted in `requests_total`. The plan says every rejection or recovery increments `requests_total{status}`.
- evidence: `TestReviewB_PanicNotInMetrics`: status 500, `requests_total{/in,500}=0`.
- verdict: CONFIRMED
- fix: Record in a deferred function in `observe`, or put `observe` outside `recoverer`.
**F9 — low — operability / spec deviation**
- location: `internal/server/middleware.go:46-49`
- summary: Rejection logs carry no `trace_id`, although the plan requires it "if parsable". The counting log gets nothing for 4xx rejections, despite README's "Every request writes one block" and `grep '<trace_id>'` returns one transaction.
- evidence: `TestReviewB_RejectLogHasNoTraceID`: the log line is `msg="request rejected" route=/in remote=… status=400 reason="recipient: invalid e-mail address"`, without TRACE-XYZ. This combines with F1 to make the dropped counts invisible.
- verdict: CONFIRMED
- fix: Pass the sanitised trace_id and route into `reject`, and write a `rejected: <reason>` line to the txlog.
**F10 — low — information exposure**
- location: `internal/server/server.go:150,165,236,263,301`; `internal/config/config.go:117` (default `admin_addr=":9095"`, all interfaces, no auth, no allowlist)
- summary: 503 bodies, and `/health` (unauthenticated), return raw Redis and CDB errors that include internal addresses and paths. The default admin bind is public, while the code comment says to "bind admin_addr locally".
- evidence: `TestReviewB_ErrorLeak`: API returns `{"error":"redis unavailable: redis incr: dial tcp 127.0.0.1:52115: connect: connection refused"}`, and `/health` returns `{"error":"redis: dial tcp 127.0.0.1:52115: …"}`.
- verdict: CONFIRMED
- fix: Return generic messages and keep the details in the app log. Default `admin_addr` to `127.0.0.1:9095`, or warn when it is not loopback.
**F11 — low — notification contract (idempotency)**
- location: `internal/notify/notify.go:157-213`
- summary: Up to 5 attempts, with a retry on client timeout, transport error, 429 or 5xx, against a non-idempotent web action. The A branch (`Core.Email.incoming/outgoing_limit_exceeded!`, `core/email.ex:3572-3584`) emails the customer on every call. The canary Telegram message is also sent before the O-level action can raise.
- failure scenario: The web app takes more than 5 s (the per-attempt timeout) but completes. The service retries, and the customer gets 2–5 "limit exceeded" emails plus duplicate canaries. Lua sent once.
- verdict: PLAUSIBLE
- fix: Retry only on connect errors and 5xx-before-processing, or add an idempotency key (trace_id+level+id+flow+day) that the web app can dedupe on. Consider a longer timeout with fewer retries.
**F12 — low — config validation**
- location: `internal/config/config.go:257-266`
- summary: Sentinel mode is accepted without `redis.password`, although the deployment has one Redis password.
- failure scenario: The password is missing or the env var is unset. The daemon starts, every INCR gets NOAUTH and returns 503, and mailqueuer ignores 503s, so nothing is counted fleet-wide. Only `/health` and `redis_errors_total` show it.
- evidence: `TestReviewB_MisconfigStarts` "sentinel without password": Load OK, `warnings=[]`.
- verdict: CONFIRMED
- fix: Require the password in sentinel mode, or at least warn. Make a NOAUTH/WRONGPASS reply to the startup PING fatal.
**F13 — low — test-suite gaps (tests pass for the wrong reasons)**
- `cmd/counterctl/main_test.go:180-208` never tests the 31/32-day boundary, which hides F6.
- `server_test.go:482-497` checks the panic answer but not the metric (F8).
- `:422-454` checks that rejections are logged but not the trace_id (F9).
- `:372-395` never covers the empty-token open mode (F4).
- Nothing covers an ambiguous Redis failure (F3), plus-tags (F2), or a malformed org id (F5).
- `TestValidation_JSONAndFields` asserts as correct the 400 on a `mail_from` the system never counts (F1).
- Watcher tests use a 16-slot buffered `done` channel plus sleeps, so a stale signal from an earlier event could satisfy `c.wait` in `TestWatcher_ReloadsOnAtomicReplaceTwice`. This is a mild flakiness and false-pass risk.
- verdict: CONFIRMED (by inspection and by the F6/F8/F9 tests above)
## Checked, no issue found
- Middleware order matches the plan's table: method → declared size → token → source IP → content-type → MaxBytesReader. Note that auth runs before the allowlist, so a non-allowed source can tell a valid token (403) from an invalid one (401); this is minor.
- Body size: chunked 100 KB body → 413 (`TestReviewB_ChunkedTooLarge`). The declared Content-Length check sits before auth.
- Auth: missing, `Basic` and empty `Bearer ` → 401 with `WWW-Authenticate`. ConstantTimeCompare leaks only the length. The `Bearer` scheme is case-sensitive, which is fine for mailqueuer (`routing.go:257`).
- Content-Type: parameters are ignored, other types → 415.
- JSON: syntax errors, two objects and trailing data → 400. `null` / null fields → "missing" 400. Wrong types → 400 (the message exposes the Go type name). Duplicate and case-insensitive keys: last wins. That is harmless because the caller is trusted.
- allowed_cidrs: `Unmap()` handles IPv4-mapped clients. Zoned and IPv6-mapped config entries fail closed.
- Unknown paths → 404 JSON. HEAD/PUT/OPTIONS → 405 with `Allow`. `//counters` → 301 → works. Metric labels: route is bounded to 4 values, and status/level/flow/op/result are bounded.
- Log injection: the app log uses slog (values quoted). The counting text log can't receive newlines or ESC: addresses reject Cc/whitespace and trace_id strips controls. trace_id still allows spaces, `>` and non-ASCII, which deviates from the plan's "printable ASCII" but has low impact because mizu generates it.
- Redis key injection: an address has exactly one `@`, the domain is restricted to `[a-z0-9.-]` and the org id is numeric, so no collision across customers or levels is possible. `/counters?domain=` is unvalidated but read-only and confined to `cnt/D/…`.
- Notify URL contract: all six params are in the default templates. `id` is the address for A, the domain for D and the org id for O. Values are QueryEscaped. `/api/limits` routes to `InternalApi.NotificationController` with no route collision. `accepts json` is fine because there is no Accept header. COUNT is now the new value (= limit) where Lua sent limit−1; the plan intends this, and the web app only uses it in canary text.
- Exempt via `/out` mail_from: mitigated by mizu `ResolveSender` / rcptd `allowed_from`. On `/in` it behaves the same as Lua.
- go-redis txPipeline: read errors are not retried, so there is no double-increment.
- Config: db≠0 is rejected, relative `domains_cdb` is resolved against the config dir, env overrides apply only when non-empty, notify URLs are checked for an http(s) scheme.
- counterctl: config discovery, env token override, `:8095` → `http://localhost:8095`, trailing slash on `--url` works, `--url …/api` → 404 (expected), `!` uses ≥ (no zero limits exist in the CDB), exit codes are 0/1.
- `/health` checks CDB readability and a 200 ms write-side PING. The README's "drain ≤ 30 s" matches the code; the plan said 10+10 s, a documentation-only difference.
- CDB fixture: 93,549 rows, every row has all 6 base keys, there are 22,995 lowercase overrides, no non-numeric org ids, no zero or negative limits, and 112 punycode domains.