mxhop

Concepts · queue

Two files per message, in a directory you can read.

The spool is the only state mxhop has. It is plain files, on purpose: you can inspect it with ls, back it up with cp and understand it at three in the morning without a client library.

When a message is queued at all

Most messages are never written to disk. mxhop attempts delivery before answering end of DATA, so the usual path is: take the message, deliver it, pass the destination's reply back to the sender. Nothing touched the spool.

A message goes to the spool only when the destination deferred it — a 4xx, a timeout, a connection that failed. At that moment mxhop answers the sender 250, which means it has taken responsibility, so from then on losing it would be a lie.

destination said 2xxSender gets 250. Nothing queued.
destination said 5xxSender gets that refusal, in the destination's own words. Nothing queued, no bounce invented.
destination said 4xx, or did not answerMessage written to the spool, sender gets 250, retries begin.

Layout

/var/lib/mxhop/spool/
├── tmp/        being written. renamed out of here when complete
├── queue/      waiting for the next attempt
└── frozen/     given up on. nothing here is retried again

Each message is two files sharing an id:

queue/1795110377370-000003-0007.eml    the message, byte for byte
queue/1795110377370-000003-0007.json   everything mxhop knows about it

Written to tmp/, then renamed. A rename within a filesystem is atomic, so a crash mid-write leaves a file in tmp/ that is ignored on restart. Nothing in queue/ is ever half a message — that is the whole reason for the two-step.

The id is sortable: milliseconds, then a counter, then a per-process sequence. ls queue/ is in arrival order.

The sidecar, field by field

jq . /var/lib/mxhop/spool/queue/1795110377370-000003-0007.json
FieldTypeMeaning
idstringSame as the filename. The thread through every log line about this message.
fromstring | nullEnvelope sender, after SRS rewriting. null is a legitimate null sender, which is how bounces identify themselves — and the reason no report is ever sent about one.
toarrayResolved destination mailboxes still outstanding. Shrinks as copies are delivered; an entry disappearing means that copy is done.
helostringWhat the sending host called itself.
peerstringAddress it connected from.
received_at_unix_msintegerWhen it arrived. With lifetime_hours, this decides when the message is given up on.
sizeintegerBytes. A count, never content.
receivedstringThe Received: trace mxhop prepended.
authentication_resultsstringThe stamped verdict. Empty when [auth] is off.
arcstringThe ARC header set for this hop. Empty when [arc] is not configured.
last_replystring | nullWhat the destination said most recently. Read this first when a message is stuck — it is usually the whole answer.
attemptsintegerDeliveries tried. Indexes into retry_minutes.
next_attempt_unix_msintegerWhen the runner will next pick it up.
viaobjectPer-domain via hosts, so a queued message keeps routing the way it was routed when it arrived — not the way the config says now.
sender_authenticatedbooleanWhether the sender was proved: SPF pass, or an aligned DKIM pass. This single field decides whether a delivery report may be sent.

Retries

A runner walks queue/ every sweep_seconds (default 60) and attempts any message whose next_attempt_unix_ms has passed. The waits come from retry_minutes, and the last entry repeats:

# the default schedule
retry_minutes = [1, 5, 15, 60, 240]
lifetime_hours = 48

# which means, counting from arrival:
+1m   +6m   +21m   +1h21m   +5h21m   +9h21m   +13h21m   ...
                                      every 4 hours from here,
                                      until 48h, then frozen

The one-minute first retry is for greylisting: a deliberate first-attempt rejection used by many servers, where the same message a minute later is accepted. Without a short first retry, greylisted mail is delayed by however long your second interval is.

BoundValueWhy
intervals1–32The last repeats, so a longer list is a schedule nobody reads.
each interval1–1440 minNot 0: retrying in the same instant a destination said "later" gets you rate-limited. Not more than a day: a day between attempts inside a lifetime measured in hours retries almost nothing.
lifetime_hours1–72048 is conventional and what senders expect.
sweep_seconds1–3600Granularity of noticing, not of retrying.

Duplicates are deliberate

If a delivery succeeds but the queue entry cannot then be removed — a full disk, a permissions change, a crash in the microsecond between — the message is attempted again on the next sweep, and the destination gets a second copy.

This is a chosen trade, not an oversight. Nothing on disk distinguishes "delivered but not cleaned up" from "never attempted." Of the two possible mistakes, a reader can delete a duplicate; nobody can read a message that is gone.

In practice it is rare, and the cause is almost always a spool that has run out of disk — which the spool line in mxhop health reports.

frozen/, and delivery reports

When lifetime_hours runs out, the message moves to frozen/ and is never retried. mxhop then tries to tell the sender, and this is the part with a rule attached.

A report (RFC 3464) is sent only when the sender can be proved: SPF passed, or DKIM passed and aligned with the From: domain. Otherwise nothing is sent and the suppression is counted.

Suppressed whenBecause
the envelope sender is nullIt is itself a bounce. A bounce about a bounce is a loop.
the sender was not authenticatedThe address is unproved and may be forged. A report sent to it is backscatter — spam that you are sending, to a victim who never wrote to you.
the sender is not an addressNowhere to send it.

This means some senders are never told their mail failed. That is the correct behaviour: the alternative is a backscatter source, which gets your host blocklisted and genuinely harms the forged victim. Whoever sent unauthenticated mail to your domain has a bigger problem than the missing report.

Files in frozen/ are kept, not deleted — the message is still readable. Pruning them is your decision; mxhop will not act on them again.

Working with the queue by hand

There is no mxhop queue subcommand. The queue is files, the shell is the interface, and that is the design rather than a gap.

# how much is waiting
ls /var/lib/mxhop/spool/queue/*.eml 2>/dev/null | wc -l

# why it is waiting — the one question worth asking first
jq -r '[.id, .attempts, .last_reply] | @tsv' /var/lib/mxhop/spool/queue/*.json
1795110377370-000003-0007  3  451 4.3.2 Try again later
1795110412901-000004-0001  7  421 4.7.0 Too many connections

# what has been given up on
ls /var/lib/mxhop/spool/frozen/*.eml | wc -l

# retry a frozen message: write the sidecar first, then move the body
ID=1795110377370-000003-0007
cd /var/lib/mxhop/spool
jq '.attempts = 0 | .next_attempt_unix_ms = 0' frozen/$ID.json > queue/$ID.json
mv frozen/$ID.eml queue/

Write the sidecar before moving the .eml. The runner keys off the .eml, so a message whose body is in queue/ without its sidecar is one the runner cannot act on. Doing it in this order means the pair is complete the instant it becomes visible.

Editing the spool of a running mxhop is safe for a message not currently being attempted, and the window is small. If you are moving many, stop the service first.