Hermes Output: Streaming vs Final Blocks, When to Pick Which

Pick the wrong output shape and you either burn the user’s wait time in silence or you fragment an audit log into noise. Hermes gives you two shapes — streamed token-by-token, and a single final block — and the surface you send the response to decides which one is correct. The choice is not aesthetic. It is a property of the channel.

Two output shapes

Hermes returns responses in two shapes, chosen at chat invocation:

• Streaming — tokens arrive one at a time as the model generates them. The user sees letters accumulate, like a typewriter effect.
• Final block — a single complete response arrives after the model finishes.

Each surface has a strong default:

Surface	Default	Why
CLI (hermes chat -q "...")	Final block	Single-shot batch; no need for typewriter
Telegram gateway	Streaming	The chat feels alive; long responses don’t time out
Mobile push notification	Final block	Push notifications collapse to the end
Web UI on agenticbotsitter.com	Streaming	Reader sees structure as it forms
Cron pipeline output	Final block	The downstream step waits for the whole anyway
Cron pipeline intermediate logs	Streaming	Watch in real time as the model thinks

The defaults are not arbitrary. Each one matches the failure mode of the channel: a Telegram chat that stays silent for 30 seconds is broken, but a push notification that streams half a paragraph before the notification service collapses it is also broken — they fail in opposite directions. The CLI is the easiest case to reason about because it has no interactivity to preserve — you asked a question, you got an answer, the terminal printed it. Streaming the CLI output would still work, but it would feel busy rather than alive, and operators running batched scripts usually want the answer once, at the end, so they can pipe it.

Cron pipelines are worth distinguishing carefully. The “Cron pipeline output” row in the table above refers to whatever the cron step writes to disk or passes to the next step — that should be a final block, because the next step cannot act on a half-written file. The “Cron pipeline intermediate logs” row refers to whatever the operator watches in their terminal while the cron is running — there, streaming is correct, because the operator is watching a human in real time and “feels alive” applies. The same cron can have both: a final-block payload to disk, and a streaming mirror to the operator’s tail.

The decision matrix

Three axes drive the pick:

1. Is the user waiting? If yes → stream.
2. Is the downstream waiting? If yes → final block (the model is done before downstream picks it up).
3. Is the response so long it might time out? If yes → stream to avoid timeout.

For Telegram, “is the user waiting” wins. For crons, “is the downstream waiting” wins. For Caddy-served static HTML via terminal agents, the answer depends on the agent.

The three axes usually agree. The cases worth thinking about are the ones where they disagree — push notifications, where the channel is technically waiting (the user sees the lock screen) but the medium is single-shot; long-form cron outputs, where the downstream is waiting but the upstream timeout is the real constraint. In both cases, follow the medium, not the user.

A worked example helps. Take the three surfaces above and run them through axes “1”, “2”, and “3”:

• Telegram: axis 1 yes (user waiting), axis 2 no (no downstream), axis 3 sometimes (long answers time out at 30s). All three point to streaming. Pick streaming.
• Cron pipeline output: axis 1 no (no human in the loop), axis 2 yes (the next step is waiting for the file), axis 3 no (the file write is local). Pick final block.
• Push notification: axis 1 yes-ish (the user will see the lock screen), axis 2 no (no machine downstream), axis 3 no (push payloads are short). The medium is single-shot, so even though axis 1 says stream, axis 2’s absence combined with the medium’s collapse-to-end behavior says final block. Pick final block.

When the axes disagree, the medium wins. The medium is the constraint the response must fit inside, and no amount of streaming can fix a medium that does not preserve partial state.

The streaming-failure case

Streaming responses can fail mid-stream. The user sees the partial response and the connection drops. Recovery: the gateway resends the last half. The browser/user-on-some-app sees the full response eventually.

This is the price for “feels alive.” Pick streaming for surfaces where partial-failure recovery is cheap (interactive chat). Pick final block for surfaces where a dropped partial is a real failure (push, audit log).

The retry-on-mid-stream-failure behavior is what makes streaming safe for Telegram and dangerous for audit logs. Telegram’s bot API is idempotent within a chat session — the gateway can edit the same message in place as more tokens arrive, and if the connection drops, the next retry picks up where the partial left off. An audit log is not idempotent. A second logger.info() call after a partial failure is not “the rest of the line” — it is a second line, and your log is now lying.

The same reasoning extends to anything that has already been written down. Email digests, RSS feed items, and webhook payloads are all “the record exists, you cannot amend it” mediums. The first byte a webhook receiver persists is the byte it has to keep, even if the connection drops ten milliseconds later. If your downstream is on the receiving end of a fire-and-forget POST, do not stream into it. Buffer locally, finish the response, then send the completed payload in one shot.

There is also a cost asymmetry worth naming. Streaming forces the gateway to hold a long-lived connection open across the full generation. If your generation takes 60 seconds, that connection is held for 60 seconds. A final-block request can return as soon as the model finishes, but the model still has to finish — there is no latency saved at the model layer, only at the connection layer. Do not stream “to feel faster.” The model speed is the model speed.

What not to do

• Don’t stream to a push notification — the notification service delivers once, with whatever arrived first.
• Don’t send final block to a Telegram chat that expects interactivity — the user waits 30s of silence before the answer appears.
• Don’t stream an audit log entry — the log collector needs the whole record at once.
• Don’t pick streaming just because it “feels modern” — pick it for the right surface, not for vibes.

The four rules above are negative space; the “What not to do” list is where the framework gets teeth. The first three are cases where the channel mechanics punish the wrong shape — push notifications collapse, Telegram times out, logs split. The fourth is the most common operator mistake: reaching for streaming because it is the default in modern chat UIs and looks impressive in a demo. A streaming response to an internal cron job that writes to S3 looks impressive for the first 30 seconds, then it fragments the JSON file the next step is trying to parse. Pick the shape that the channel needs, not the shape that demos well.

Verification checklist

#	Question	Rule
1	Is the user waiting?	Stream
2	Is the downstream waiting for a final block?	Final block
3	Will partial streaming fragment the output medium?	Final block

If the answer to question 1 is yes and the answer to question 3 is no, stream. If the answer to question 2 is yes or the answer to question 3 is yes, take the final block. That is the whole decision tree. The rest is remembering that the medium has an opinion.

Sources

• Hermes Agent API reference
• Hermes gateway docs
• ABS companion: Hermes Protocol: The System Prompt, the Context Window, and the Rule Boundary
• ABS companion: Hermes Agent Telegram Gateway Setup

Last verified: 2026-07-14 against Hermes v0.18.2 (2026.7.7.2).