KarstenSamaschke
Back to all writing

Field notes

One Day, 15 Merges: The VanillaCore Agent Harness Learned to Survive a Restart

Fifteen merges in one day taught the VanillaCore Agent Harness to resume checkpointed work after a restart, run bounded subagents and reload tools between turns. Each change also documents where it stops.

Karsten Samaschke · · 14 min read

One Day, 15 Merges: The VanillaCore Agent Harness Learned to Survive a Restart

On Sunday between 08:00 and 20:00 Berlin time, the public VanillaCore Agent Harness merged 15 pull requests into harness/core, published 14 release images and gained two new public repositories next to it. The changes with the most consequence are about what happens when things go wrong: a restart in the middle of a task, a stop command that arrives at the wrong moment, a tool that needs replacing while the agent is still working.

A closed dark notebook with a light-blue ribbon bookmark next to an unplugged power cable on a dark desk

If you leave Hermes Agent, Claude Code, Codex or OpenClaw running on something longer than a single answer, you know the uncomfortable part. A pod gets replaced, a process crashes, and either the work is gone or the agent starts again without knowing what it already did. My thesis for this day of Harness work: an agent that resumes work on its own is only acceptable if it also tells you, in writing, which effects it cannot take back. Most of these 15 merges add a capability and then document precisely where it stops. Note that everything below describes Harness source changes; none of it says anything about features or compatibility in the four products I just named, and nothing here proves that any installation runs these versions.

Picking up after a restart

#73, merged at 13:28, is the core of the day. The Harness now stores the original accepted chat request separately from its inbox, together with a task checkpoint: the last continuation decision, the concrete next step, the chain depth, an inherited numeric deadline and a host-assigned task identity. When the Harness comes back up and its channels are ready, an active checkpoint with a concrete next step can reconstruct exactly one continuation, and only if no current or queued input exists for that conversation. The ordinary durable inbox takes precedence, global pause still blocks execution, and waiting, blocked, completed, stopped, expired or exhausted checkpoints start nothing (task recovery docs at the window-end commit).

The documentation is the part I would print out. Recovery does not invent a next step; a successful turn without an explicit or semantic continuation decision simply becomes "waiting". Checkpoints only exist for requests processed by this version, and old transcripts are not converted into permission to work. Execution is at-least-once: a tool may have committed an external effect before its result was saved, so the recovery prompt tells the agent to verify completed side effects before acting, and the docs say plainly that this cannot undo effects or provide exactly-once behavior for arbitrary tools. They also state the operational precondition, which is that storage has to survive pod replacement and the Kubernetes termination grace has to exceed the drain budget. An agent runtime that writes this down is more trustworthy than one that promises flawless recovery and lets you discover the duplicate invoice yourself.

Deciding whether the work is finished

Recovery needs to know whether there is something left to recover. #63 adds a semantic completion fallback: after a successful working turn without an explicit continuation checkpoint, a tool-free, ephemeral reviewer session classifies whether the task is finished. Explicit waiting, blocked or completed plans are never overridden, and API runs, nightly controller work, paused instances, queued work, cancelled turns and exhausted chains are excluded. As merged, #63 only ran when installation configuration supplied a task scope and deadline for a specific conversation.

#64, thirteen minutes later, widens that. Without a configured grant, the Harness now uses the original accepted chat message as the task anchor, and generated follow-ups inherit that anchor instead of the assistant's own text. At the window-end commit, the CLI wiring passes the assessor for every Harness instance and falls back to that chat anchor whenever no installation grant exists; I found no separate switch for the chat path. So "opt-in" describes the installation-grant path, not every chat conversation. The safeguards are in the completion-check docs: the classifier must tell work requests apart from questions, status requests and quoted history, it is instructed to return "uncertain" instead of guessing when a task mentions a time limit without a numeric grant, invalid or failed assessments do not continue, and there is a ten-second deadline. The docs also call the whole thing model-advisory, "not a formal authorization proof", with no claim of semantic accuracy. I agree with that label, and I would add that a model judging whether another model is done remains a judgment call, however well it is fenced.

#70 closes a gap in that fence. Numeric grant deadlines are now persisted on generated follow-ups, inherited without extension (a chain keeps the earliest deadline) and checked again right before execution, including after asynchronous runtime setup. Expiry prevents a new working prompt; it does not interrupt a prompt that is already running. A fresh request from the operator still runs under an expired grant, which is the correct behavior: the deadline limits what the agent does on its own, not what you ask it to do.

Stop means stop

Three fixes deal with cancellation arriving at an awkward moment. #66 rechecks a /stop that lands during completion assessment, so the turn is recorded as cancelled, not successful, and no continuation is scheduled. #69 honors a stop that arrives even later, after memory capture and telemetry: already committed memory is kept and not repeated, and in Matrix the "cancelled" state replaces "completed" and removes the outdated reaction. #65 serializes Matrix typing requests per room, so a slow "stopped typing" request from one turn can no longer switch off the indicator of the next turn. None of these is a headline on its own; together they decide whether "stop" in a chat window is a command or a suggestion.

Bounded subagents

#74 gives the main agent a plugin_subagent tool and operators an admin-only /subagent command to start, list, inspect, cancel and override delegated child sessions. A handover must spell out seven fields explicitly: task, context, constraints, expected result, acceptance criteria, model and reasoning level, each up to 8,000 characters. The main agent picks model and reasoning per child, and an operator override saved per parent conversation wins for future starts. Difficulty profiles and automatic model selection are not implemented (subagent docs at the window-end commit).

The bounds are tight on purpose. Each child gets one API turn (which may contain several tool calls), no automatic continuation and no children of its own; loop, continuation, delegation and improvement tools are denied inside children. At most four children may be queued or running at once across the Harness, 200 records are retained without automatic pruning, and a ten-minute absolute deadline counts queue and offline time too. A parent /stop cancels its children. Approval prompts go to the parent's trusted operator destination and are never recorded as child results; without such a destination, approval is denied. After a restart, a start with no surviving input or result is marked failed instead of being silently resubmitted.

The docs put the main limitation in bold, and so do I: these sessions are not security sandboxes. Children share the runtime, filesystem, credentials, trusted plugins and repository with the parent, and handover constraints are instructions, not isolation. Cancellation is cooperative, so it cannot undo external effects or guarantee that an uncooperative tool stops immediately. Results are polled with list and result; a finished child does not wake or notify its parent. If you compare this with subagent features elsewhere, compare exactly these points: what the child can touch, who sees its approval prompts, and what happens to it after a crash.

Changing tools without a restart

Four merges let an operator change integrations while the agent keeps running, each with a different trust level.

#67 handles MCP. /reload-mcp, an alias of /reload-skills, schedules a reload of the session's SDK resources at the next prompt boundary without interrupting a running turn. MCP-enabled sessions also compare the content of user and project mcp.json before each prompt and reload when it changes. This is turn-boundary discovery, not a background watcher, and credential changes that only touch the environment are not detected. Invalid JSON blocks the prompt before the previous connections are closed, project-local configuration only loads if the project has a stored decision to trust it, and chat cannot grant that trust. The reload also re-reads trusted runtime extensions and settings, but not general Harness plugins, and the MCP docs explicitly rule out transactional rollback for other reload failures.

#71 adds reloadableToolPlugins and an admin-only /tool-plugin status|reload. Trusted ESM modules that register only tools can be replaced by a new generation: calls already running keep the old generation, its cleanup runs after its last call, new calls use the replacement, and an invalid candidate leaves the previous tools active. Tool names, descriptions and schemas must stay the same; anything else needs a restart. #72 makes this automatic if you set autoReloadToolPlugins: true (the code default is false): entry files are compared before each working turn, and a broken replacement keeps the old tools but fails the new turn before the model runs. The plugin docs list the costs. Changes in imported dependencies are not detected, Node never unloads old modules so a restart is required after 50 load attempts, and these modules keep full process privileges with no timeout, no isolation and no rollback of side effects.

#68 is the counterweight: experimental WebAssembly tools, configured by infrastructure under wasmTools. Modules may have no imports at all, WASI included, and get no environment, history, credentials, filesystem, network or host callbacks. Input and output are UTF-8 strings of at most 64 KiB each, module files at most 1 MiB, declared memory at most 16 MiB, at most four calls run at once across all Wasm tools, and each call has a deadline between 1 ms and 30 s (default 5 s). The module file is read per invocation, so replacing it takes effect on the next call. The Wasm docs state that this is not OS or process isolation, that compilation and V8 internals sit outside the memory accounting, and that hostile, unreviewed modules should not run on the strength of this API alone. I like the honesty; it means the real boundary is still your container limits.

Pinned skill bundles

#62, the first merge of the window at 08:15, lets installation YAML pin a skill bundle by HTTPS URL and SHA-256. The bundle may only contain SKILL.md text, up to 64 skills and 512 KiB, downloaded without redirects within 15 seconds. Validated files land in fresh directories, a failed activation restores the previous skill root, and a restart rebuilds the files only if the saved bundle still matches the configured pin. The update docs close with the right sentence: "This is not a general-purpose remote updater." No scripts, no plugin or MCP updates, no sandbox and no transaction across components; skill instructions remain trusted content, and a digest does not make them safe. Old generation directories are kept and not cleaned up automatically. The same PR swapped the tool-start marker in chat progress for an hourglass, 35 minutes after the morning's last merge, #61, had swapped it for a middle dot.

Knowing what it costs

#75 brings back /usage, with /tokens as an alias, as an ordinary non-admin command listed in /help. It reports the current runtime session's cumulative input and output tokens, cache reads and writes, total tokens and an estimated USD cost. Numbers are partial while work is running, and counters a runtime does not supply show as "unknown" instead of zero. The usage docs exclude account billing, an all-agent ledger and a combined parent and subagent count, and they note that a zero estimate does not prove a provider charged nothing. Small command, real value: if a subagent feature lands on the same day, you want to see what a session costs.

Review boundaries

The last merge, #76 at 18:04, fixes a problem with the mandatory code review. Before a working turn, a review task that already passed may now be archived when the worktree is clean, HEAD equals the configured remote-tracking base and the old base is its ancestor. That records an accepted upstream boundary, not approval of historical commits: the new state starts pending with no approved fingerprint. Missing upstream, local-only commits, dirty work and unresolved or escalated tasks cannot move the base or reset the review budget, and new working-turn edits stay subject to the configured mandatory review policy with the unchanged 512 KiB limit (review docs). Read this as narrower scope, not as review being relaxed.

Two new public repositories

Shortly after 12:14, two repositories appeared in the public harness organization. harness/plugins received its first content through PR #1 at 12:46: the context, continuation, gateway, loop and review adapters, moved as source. I compared the five files by SHA-256, and they are byte-identical to the versions in core at the commit the PR names (1105f0b) and still at the window-end commit. Runtime engines, policy enforcement and the plugin API stay in core, and so does memory. The README describes the integration: core records an immutable revision plus per-file SHA-256 in plugins.lock.json, materializes the files locally, and never fetches or executes plugin source dynamically at runtime. At the window-end commit, the public core main contains no plugins.lock.json yet, and the five adapters still live in its src/ directory, so the materialization described there is not visible in public core within this window.

harness/deployment-base got PR #1 at 12:19: one versioned Kubernetes agent template with @AGENT@ and @DISPLAY_NAME@ parameters and no credentials. Consumers are supposed to materialize it locally and pin revision and SHA-256 in deployment-base.lock.json, so Argo never fetches a remote Kustomize base. That makes template consumption reproducible, which is not the same as a fully pinned deployment: the template still references the core image without a tag. Neither repository adds a new end-user capability, and a template is not evidence that anything was deployed.

What merged and what shipped

All 15 core merges, #62 to #76, are on main within the window; the window-end head is 0dabd74 from #76. Fourteen release images were published in the window, v0.1.16 at 08:27 through v0.1.29 at 18:07, each built from one merge commit on main. Release ancestry shows that #63 has no release of its own; it first shipped together with #64 in v0.1.17. Task recovery is in v0.1.26, subagents in v0.1.27, /usage in v0.1.28. The two new repositories published no releases. "Shipped" means a published image and nothing more; the PRs repeatedly state that no deployment or rollout was performed, and this post makes no claim that any installation runs these versions. I also have no benchmark or live model test behind any of this; the PR bodies report test, typecheck and build runs and reviews, which is the evidence available.

Scope of this post

This post covers 25 October 2026 from 08:00 to 20:00 Berlin time (CET), which is 07:00 to 19:00 UTC; all times above are Berlin time. It follows the earlier post on the night before, which ended with #61. When checked on 25 October 2026, the public harness organization listed three repositories, core, plugins and deployment-base; that is a statement about the currently listed public inventory, not about private repositories or past visibility. Queries for open pull requests in all three repositories returned none when checked, which says nothing about unpublished work.

Verdict

Restart recovery, subagents and hot reload are the features every agent runtime advertises sooner or later, and they are also where most of them get vague. The Harness spent this Sunday building all three and attaching a limits section to each: at-least-once instead of exactly-once, shared credentials instead of sandbox, trusted code instead of isolation, an estimate instead of a bill. That list is the most useful artifact of the day, and it is the list I would demand from Hermes, Claude Code, Codex and OpenClaw before I let any of them run unattended.

Sources used