The Forge — Cold Anvil Studios

What it is

The Forge is the system that runs the studio's agents. I ask for something in plain language, the agents take it from there, and I approve at the moments that actually need me. It's built around one belief about AI agents: most of the difficulty sits outside the model. It's in knowing what an agent actually did, catching it when it goes wrong, and keeping the important decisions with a human.

The studio's projects and workflow live in Linear, and the Forge is tightly integrated with it: work can start from a Linear ticket, and a run's progress lands back on the ticket it came from.

None of the machinery below makes an agent clever. It makes an agent's failures visible early, which in practice matters more.

The Forge's cockpit interface: a dark operator console showing the run lifecycle from Trigger to Product decision, a plain-language request box, run observability details, and several controls visibly labelled as locked or read-only. — The cockpit, July 2026, captured by the agent that tests it during a healthy run check. The locked and read-only labels are the honest state of a system in development: what isn't ready yet says so on the surface. Click to expand.

A run, start to finish

Every piece of work moves through the same lifecycle, and every run ends in a named state: shipped, handed back to me, stopped or rejected. There's no state where work just trails off.

01

The askA request in plain language, typed directly or picked up from a Linear ticket, becomes a task. If it's ambiguous in a way that matters, the run pauses and asks, rather than guessing.

02

ContextThe agents gather what the work needs to be done properly: the relevant code, the conventions, what past runs learned. Reading first is cheaper than guessing.

03

The planWritten down before any work starts, with the expected cost shown up front. Changing scope mid-run means revising the plan, visibly. There's no quiet drift.

04

The plan gateTwo separate checks before anything gets built. Details below; it's the cheapest place to catch a mistake.

05

The buildAgents do the work inside a sandbox, and every command they run is recorded, allowed or refused.

06

The work gateTwo more checks, on the finished work this time. If it fails three rounds of revision, the run stops itself and asks for help instead of grinding.

07

My reviewShip it, send it back, or close it. Nothing merges without a human deciding it should.

08

The recordThe whole run is sealed into a log that can be read back later, along with anything it learned that's worth keeping.

How work gets divided

The Forge isn't one agent grinding through everything. Annie runs the show: she breaks an approved plan into pieces of work and hands each piece to a worker. A worker gets one job, does it in steps, and reports back.

The division is visible. Every worker's progress shows step by step, in plain language, and a failure points at the exact step that failed and why. In the capture below, one worker is mid-job whilst another has failed its safety check because a snapshot was missing. That's the level of detail I get without asking for it.

The Forge's worker activity screen showing two workers: one running a job called adding the visual comparison with its steps shown as progress bars in plain language, and one failed on a job called checking the undo is safe, with the failed step highlighted in red and the reason given as a missing snapshot. — Worker activity, June 2026, showing demo data. One worker running, one failed with the failing step named. The steps read in plain language because I have to read them. Click to expand.

Before any work: the plan gets checked twice

A plan has two ways to be wrong. It can be the wrong thing to build, or the right thing engineered badly. Those need different reviewers, so the Forge uses two.

I approve the intent: is this actually what I asked for? A separate model then reviews the engineering: will this actually work in practice? That reviewer is deliberately powerless beyond its opinion. It can't rewrite the plan, can't approve it and can't start the work. It can only say what's wrong with it.

The outcome is that bad plans die on paper, where they cost a review instead of a build.

After the work: checked twice again

Finished work has to pass two checks. The first is technical: does it actually work? The second is product: does it serve the person it's for? Both have to pass, and a product fail is never quietly overridden.

The technical check has one unusual rule. The checker receives the plan and the finished work, and never the builder's explanation of it.

The checker sees

The plan The finished work

The checker never sees

The builder's reasoning

The checker works in isolation. The same idea as JackBench's blind judging, applied inside the Forge.

Why that matters: an explanation can be persuasive whilst the work is wrong. A checker that never hears the explanation can't be talked into agreeing. It judges the work itself, however confidently the builder might have described it.

Everything leaves a record

Everything that happens in a run is written to one log, and only one part of the system is allowed to write it. Dashboards and monitoring read from that log. They can watch a run, and they can never steer one.

Even refusals are records. When the sandbox blocks a command, that's written down as a visible event with a named reason, never swallowed silently. If something looks wrong a week later, the question is what the log says rather than what anyone remembers.

This is the same receipts discipline JackBench applies to benchmark evidence, applied to everyday work.

Memory

The studio keeps four separate kinds of memory, because "what we always do" and "what happened in this run" shouldn't live in the same drawer.

What the studio knows

Shared knowledge: conventions, voice, how things are done here. Curated by a human, readable by every agent.

What each project taught

Per-project learnings that accumulate from the run records: typical costs, common failure modes, patterns that held up.

The run's scratchpad

Working memory for one run. It lives while the run lives, and anything worth keeping gets saved deliberately.

What each agent is learning

Per-agent patterns across projects. Honestly the weakest of the four, and flagged for review once there's enough evidence to judge it.

One rule holds it together: agents can suggest additions to the shared knowledge, and a human always applies them. Nothing writes itself into "what the studio knows".

The sandbox

Agents build inside an operating-system sandbox, and the sandbox fails closed: if it can't be set up properly, the command is refused rather than run unprotected. Some things are never allowed at all, no matter how the system is configured. Reading credentials is one. Broad deletion is another.

When something genuinely needs my sign-off, the approval is for one named action in one named run, used once. There are no standing permissions to accumulate and forget about. The outcome is boring failure: when an agent goes wrong, it goes wrong inside a box, visibly, and stops.

Model choice

Model choice sits with me, per run. The default is frugal: small local models on the studio's own hardware do the everyday work, and a run reaches for bigger cloud models only when the work genuinely needs them. The cockpit states the policy right on the trigger: local workers first, cloud only on escalation.

Wanting the best available brains for something is an explicit choice with the price on the label. The cost of using the big cloud models is shown at plan approval, before I say yes. And there's a studio-wide switch for it, called Overdrive: best-models mode, designed and visible in the settings, and deliberately locked until its guard rails exist, a daily budget cap among them. Every use of a frontier cloud model is recorded in the run's evidence.

The Forge's settings screen showing Overdrive, a best-models mode, marked as planned and locked with a list of what's needed first: explicit user enablement, a model availability check and a daily budget cap guard. A sign-ins panel lists credential names only, with a note that secrets never appear in the app. — Settings, June 2026. Overdrive is designed, visible and locked until its guard rails exist. The sign-ins panel shows names only; the app never displays a secret. Click to expand.

The money

Every run gets a spending cap at plan approval, and the studio has a daily cap over all of it. Money is reserved before it's spent, then settled against provider receipts afterwards, so the ledger deals in what actually happened.

When the ledger can't be sure, it says so and stops. If a payment is stuck or a record is in an unknown state, the ledger treats the whole day as uncertain. An uncertain day means it can't know how much money is left, so it blocks new spending instead of estimating. Clearing a stuck record is a human action that releases the money and records who did it. It never invents a cost.

Open source, eventually

The intent is to open-source The Forge, as a harness evaluated properly for near-frontier models and small local models. There's no date attached, and there won't be one until the evaluation numbers justify it.

One thing worth stating plainly: the system on this page is built around my use cases, and mine are complex. The open-source version will be stripped back. Simpler by default, with the heavier machinery optional, so you can build it up for your needs rather than untangle it from mine.

What "evaluated properly" means here is the same treatment the harnesses in the studies get: the same tasks, the same strict automated grading, and results published with their error bars. When The Forge has results of that kind, they'll be on the research pages, whether or not they flatter it.