Cold Anvil Studios

Cold Anvil Studios — Research2026-08-17T00:00:00Zhttps://coldanvil.com/Jack TylerFirst Pi–OpenCode Baseline: Kimi K3 Paid 4× More for the Same Qualityhttps://coldanvil.com/research/first-pi-opencode-baseline/2026-08-15T00:00:00Z

Three near-frontier models under Pi and OpenCode: a pooled null result, and Kimi K3 tying on quality whilst spending 4× more under OpenCode. The JackBench programme's first published study.

Blinded Judge-Panel Qualification: LLM Judges Sit an Exam Before They Score Anythinghttps://coldanvil.com/research/blinded-judge-panel-qualification/2026-08-16T00:00:00Z

Frontier models sit a blind exam against locked known answers before they may judge anything in JackBench. Five judges from five labs have qualified; one local candidate was refused.

Local Qwen 3.6 Harness Baseline: One Harness Swap Moved the Same Model 19.4 Pointshttps://coldanvil.com/research/local-qwen-36-harness-baseline/2026-08-15T00:00:00Z

Qwen 3.6 27B solved 103/144 tasks in Pi and 75/144 in OpenCode on identical tasks with deterministic grading, at zero provider cost. A research note from the JackBench programme.

Qwen 3.8 Regular versus Uncensored: A Pre-Registered Security Comparisonhttps://coldanvil.com/research/qwen-regular-vs-uncensored-security/2026-08-17T00:00:00Z

Two builds of the same local model, one regular and one uncensored, will face 216 authorised security sessions in a sealed sandbox. Hypothesis and method locked; results coming soon.

Seven-Model Pi–OpenCode Baseline: Harness Effects Are Real and Model-Specifichttps://coldanvil.com/research/seven-model-pi-opencode-baseline/2026-08-15T00:00:00Z

Seven hosted models on 72 shared tasks in Pi and OpenCode: one Holm-surviving harness effect (GPT-5.6 Sol, +19.4 points in Pi) and no universal harness winner. A research note from the JackBench programme.