Contents · 1 / 6
Qwen 3.8 Regular versus Uncensored: A Pre-Registered Security Comparison
Qwen 3.8 27B is about as capable as a small local model gets right now, and the studio runs two builds of it: the regular release and an uncensored variant. The question is simple: which build is better at security work? This study measures the answer rather than guessing at it. Both builds face 216 sessions on a throwaway copy of Cold Anvil's own defences, some probing a component for the way in (red team), some handed a planted vulnerability to find and repair (blue team). The hypothesis, method and scoring are locked before a single result exists, and this page holds that design so the target can't be quietly changed once results start landing. The findings appear here when the run is sealed. One limit up front: the two builds differ in more than censoring (they use different quantisation methods), so this measures the deployed bundles as they actually run, and it will not claim to isolate what uncensoring alone does.
Why this study
Security work is an awkward fit for language models. The same refusal training that stops a model helping an attacker can also get in the way of legitimate defensive work: auditing an account system, probing your own fixtures for weaknesses, repairing the hole you just found. The uncensored-model community says the safety training is the problem. The labs say the capability loss is a myth. There's no clean measurement either way, so this study makes one.
The studio already runs both builds, on the kind of work its products depend on. So before either is trusted with real security work, both sit the same exam.
What's being compared
Two checkpoint bundles of Qwen 3.8 27B, exactly as deployed on the studio's fleet: the regular release in AWQ INT4 quantisation, and an uncensored variant in ModelOpt NVFP4. Bundle is the honest word. The two differ in censoring and in quantisation method, and there is no way to unpick those two factors with these artefacts. The study is registered as a practical comparison of the deployed bundles, with an analysis boundary that forbids attributing the difference to uncensoring alone.
That confound is stated on purpose. Ignoring it would let the study claim more than these artefacts can support.
How it runs
216 sessions against 36 synthetic components, each one bound and qualified by an automated readiness check and fully resettable, so every session starts from a known state. Each session takes one of two roles.
Probes a component for a way in, under an explicit allowlist. It wins by finding the vulnerability.
Takes a component with a planted vulnerability and has to fix it. It wins with a clean repair that stays in bounds.
The 36 components span 12 families of security boundary, the kinds of hole that show up in real breaches. They group into four sorts of job.
Keeping secrets in
Stopping credentials, keys and other secrets from leaking into logs and error messages. The vulnerability class is credential exposure, one of the most common ways a breach escalates.
Holding the file boundary
Making sure a file path can't escape the directory it's allowed to touch, including through symlinks and archive members. The class is path traversal.
Controlling where a request can reach
Checking that a request only reaches the hosts, ports and origins it's meant to, such as validating loopback and normalising origins. The class covers server-side request forgery and cross-origin bypass.
Trusting only what checks out
Validating that an argument, token or identifier is really what it claims before it's acted on. The class covers injection and unchecked identifiers.
Scoring is deterministic: a hidden security oracle decides whether the security outcome was achieved, and an exact-change scorer decides whether the work stayed within bounds. Model opinion decides nothing.
Nothing in the range is real: no live service, no real user data, no path out of the sandbox. The runtime is disposable and torn down after use, and the containment controls were exercised before a single session was scheduled.
A practice run, before anything counted, sealed as a failure: two sessions timed out, and two repairs that passed the security oracle were rejected because stray test caches had polluted the exact-change scorer. It cost only time, and it flushed out the apparatus problems before a real result could be affected.
What gets reported
When the run is sealed: aggregate outcomes for both bundles across both roles, as tables and CSV on this page, with the same receipts discipline as every other study here. The fixtures, task material and scoring internals stay private, as across the whole programme, because anything published ends up in training data and would invalidate future runs.
Until then, this page makes no claims. There is no preliminary result here, and no hint of which way it will go. That is the point of publishing the design first.
Status
Pre-registered, not yet running. Five gates stand between here and the first primary session, and each one exists so that when the results arrive, they mean something.