Case record
The Portugal game
A CivBench run in which the agent, playing Portugal, built a leading diplomatic position, then spent fifty turns and two nuclear weapons stopping a rival's culture victory while losing to that rival's diplomatic one.
Why this is recorded as a case
Three essays had recorded the same blockage: an episode with actors, instruments,
a decision under constraint and an outcome, which could not be filed because the
only case payload in use required a jurisdiction, a policy domain and a wave of
technocratic thought. This record is the first use of the second payload,
agent-run.
The alternative was to widen the governance payload until a Civilization game and the Dutch childcare benefits scandal validated against the same fields. That would have cost the comparison the governance records exist to support, which is the reasoning set out in the vocabulary policy.
What the payload had to capture that governance does not
The system and the environment are separate objects because the same model is run against several environments and the same environment against several models. Collapsing them is what makes benchmark claims unreproducible.
environment-defective is a verdict in its own right rather than a kind of
failure. A run in which the apparatus prevented an action is evidence about the
apparatus, and pooling it with genuine agent failure inflates the failure rate.
This run is not recorded under that verdict, because the game did reach a natural
ending, but three of its defects are recorded against the environment and the
harness rather than the agent.
The attributable_to field on each failure exists for the reason set out in
contested_claims above. The agent stopped watching the clock it lost to, and the
tool that would have let it watch that clock was broken. A schema that forced a
choice between those two would produce a tidier record and a less honest one.
Links to
- orchestrated by
- CivBench
- produced finding
- Losses where an imminent rival victory was never checked
- ran through
- civ6-mcp
Referenced by
- depicts
- The nuking of Toulouse
Source: knowledge/cases/civbench-portugal-toulouse.md
Generated by claude-code/claude-opus-5 on 2026-07-26