CivBench
A collaborative benchmark for long-horizon, tool-mediated agents in Civilization VI, with a paper accepted at NeurIPS 2026.
ProjectSources & confidence: Qualified confidence
Checked 4 October 2026 · AI-assisted review
Collaborative benchmark, published pilot and author-announced NeurIPS 2026 poster acceptance.
- CivBench project website
Public evaluation project and current website presenting game results and ELO rankings.
- Liam Wilkinson — I Gave an AI a Civilization to Run
The author’s June 2026 description of GovBench, CivBench and his then affiliation; reported results are not independently reproduced.
- CivBench paper
Benchmark framing, pilot scope and authorship.
- CivBench — research update
Author-announced NeurIPS acceptance.
What this check doesn’t establish
- The pilot’s results have not been independently reproduced in this catalogue.
- NeurIPS acceptance follows the author’s announcement; the linked paper is the arXiv version.
Confidence applies to the scope above, not every claim made by the underlying source. How confidence works
- Kind
- benchmark (own work)
- Repository
- github.com/lmwilki/civ6-mcp
CivBench’s paper studies how AI agents monitor the world and act on their plans over long games of Civilization VI. Built on civ6-mcp, it turns a shared interest in strategy games into research on sustained decision-making.
The paper was accepted as a poster at the NeurIPS 2026 Evaluations and Datasets Track. It grew through collaboration with Austin Tudor David Andrews, Jamie Heagerty, Harry Coppock, Jakob Nicolaus Foerster and Rui Ponte Costa.
The essay describes the early experiments. The paper’s 23 admissible runs are a pilot, not a model ranking; later website results should be read separately from that fixed study.
In the essays
- I Gave an AI a Civilization to Run. It Built a Nuke.
Read the essay
Sources
People
- Liam Wilkinson
Built by
Projects
- civ6-mcp
Part of
Findings
- Games reaching an ending without the external diary
Findings
- Losses where an imminent rival victory was never checked
Findings
- Plan follow-through, Claude Opus 4.6
Findings
- Plan follow-through, Gemini 3.1 Pro
Findings
- Plan follow-through, GPT-5.4
Findings
- Share of agent actions spent checking the whole board
Findings
- Victory-condition checks performed against the sixteen instructed
Findings
Cases
- The Portugal game
Organises
Record history
Catalogue source: knowledge/projects/civbench.md
Generated by claude-code/claude-opus-5 on 2026-07-25