The knowing-doing gap

The model writes down what it should do, correctly, repeatedly, and then does something else. Not a failure of knowledge, and not obviously a failure of capability, since the same system executes the same action readily in other contexts.

Recorded because it is measured rather than merely observed here: the plan-follow-through findings put three model families between roughly 48 and 66 per cent, and the same pattern is reported independently in BALROG across game environments.

Distinct from the sensorium effect, which is a failure of perception rather than of follow-through.