Mechanism 2: Factual priming

Once we analyze the pure reasoning traces generated for easy factual questions, we discover a typical sample. The fashions aren’t writing out logical proofs; they’re surfacing associated information.

In human cognition, there’s a idea often called spreading activation, the place processing a particular idea primes associated ideas in semantic reminiscence, making them simpler to retrieve. We hypothesize that language fashions exhibit an analogous generative self-retrieval mechanism, which we name factual priming. By producing information topically associated to the query, the mannequin builds a contextual bridge that facilitates the retrieval of the right reply.

To check hypotheses, we extract simply the concrete information from the mannequin’s reasoning traces, making use of strict filtering to strip away any filler textual content, search plans, or specific mentions of the ultimate goal reply. We then isolate the impact of the recalled information, and present that conditioning on a brief listing of recalled information recovers most of reasoning’s beneficial properties and helps even when reasoning is OFF.