World models are intended to help autonomous AI agents predict how environments change after actions. A new paper argues that many current systems miss a crucial factor: the beliefs, goals, and social expectations of the people in those environments.
Systems including Sora, Genie 3, JEPA, and Marble primarily represent physical details such as objects, positions, movement, and occlusion. For service robots, medical assistants, and collaborative agents, that physical description is incomplete because hidden mental states often determine human behavior.
For example, if someone’s cup is moved into a cabinet while they are not looking, a physical model can represent the new location correctly but may still predict the wrong next action. A more capable model must also represent the person’s belief that the cup remains elsewhere.
The researchers call their framework Mental World Modeling (MWM), which they published on GitHub. It adds beliefs, attention, goals, intentions, emotions, norms, and social relationships to conventional world models. The target agent receives an egocentric, partial view, while the model maintains a complete representation of the scene.
MWM separates every action into a physical carrier—such as speaking, pointing, or grasping—and a mental payload, such as comforting, deceiving, or rejecting. The same act of sliding a cup across a table can have different meanings depending on these hidden variables.
MWM links physical and mental states, renders the target agent’s first-person perspective, and simulates how actions change both types of state. The authors do not claim that the system models consciousness. Instead, mental states are hypotheses inferred from behavior and context, so the framework is intended to represent uncertainty and expose its assumptions.
MENTIS and the Menti-Bench evaluation
To test the approach, the researchers created MENTIS, a modular, training-free pipeline. It parses a scene, renders the agent’s perspective, separates candidate actions into physical and mental components, and simulates the resulting states in parallel. Each branch is scored for physical plausibility, mental consistency, and social appropriateness before a deterministic choice is made. Machine-readable intermediate outputs make it possible to identify where an error occurred.
The accompanying Menti-Bench dataset contains 448 decision scenes: 320 text descriptions, 100 picture stories, and 28 sound-video clips. Each scene offers six response options and includes a human reference describing the correct action and its physical and mental context. At least two characters appear in 78 percent of the scenes.
Structured modeling outperforms repeated guessing
The team evaluated eight language models: five from OpenAI, including GPT-5.6-Sol and GPT-4.1, and three from Anthropic, including Claude Fable 5, Claude Opus 4.8, and Claude Haiku 4.5.
Using F1 as the accuracy measure, direct answers scored 63.3. Self-consistency—answering six times and selecting the most common response—raised the score to 77.9. The complete MWM pipeline reached 87.9, compared with 98.5 for humans under the same protocol.
The weakest model using MWM, GPT-4.1 at 84.9, outperformed the strongest model using direct answers with self-consistency, GPT-5.6-Sol at 83.6. Removing the mental channel reduced scores by an average of 12.1 points, while removing the physical channel reduced them by 16.5 points. Predicting the two transitions independently caused a further 6.4-point loss.
Correct state transitions produced the largest improvement, identifying next-state simulation as the main bottleneck. Mental modeling helped most in interpersonal scenes, where F1 increased by 26.4 points, compared with 14.0 points in object-focused scenes. Weaker base models gained more from the explicit structure: the improvement over direct answers was 28 points for GPT-4.1 and 21 points for GPT-5.6-Sol.
Further experiments replaced individual pipeline stages with human reference solutions. Perfect state transitions produced a 3.5-point gain, followed by a perfect initial state at 2.8 points and perfect observation at 1.7 points. Replacing all intermediate steps with reference answers raised the pipeline to 97 points.
The researchers report that roughly 80 percent of the remaining difference from human performance comes from errors in intermediate stages, especially transition simulation. Their conclusion is that the central challenge is not describing the current scene, but predicting how its coupled physical and mental state will change.
An unsettled definition of world models
World models have become a major focus beyond pure language models. Demis Hassabis, who recently stepped down as operational head of Google DeepMind, has said he spends much of his research time on the subject and expects a “ChatGPT moment” for these systems. Startups such as Odyssey have attracted hundreds of millions of dollars from investors.
However, researchers still disagree about what qualifies as a world model. An international team led by Peking University proposed a narrower definition that excludes text-to-video systems such as Sora because they lack feedback loops with the real world. Yann LeCun has likewise criticized generative approaches and advocated abstract representations.
The MWM paper places Sora, Genie, and JEPA in the same broad family and argues that they share the omission of mental states. That issue connects to Theory of Mind research, where language models have repeatedly struggled. Researchers from Meta’s FAIR lab, the University of Washington, and Carnegie Mellon University found that models fail on difficult belief-attribution tests and perform even worse when tracking world states.
MWM supplies mental-state representations externally through preprocessing. Anthropic’s discovery of an internal scratchpad in Claude points to a related development inside models: the scratchpad contains word-like thoughts that are not output, and removing it disrupts multi-step reasoning.