World models: what predicting the next moment buys an agent on a fixed camera
A world model learns what a scene will do next. From Ha and Schmidhuber in 2018 to V-JEPA, Genie, Sora and Cosmos, the research is moving toward video. What that could mean for a camera over a loading dock, and what is still a claim rather than a result.
A detector tells you what is in the frame. A world model tells you what the frame will probably look like in two seconds, and an agent that can anticipate can act before the event rather than after it.
The 2018 paper that named it
World Models (Ha and Schmidhuber, 2018) is the paper most people cite for the term. A vision model V compressed each frame into a small latent vector. A memory model M, a recurrent network with a mixture-density output, learned to predict the next latent from the current one and the action taken. A tiny controller C chose actions from the latent and the memory state. The agent could be trained entirely inside the "dream" generated by its own model, and the policy transferred back to the real game. The game was a racing simulator, but everything since has kept the shape: compress the frame, then predict in the compressed space.
Danijar Hafner and colleagues made it practical. Dreamer (Hafner, Lillicrap, Ba and Norouzi, 2019) learned behaviours by "latent imagination," propagating gradients through imagined trajectories in the learned model rather than through the real environment. DreamerV3 (Hafner, Pasukonis, Ba and Lillicrap, 2023) reached across more than 150 tasks with a single configuration and collected diamonds in Minecraft from raw pixels with no human data. In both, the model does its work in a compressed latent space; pixels are only reconstructed when someone wants to look.
Predict the representation, not the pixels
That last point became a position. In June 2022 Yann LeCun released A Path Towards Autonomous Machine Intelligence (version 0.9.2), arguing that predicting pixels wastes capacity on detail that cannot be predicted (the position of every leaf in the wind) and that a model should instead predict in an abstract representation space where unpredictable detail can be discarded. He called the design a joint embedding predictive architecture, JEPA. Two encoders produce representations of an input and a target, a predictor maps one to the other, and nothing is decoded back to an image.
V-JEPA (Bardes et al., Meta, 2024) is the video instance. It was trained on about two million public videos with feature prediction as the only objective: no pixel reconstruction, no text, no pretrained image encoder. With the encoder frozen, the largest model reached 81.9 percent on Kinetics-400, 72.2 on Something-Something-v2 and 77.9 on ImageNet. The second number is the one to notice, because Something-Something is a motion benchmark where appearance alone does not help.
The generative branch
The other camp predicts pixels. Genie (Bruce et al., Google DeepMind, 2024) trained an 11 billion parameter model on unlabelled internet video of platform games and learned three things at once: a spatiotemporal video tokenizer, a dynamics model, and a latent action model that discovers a small set of controllable "actions" with no action labels in the data. The authors call it a foundation world model; you can hand it a single image and play.
OpenAI's Sora report, Video generation models as world simulators (February 2024), made the boldest version of the claim in its title. The model treats video as a sequence of spacetime patches, a descendant of the ViT idea, and scales a diffusion Transformer on them. OpenAI listed a limited ability to simulate complex physics and to understand causality among the model's shortcomings, so what it generates is plausible video rather than verified physics.
NVIDIA's Cosmos (2025) is the most explicitly industrial. The authors describe a "world foundation model" as a general-purpose model to be fine-tuned into custom world models for downstream uses, and the platform ships a curation pipeline, video tokenizers, pretrained models and post-training examples, with open weights. The framing is that physical AI has to be trained digitally first, with a world model as a twin of the environment. Robots and driving are the intended customers; fixed cameras are not mentioned.
What it could mean for a camera on a pole
Take a warehouse camera watching a cross-aisle. A world model that has watched this camera for a month has a distribution over what comes next: the forklift usually turns left at this rack, and people from the break room cross here at shift change. Two uses follow.
Anticipation. If the predicted forklift path and the predicted pedestrian path intersect in the next two seconds, the alert can go out before the near miss rather than as a report of one. Occlusion. When a person walks behind a rack, a detector loses them and a tracker guesses linearly; a model with a learned prior over this scene can hold a better estimate of where and when they will re-emerge. A frame the model assigns low probability to is also, almost by definition, an anomaly, without anyone having enumerated what anomalies are.
Fixed cameras are a friendlier case than the papers above. The background is static and the routines repeat daily, so the prediction problem is much narrower than "all of YouTube." A model with a fraction of Genie's parameters could plausibly learn one dock.
What is still speculative
The evidence is thin. None of the papers above reports a result on fixed security or operations cameras. The behaviour-learning results (Dreamer, Genie) are in games and simulators, and V-JEPA's numbers are on recognition benchmarks rather than on predicting the next event at a site. Sora and Cosmos are generators, and a generator that can draw a convincing forklift does not thereby know where this forklift is going. I could find no published evaluation of anticipation accuracy on real facility footage, and until there is one, "predictive" on a product sheet should be read as a tracker with a longer horizon.
The direction is not in doubt. In 2018 a world model played a racing game in its own dream. In 2024 one learned motion from two million videos without reconstructing a pixel. The question for this field is whether the same trick works on the one camera that never moves.