World models: what predicting the next moment buys an agent on a fixed camera
A world model learns what a scene will do next. From Ha and Schmidhuber in 2018 to V-JEPA, Genie, Sora and Cosmos, the research is moving toward video. What that could mean for a camera over a loading dock, and which parts are still claims with no result behind them.
In 2018 a neural network learned to drive a racing simulator entirely inside its own dream of the game, and the policy it learned there worked when it was dropped back into the real one. That is what people mean by a world model: a model of what the scene will probably look like in two seconds. A detector tells you what is in the frame now. An agent with a world model can act before the event.
The 2018 paper that named it
World Models (Ha and Schmidhuber, 2018) is the paper most people cite for the term, and the racing simulator was its test case. A vision model V compressed each frame into a small latent vector. A memory model M, a recurrent network with a mixture-density output, learned to predict the next latent from the current one and the action taken. A tiny controller C chose actions from the latent and the memory state. The agent could be trained entirely inside the "dream" generated by its own model, and the policy transferred back to the real game. Everything since has kept that shape: compress the frame, then predict in the compressed space.
Danijar Hafner and colleagues made it practical. Dreamer (Hafner, Lillicrap, Ba and Norouzi, 2019) learned behaviours by "latent imagination," propagating gradients through imagined trajectories inside the learned model, with no calls to the real environment during that step. DreamerV3 (Hafner, Pasukonis, Ba and Lillicrap, 2023) reached across more than 150 tasks with a single configuration and collected diamonds in Minecraft from raw pixels with no human data. In both, the model does its work in a compressed latent space; pixels are only reconstructed when someone wants to look.
Prediction without pixels
That last point became a position. In June 2022 Yann LeCun released A Path Towards Autonomous Machine Intelligence (version 0.9.2), arguing that predicting pixels wastes capacity on detail that cannot be predicted (the position of every leaf in the wind) and that a model should predict in an abstract representation space where unpredictable detail can be discarded. He called the design a joint embedding predictive architecture, JEPA. Two encoders produce representations of an input and a target, a predictor maps one to the other, and nothing is decoded back to an image.
V-JEPA (Bardes et al., Meta, 2024) is the video instance. It was trained on about two million public videos with feature prediction as the only objective: no pixel reconstruction, no text, no pretrained image encoder. With the encoder frozen, the largest model reached 81.9 percent on Kinetics-400, 72.2 on Something-Something-v2 and 77.9 on ImageNet. The second number is the one to notice, because Something-Something is a motion benchmark where appearance alone does not help.
The generative branch
The other camp predicts pixels. Genie (Bruce et al., Google DeepMind, 2024) trained an 11 billion parameter model on unlabelled internet video of platform games and learned three things at once: a spatiotemporal video tokenizer, a dynamics model, and a latent action model that discovers a small set of controllable "actions" with no action labels in the data. The authors call it a foundation world model; you can hand it a single image and play.
OpenAI's Sora report, Video generation models as world simulators (February 2024), made the boldest version of the claim in its title. The model treats video as a sequence of spacetime patches, a descendant of the ViT idea, and scales a diffusion Transformer on them. OpenAI listed a limited ability to simulate complex physics and to understand causality among the model's shortcomings, so what it generates is plausible video, with no guarantee the physics is right.
NVIDIA's Cosmos (2025) is the most explicitly industrial. The authors describe a "world foundation model" as a general-purpose model to be fine-tuned into custom world models for downstream uses, and the platform ships a curation pipeline, video tokenizers, pretrained models and post-training examples, with open weights. The framing is that physical AI has to be trained digitally first, with a world model as a twin of the environment. Robots and driving are the intended customers; fixed cameras are not mentioned.
What it could mean for a camera on a pole
Take a warehouse camera watching a cross-aisle. A world model that has watched this camera for a month has a distribution over what comes next: the forklift usually turns left at this rack, and people from the break room cross here at shift change. Two uses follow.
Anticipation. If the predicted forklift path and the predicted pedestrian path intersect in the next two seconds, the alert can go out before the near miss, while a shout still helps. Occlusion. When a person walks behind a rack, a detector loses them and a tracker guesses linearly; a model with a learned prior over this scene can hold a better estimate of where and when they will re-emerge. A frame the model assigns low probability to is also, almost by definition, an anomaly, without anyone having enumerated what anomalies are.
Fixed cameras are a friendlier case than the papers above. The background is static and the routines repeat daily, so the prediction problem is much narrower than "all of YouTube." A model with a fraction of Genie's parameters could plausibly learn one dock.
What is still speculative
The evidence is thin. None of the papers above reports a result on fixed security or operations cameras. The behaviour-learning results (Dreamer, Genie) are in games and simulators, and V-JEPA's numbers are on recognition benchmarks, which say nothing about predicting the next event at a site. Sora and Cosmos are generators, and a generator that can draw a convincing forklift does not thereby know where this forklift is going. I could find no published evaluation of anticipation accuracy on real facility footage, and until there is one, "predictive" on a product sheet should be read as a tracker with a longer horizon.
V-JEPA learned motion from two million videos in 2024 without reconstructing a pixel. Nobody has yet published what happens when that trick is pointed at the one camera that never moves.