What a video model needs that a language model never did
Text sits still. The physical world moves, hides things, and punishes slow answers. Four requirements that make video agents a different engineering problem from chatbots.
Language models had a gift nobody earned: the internet. Decades of text, already written, already organised, sitting on disk. Yann LeCun likes to point out that a large model reads on the order of 10^14 bytes, more than a person could get through in a million years, while a five-year-old's visual system takes in a comparable amount in the first few years of life. His quip: "LLMs alone are a dead-end if human-level AI is the aim."
Video agents live on the other side of that line. Here is what they need that a chatbot never did.
1. Time
A sentence does not change while you read it. A loading dock does. A video model has to hold a scene across seconds and minutes: the pallet that was there, the person who walked behind it, the door that opened. The window between September 2023 and March 2024, when the major labs made images and video first-class inputs, is what made this possible at all. Temporal reasoning is still the weakest of the new abilities, and it is the one physical work depends on most.
2. Verification
A language model that is wrong produces a bad paragraph. A video agent that is wrong stops a machine or wakes a supervisor at 3 a.m. Spot AI's engineers built what they call a proposer-verifier framework, in which a second model "functions as a safeguard by cross-checking detected events against stringent accuracy thresholds" before anything fires. Alert fatigue is not a UX problem. It is the mechanism by which these systems get switched off.
3. A latency budget
Ask a chatbot a question and ten seconds is fine. Ask a camera whether someone is in the forklift lane and ten seconds is an accident. The useful frame is a perception-to-action budget: capture, encode, infer, decide, actuate, network. Security tolerates a second or two. Robotics tolerates tens of milliseconds. Every architecture decision, including the current preference for edge hardware, falls out of that number.
4. Action
This is the real divide. A vision-language model produces descriptions. A vision-language-action agent produces interventions: lock the door, start the clip, open the ticket. The output is not a token, it is a change in the world, which means the model needs tool schemas, policy limits, and an audit trail. Spot AI's definition is the plainest one in circulation: "software that connects computer vision, AI reasoning, and workflow automation."
The utility test
One way to put all four together: the value of a video agent rises with perception accuracy and reasoning depth, and falls with latency. Early use cases are the ones where today's numbers already clear the bar. Security clears it because the incumbent, a person watching forty screens, sets a low one. Fine-grained manufacturing control will take another turn or two of the models.
That ordering is not a marketing roadmap. It is the physics of the problem, and it explains why the first agents showed up on loading docks rather than in living rooms.