What a video model needs that a language model never did
Text sits still. The physical world moves, and it punishes slow answers. Four requirements that make video agents a different engineering problem from chatbots.
Language models started with an unearned gift: decades of text on the internet, already written and organised and sitting on disk. Yann LeCun likes to point out that a large model reads on the order of 10^14 bytes, more than a person could get through in a million years, while a five-year-old's visual system takes in a comparable amount in the first few years of life. His quip: "LLMs alone are a dead-end if human-level AI is the aim."
Video agents live on the other side of that line. Four things they need that a chatbot never did.
1. Time
A sentence does not change while you read it. A loading dock does. A video model has to hold a scene across seconds and minutes: the pallet that was there a minute ago and the person who walked behind it. The window between September 2023 and March 2024, when the major labs made images and video first-class inputs, is what made this possible at all. Temporal reasoning is still the weakest of the new abilities, and it is the one physical work depends on most.
2. Verification
A language model that is wrong produces a bad paragraph. A video agent that is wrong stops a machine or wakes a supervisor at 3 a.m. Spot AI's engineers built what they call a proposer-verifier framework, in which a second model "functions as a safeguard by cross-checking detected events against stringent accuracy thresholds" before anything fires. Alert fatigue is how these systems get switched off, and the verifier is the engineering answer to it.
3. A latency budget
Ask a chatbot a question and ten seconds is fine. Ask a camera whether someone is in the forklift lane and ten seconds is an accident. The useful frame is a perception-to-action budget: capture, encode, infer, decide, actuate, network. Security tolerates a second or two. Robotics tolerates tens of milliseconds. Every architecture decision, including the current preference for edge hardware, falls out of that number.
4. Action
This is the real divide. A vision-language model produces descriptions. A vision-language-action agent produces interventions: lock the door, open the ticket. The output is a change in the world, which means the model needs tool schemas and policy limits, and every action has to leave an audit trail. Spot AI's definition is the simplest one I have seen: "software that connects computer vision, AI reasoning, and workflow automation."
The utility test
One way to put all four together: the value of a video agent rises with perception accuracy and reasoning depth, and falls with latency. Early use cases are the ones where today's numbers already clear the bar. Security clears it because the incumbent, a person watching forty screens, sets a low one. Fine-grained manufacturing control will take another turn or two of the models.
That ordering comes from the physics of the problem. It is also why the first agents showed up on loading docks.