The evaluation harness is the product
Frontier labs now spend more of their compute on post-training than on pre-training. For video AI, that means the winners will be whoever lets a customer measure and tune a model on their own footage.
Tag
Engineering Corner
World models: what predicting the next moment buys an agent on a fixed camera
A world model learns what a scene will do next. From Ha and Schmidhuber in 2018 to V-JEPA, Genie, Sora and Cosmos, the research is moving toward video. What that could mean for a camera over a loading dock, and which parts are still claims with no result behind them.
The switchboard idea: cameras that borrow each other's compute
Our earliest filing, from December 2018, proposed running video analytics as a pool of small endpoints on cameras and appliances, with a controller that patches busy cameras to idle compute. It was never granted. Most of it is how the system works today.
Segment Anything, embeddings, and how a site manager teaches a camera a new object
SAM turned segmentation into a prompt. SAM 2 gave it memory across frames. Put a self-supervised embedding behind them and a site manager can define a new object class from a handful of clicks, with no retraining. The design, the papers, and the limits.
Four benchmarks for an agent that has to act
Describing a scene is a solved demo. Doing the right thing in it every hour of a shift is the harder test, and buyers have started writing four metrics for it into their evaluations.
Three agents for the risks that cost the most: fire, cash, break-ins
A new class of agents that watch for a specific behaviour and its consequence, shipped in July 2026. What each one does, and the small interface change that matters more than it looks.
Attention is a better word than surveillance
Surveillance records people so that someone can assign blame later. The safety directors running video agents describe something else: a camera that notices a forklift near-miss now, so the next one never happens.
A field guide to the agent features vendors announced this quarter
June to September brought a dozen agent features from seven vendors. Sorted by what each one does, and whether it ships, is in beta, or exists only in a press release.
Why the edge box is winning the argument over the cloud
The cloud was supposed to eat everything. For video agents it lost on arithmetic: a facility with 64 cameras produces 30 terabytes a month, and a lock has to fire in under a second.
Build a video agent by describing it
Custom video detection used to mean an ML team and six to eight weeks of work. Iris made it a ten-minute conversation. What that changes.
What an agent is, mechanically
Our second granted patent describes the machinery under a video agent: a template, a set of filters over detected objects, a zone and an action, with a trace of every step so you can see why it fired.
Milestone's pitch for AI in XProtect starts with where the training video came from
The Copenhagen VMS maker says its summarization and search tools were tuned on a 300,000-item library of customer video, gathered under fair-exchange agreements and anonymized before use. Summaries ship now; search arrives at the end of 2026.
How a large vision model gets onto a box next to a camera
The models that win benchmarks assume a data-center GPU. The box behind a loading-dock camera has a 15-watt budget. Distillation, quantisation, and pruning are how one becomes the other, and a latency budget decides what stays behind.
A decision framework from a detective
Rish Gupta says every big call at Spot AI runs through one line of Sherlock Holmes. The same four steps are a sensible way to decide what a video agent is allowed to do on its own.
What a video-language model is, and where the action comes from
A vision-language model is two encoders and a shared vector space. Feed it video and the hard part becomes time, and asking it to act takes a different model again. The papers, in order, from ImageNet to RT-2.
Let your AI coworkers watch video
Over ninety percent of security alerts are harmless. The case for spending human attention on the ten percent that are not, from the first edition of The Edge.
From composable models to reasoning: how video agents got built
Spot AI stacked small detection models together until the stack ran out of judgement. A large multimodal model closed the gap, and the agent category came out of the seam between the two.
What a video model needs that a language model never did
Text sits still. The physical world moves, and it punishes slow answers. Four requirements that make video agents a different engineering problem from chatbots.
Describing an event in a sentence and letting the camera look for it
Most of what customers want to detect has no off-the-shelf model. Our first granted patent covers the workaround: put frames and sentences in the same vector space and treat a description as a trigger.
Rhombus gives outside AI agents a map of the cameras before they answer anything
A July 2026 post by Rhombus's Jeremy Gulley describes a command-line tool that exports a live snapshot of every site and device, and a Claude plugin that loads it before each query. The premise is that an agent that does not know where the cameras are cannot be trusted with much.
Camera health decides churn before accuracy does
Ask an advisor why monitoring customers leave and the answer is a camera that stayed dark for a week. The support queue and one 82-second clip say the same thing.
How a video system answers "show me the forklift near the dock at 3 a.m."
Retrieval over video borrows its shape from text RAG, then has to solve two problems text never had: where in the frame, and when in the stream. The research that makes it work, and the checking step that keeps false positives down.
