The switchboard idea: cameras that borrow each other's compute
Our earliest filing, from December 2018, proposed running video analytics as a pool of small endpoints on cameras and appliances, with a controller that patches busy cameras to idle compute. It was never granted. Most of it is how the system works today.
Before there was a recorder, there was a question about where the compute should go. In December 2018 our co-founder Tanuj Thapliyal filed a provisional application titled "System and method for distributed image processing". The full application followed a year later and was published in June 2020 as US 2020/0186454 A1. It was later abandoned and no patent issued. We are writing about it because the published document is a clear record of an architectural bet, and because the bet held.
The bottleneck
The 2018 filing describes the standard analytics controller of the time: a switch feeding camera streams into one powerful machine that decodes the packets and runs the models, on a CPU, a GPU or a dedicated chip. Or the same thing in the cloud, with every stream shipped up first. Either way one box sits in the path of every pixel. Adding cameras, raising frame rates or running heavier models means a bigger box, and the box is idle most of the day because most cameras see nothing most of the time.
Sensor endpoints and analytic endpoints
The proposal splits every device into two kinds of endpoint on a software-defined network, each with a discoverable name.
A sensor endpoint is a small web server on a camera. It answers a request for video at a given time, for a given duration, at a given quality, or it writes the video to shared storage and answers with the location.
An analytic compute endpoint is a container, on a camera or on an appliance, that runs a detection filter using whatever processor it has: a CPU, a GPU or a small tensor accelerator.
A controller sits between them. The filing's analogy is a telephone switchboard operator. The controller finds an analytic endpoint with free capacity and patches a sensor endpoint to it. The analytic endpoint then pulls the frames directly from the camera, processes them, and writes the result asynchronously to a shared database. The controller never handles video, so it is not CPU-blocked by high-bandwidth traffic and can coordinate many cameras from a small processor.
An ingress and load balancer in front of the analytic endpoints picks the one that is idle or has the shortest queue. The filing lists other factors it could weigh: a target latency, a required confidence, the cost of cloud compute, whether a frame should be split into sub-images and processed in parallel, and the desired frame rate. Cameras pre-filter on motion so quiet cameras send nothing. The consequence the filing draws out is the one we still find elegant. Cameras that are not seeing anything lend their processor to the cameras that are.
Segments and metadata
Storage follows the same pattern. Incoming video is cut into short segments bounded by keyframes, a few seconds each, given hashed names, and written into an object store spread across the endpoints' spare disk. Decoding to pixels happens in parallel using hardware decode blocks, then per-camera filters run: motion in a zone, faces, vehicles, or a customer-specific detector such as a hazard cone on the inbound road of a construction site.
Detected events go to a separate metadata database, with timestamps, durations and the boxed regions where they happened. To retrieve footage you query metadata for timestamps, look up the segments for those timestamps, and stitch them together on another container or on the client. Video and metadata never share a table, and the spare storage on idle endpoints absorbs footage from the active ones.
Getting in from outside
The cluster lives behind a customer's firewall or router. The filing describes exposing it through a dedicated tunnel per camera feed, with routing performed only when a person is actually trying to watch, along the fastest path from the stored segment to their client.
What did not work
The application itself. It was published on June 11, 2020, and the US case was abandoned the following year, so nothing here is protected and none of it is secret. The other thing that did not survive was the assumption in the filing's second figure that the cameras would host the analytic endpoints. Our later filings describe the recorder as compatible with any IP camera so that a customer keeps the cameras already on the wall. A camera that has to run containers would have ruled most of them out.
What shipped
The split did. Our later filings describe cameras of any make streaming to a recorder on site that runs the detection models, keeps video locally with event footage backed up to the cloud, and sends detection metadata upward for search and reporting. That is the 2018 architecture with the appliance carrying the analytic endpoints and the cameras left as sensors. The published application is now cited by eight later filings from other companies, and by our own 2025 patent on semantic event detection.