Large models enter video surveillance: from seeing to understanding
For more than a decade the job of a video surveillance system has been to record the picture faithfully. As AI vision algorithms and multimodal large models mature, the industry is turning to a different question: can the system understand what it is looking at?
Conventional video analytics relies on preset rules — line crossing, intrusion into a region, abandoned objects. Once written, those rules generalise poorly; a new site usually means reconfiguring everything. Multimodal models change that: the system can describe an event in plain language, such as "a person has been lingering along the perimeter wall for more than two minutes", without an engineer hand-coding each case.
For integrators and end users this means three practical shifts. Search moves from scrubbing through timelines to querying by meaning. Alarms become closer to the business itself, and the volume of nuisance alarms drops. And compute moves — part of the analysis migrates from back-end servers to the camera itself.
It is worth staying sober about the limits. Large models still face constraints in edge compute cost, data compliance and guaranteed accuracy. In real projects the practical architecture is a small on-device model for first-pass screening plus a central model for verification, balancing cost against results.