Vision-Language Models for Cross-Modal Retrieval in Intelligent Surveillance Systems

Abstract

Vision-language models (VLMs) that learn a joint embedding space for images and free-form text from web-scale, weakly
aligned corpora have reshaped cross-modal retrieval, and intelligent video surveillance is one of the domains where that shift
matters most in practice: an operator searching archived footage for a natural-language description, a reference photograph,
or a short example clip needs exactly the similarity-ranking capability these models provide. This article synthesizes the
literature connecting large-scale vision-language pretraining, from contrastive dual encoders such as CLIP and ALIGN through
bootstrapped and grounding-oriented architectures such as BLIP and GLIP, to the surveillance-specific retrieval problems these
backbones now underpin: text-based person search across disjoint camera views, open-vocabulary object and vehicle detection,
text-to-video retrieval for reviewing archived footage, and language-conditioned framings of anomaly recognition. We trace
the transformer-based fusion architectures that preceded and paralleled contrastive pretraining, examine how the
fine-grained attribute matching that person search demands has been addressed through attribute disentanglement and
implicit relation reasoning, and report the specific, source-verified numeric results the field has published. We close by
identifying open challenges -- the domain gap between web imagery and surveillance footage, fine-grained discrimination at
gallery scale, computational cost, cross-camera robustness, and privacy -- that remain substantially unresolved despite the
rapid progress in the underlying vision-language foundation models