How ReCAP detects and counts distinct locations in video

In an era when broadcasters manage petabytes of footage across live feeds, catch-up archives, and social media clippings, knowing exactly where every scene was filmed is no longer a luxury. It is an operational necessity. The EU-funded Real-time Content Analysis and Processing (ReCAP) project is building the technical foundations that allow machines to look at raw video frames and produce structured, searchable information about what they see.

One of the most compelling capabilities in the ReCAP toolbox is its ability to detect and count the number of distinct locations in a video. Whether the source material is a sprawling lifestyle series shot across multiple Australian cities, a single-location documentary, or a news package that cuts between Parliament House in Canberra and a beach in Bondi, ReCAP aims to give editors an automatic, frame-accurate inventory of the places that appear on screen.

Why counting distinct locations matters in modern broadcasting

For a long time, location metadata in broadcast archives was captured by human logging teams, or not captured at all. Editors marked in and out points, but the spatial context of those edits often lived only in the heads of production staff. When programmes moved between countries, streaming platforms, or sister networks, that knowledge was frequently lost. The result was a vast reservoir of footage whose true spatial value was invisible to search engines, recommendation systems, and rights clearance workflows.

Counting distinct locations in a video changes that picture. Instead of an editor scrubbing through hours of tape to remember whether the cruise episode was filmed in Darwin or Townsville, an automated system can produce a count and a list. Production companies can audit compliance with regional filming quotas, distributors can match footage with advertisers targeting specific markets, and archivists can build better taxonomic systems for their holdings.

The commercial stakes are rising as well. Rights owners across Australia and beyond are under pressure from advertisers to prove that scenic material was genuinely shot on location, and regulators want reassurance that locally produced programming actually showcases local places. An automated count offers a verifiable signal that supports both of these demands without adding to the workload of already stretched production offices.

Inside the ReCAP detection pipeline

ReCAP approaches location counting through a layered analysis pipeline. Raw video is decoded and segmented into individual shots, the basic building blocks of any moving image. Each shot is then subjected to a sequence of computer vision modules, some of which examine visual cues such as architectural style, vegetation, and lighting, while others look at audio signatures, spoken place names captured through speech recognition, or on-screen text that mentions a venue, suburb, or landmark.

The outputs of these modules are combined into a scene-level representation that captures the dominant visual and contextual features of a sequence of frames. ReCAP then groups shots into scenes, applies place recognition against a reference index of known locations, and emits a structured list of distinct locations together with a count. Because each module produces its own confidence score, the system can flag ambiguous results for human review rather than forcing a single best guess.

For broadcasters, the practical consequence is a metadata feed that arrives within seconds of a live transmission ending. A nightly news bulletin shot from a Sydney studio with a packaged report from a Melbourne press conference, a Perth weather cross, and a pre-recorded segment from Adelaide can be automatically tagged with four distinct locations, complete with timestamps and confidence indicators, ready to feed downstream applications.

Building blocks of the location recognition engine

Under the hood, ReCAP draws on a combination of deep convolutional networks, transformer-based vision models, and classical computer vision techniques. Convolutional and transformer backbones extract embeddings from sampled frames, capturing the textures, lighting conditions, and structural elements that distinguish one place from another. These embeddings are compared against a curated index of reference locations drawn from public datasets and the consortium's own annotated collections.

The place classifier itself is hierarchical, distinguishing continents, countries, regions, and specific sites. A shot filmed on the steps of the Sydney Opera House can be recognised as Australia, as New South Wales, and as a specific cultural landmark. This hierarchy lets ReCAP answer very granular queries, such as how many unique Australian capital cities appear in a season of a programme, as well as broader ones, such as how many distinct countries are present across a multi-territory co-production.

Another building block is the duplicate detection module. Broadcast archives are full of repeated material: opening title sequences used across an entire run of episodes, station idents shown hundreds of times a day, or a stock clip of a koala filmed once in Queensland and reused for years. Without deduplication, these repeated shots would inflate the count of distinct locations dramatically. ReCAP strips these repeats before they reach the location classifier, ensuring that a long-running series is not credited with dozens of phantom venues that only appeared once.

Scalability and quality across large archives

Accuracy is only part of the story. A location detector that takes a week to process a single season of television drama would be impractical. ReCAP has therefore been designed with two parallel tracks: a real-time mode that runs alongside live ingest and operates with controlled latency, and a batch mode that can chew through entire archives overnight. The trade-offs are managed through configurable thresholds, allowing operators to dial up precision for forensic applications and dial it back for routine indexing.

Quality assurance is built into the workflow. Synthetic test sets, drawn from openly licensed broadcast material and from consortium-internal recordings, allow the team to benchmark each release. Human-in-the-loop review is supported through a web interface where editors can confirm or correct the system's guesses, and these corrections feed back into the training data. Over time, the model becomes more attuned to the visual vocabulary of Australian locations, from the sandstone heritage of inner Sydney to the red desert vistas that often stand in for the outback in international co-productions.

The system is also designed to be transparent. Every distinct location it reports comes with a confidence score, a list of contributing shots, and a rationale drawn from the underlying modules. That audit trail matters when the metadata feeds into legal decisions about rights, advertising claims, or regulatory compliance under frameworks such as those administered by the Australian Communications and Media Authority.

Practical applications for Australian media workflows

The value of automated location counting lands particularly hard in Australia, where broadcasters operate across vast distances and multiple time zones. A news organisation that runs state-based editions in Sydney, Melbourne, Brisbane, and Perth can use ReCAP's output to verify that each edition actually contains local material, a relevant concern under local content expectations reinforced by public broadcasters such as the Australian Broadcasting Corporation.

Production companies also benefit. Many Australian dramas and lifestyle programmes are now shot partly overseas or in studio sets dressed to look like specific places. ReCAP's place classifier can spot when a sequence claimed to be filmed in Hobart was actually captured in a sound stage in Melbourne, helping producers and investors keep an honest record of where their money was spent. Reality formats, which often travel between cities from week to week, can be analysed at the end of a season to produce a clear map of filming locations, useful for marketing, tourism partnerships, and cultural impact reporting.

There are everyday workflow benefits too. Editors working on catch-up clips for streaming platforms can search by location rather than by textual description. Marketing teams can assemble promotional montages of every café, beach, or rooftop that appeared in a season without manually logging each shot. And archivists, who have long maintained hand-written location registers, finally have a tool that updates those registers automatically, frame by frame.

The ReCAP consortium and the path forward

Like any ambitious research project, ReCAP depends on collaboration between research institutions, broadcasters, and technology vendors. The consortium brings together specialists in computer vision, speech processing, multimedia indexing, and broadcast engineering, and you can learn more about one of the technical partners on the NMR consortium partner page. Each member contributes both algorithmic expertise and access to real-world broadcast data, which is essential for training and validating models that work in production conditions rather than only in the lab.

Over the coming years, the consortium plans to extend location counting beyond the visual channel. Plans include integrating on-screen text recognition more tightly, fusing broadcast schedule data so that the system knows whether a clip came from a sporting event or a soap opera, and improving support for multilingual content, including the multilingual reality of Australian broadcasting where Indigenous languages, English, and community languages all play a role in different programme contexts.

The consortium also intends to publish open benchmarks and reference datasets so that other researchers can replicate and extend the work. That open posture is important for a project whose outputs will eventually sit inside commercial broadcast stacks: operators need confidence that the algorithms are well understood, well documented, and not dependent on any single vendor's black box.

Distinct-location counting may sound like a narrow technical feature, but in a media environment where every frame is now searchable, monetisable, and auditable, it becomes the kind of metadata that quietly rewires how a broadcaster works. As ReCAP moves from prototype to production, the groups who integrate it earliest will spend less time chasing paperwork about where a clip was filmed and more time using that knowledge to build better programmes, sharper recommendations, and richer archives for the next generation of Australian storytelling.