ReCAP automates shot classification by camera distance and angle

Modern video libraries are growing faster than the teams responsible for cataloguing them. A single hour of broadcast footage can contain hundreds of distinct shots, each defined by subtle differences in framing, lens choice, and camera placement. Manually tagging every close-up, medium shot, and high-angle view across a season of programming is a labour-intensive process that broadcasters across Australia and beyond have wrestled with for decades.

ReCAP, an EU-funded research project focused on real-time content analysis, addresses this challenge through machine learning models trained to recognise visual patterns in moving images. By analysing pixel-level features across consecutive frames, the system can determine whether a given shot was filmed at close range, captured from a low or high angle, or framed using a wide establishing composition. The output is structured metadata that fits seamlessly into existing media asset management pipelines.

The concept of shot classification by camera distance and angle is not new to cinematographers. What ReCAP brings is automation at broadcast quality, processing hours of footage in minutes rather than weeks. This shift has practical consequences for newsroom editors cutting together morning bulletins in Sydney, sports producers assembling highlights packages for evening coverage, and archivists preserving decades of Australian television history.

For local broadcasters such as SBS, the ABC, and the major commercial networks, automated shot metadata opens up new efficiencies. A producer searching the archive for every medium shot of a presenter can now run a single query rather than scrubbing through tapes manually. The same applies to angle-based searches — finding all low-angle footage from a football match at the MCG, for example, becomes a near-instant task when the underlying shots are already labelled.

Understanding camera distance categories

Camera distance refers to how close the lens sits to the subject relative to the surrounding environment. Cinematographers have long classified these distances into standard categories that ReCAP's models are trained to recognise automatically.

An extreme close-up captures fine features — a journalist's eyes during an interview, the texture of a cricket ball gripped by a fast bowler, or the hands of a pianist mid-performance. Medium close-ups frame subjects from the chest or shoulders upward and dominate news coverage and interview segments. Medium shots include the subject from the waist up and allow gesture and posture to register without losing facial detail. Wide shots, sometimes called establishing shots, situate the subject within a larger environment — the Sydney Harbour Bridge at dawn, a stadium crowd, or a Parliament House press gallery.

Shot Type Framing Typical Subject Size Broadcast Example
Extreme close-up Face or detail only Eyes, lips, hands Reaction shots during interviews
Medium close-up Chest upward Head and shoulders News presenter framings
Medium shot Waist upward Full upper body Panel discussions
Wide shot Full environment Subject within setting Establishing location shots

ReCAP's classifiers distinguish between these categories by analysing the proportion of the frame occupied by the dominant subject, combined with depth cues extracted across multiple frames. The output confidence score reflects how strongly the visual evidence supports each distance label, allowing downstream applications to apply different thresholds depending on the editorial need.

Recognising camera angle variations

Angle is a separate axis of classification that operates independently of distance. A subject can be filmed from below, above, or at eye level, and each option carries its own narrative weight.

Eye-level shots, where the camera sits roughly at the subject's height, dominate neutral documentary and interview content. High-angle shots, filmed looking down, are often used to convey vulnerability or scale — a common technique in coverage of public events such as Australia Day celebrations on the lawns of Parliament House in Canberra. Low-angle shots, pointing upward, lend authority and grandeur, frequently appearing in sports coverage from grounds like the SCG or Marvel Stadium. Dutch angles, tilted off the horizontal axis, introduce tension and are used sparingly in drama.

ReCAP's models detect these variations by tracking vanishing points, horizon lines, and the relative position of detected faces or objects within the frame. The classification considers both the static framing of each shot and any movement that shifts the camera's relative height, which is common in handheld documentary work and live event coverage. Where multiple angle cues are present — such as a tilted camera that also shoots from below — the system records the dominant label and retains secondary descriptors for later retrieval.

The technical approach inside ReCAP

ReCAP's shot classification pipeline combines several computer vision techniques to deliver reliable results across diverse content. The system runs in two passes: a coarse classification pass that segments the video into candidate shots using scene-cut detection, followed by a fine-grained analysis that labels each shot according to distance and angle criteria.

The pipeline draws on convolutional neural networks trained on annotated broadcast content from multiple European and international sources. By exposing the models to varied lighting conditions, camera equipment, and editorial styles, ReCAP's partners have aimed to reduce bias toward any single production convention. The classifiers output confidence scores, allowing downstream applications to filter only those tags meeting a chosen threshold.

Key capabilities of the classification module include:

The broader technical scope of these capabilities is laid out in the project work plan, which describes how shot classification integrates with companion modules for face recognition, logo detection, and quality monitoring. Performance benchmarks published by the consortium indicate the system maintains consistent accuracy across both studio-recorded and field-captured footage, though results vary with shot complexity and motion intensity.

Applications across production and asset management

The practical value of automated shot classification appears most clearly in three areas: archive retrieval, highlight generation, and editorial assistance.

For archive teams managing vast libraries — including decades of footage held by organisations such as the National Film and Sound Archive of Australia — keyword searches have long been supplemented by laborious manual review. Structured shot metadata accelerates this process dramatically. A researcher investigating how Australian prime ministers have been visually framed over the past forty years can filter every available clip by angle and distance in moments rather than days.

In live sports production, ReCAP's real-time tagging allows highlight editors to assemble packages on the fly. Coverage of an AFL match at Docklands, for instance, generates thousands of shots per fixture. Automated tagging identifies every close-up of a key forward, every wide tactical view, and every sideline reaction shot, allowing editors to construct a two-minute highlights package in a fraction of the time traditional workflows would require.

Newsroom applications are equally compelling. Producers cutting a story about a press conference in Brisbane can pull all wide establishing shots of the venue with a single filter, then layer in medium close-ups of the speaker and reaction shots from the audience. The same metadata feeds automated rough-cut assembly tools, which stitch together preliminary sequences based on shot type preferences.

Editorial scenarios where automated tagging delivers immediate value include:

Integration with broadcast workflows

ReCAP's classification outputs are designed to plug into standard media production environments rather than requiring wholesale system replacement. The project adheres to widely adopted metadata schemas, ensuring compatibility with existing MAM platforms, newsroom computer systems, and playout automation tools already in use across Australian broadcasters.

For organisations operating across multiple time zones — AEST in Sydney and Melbourne, ACST in Adelaide and Darwin, AWST in Perth — the implications for live workflows are particularly relevant. ReCAP's real-time analysis can run alongside live feeds, tagging shots as they arrive in the MAM rather than waiting for post-event processing. This capability matters during overnight coverage of international events, when Sydney-based teams hand off to colleagues in Western Australia or when breaking news demands rapid turnaround.

The integration path also extends to compliance and content moderation. Australian broadcasters operate under ACMA regulations that govern certain visual content, and automated angle detection can flag shots that may require review — for example, identifying low-angle footage of restricted zones or close-ups of identifiable individuals in sensitive contexts. While final editorial judgement remains human, the metadata accelerates the review process considerably.

Broadcasters evaluating the technology typically focus on three practical concerns: accuracy against their own content, processing latency during live operations, and the effort required to retrofit existing archives with new metadata tags. ReCAP's modular design allows pilot deployments on specific content types — such as nightly news or weekend sport — before broader rollout.

ReCAP's consortium is preparing trial access for broadcast engineering teams who wish to evaluate shot classification on their own content libraries, with documentation and pilot parameters scheduled for release in the next development cycle.