ReCAP and the live tagging of pitch-side sponsor boards

The Real-time Content Analysis and Processing project, known as ReCAP, is an EU-funded research initiative developing automated tools for broadcast-grade video understanding. Its remit spans metadata extraction, quality monitoring, face and logo recognition, and duplicate detection, with a focus on supporting media production, live transmission, and asset management workflows. Among the most demanding applications is the recognition and tagging of stadium advertising boards during live sporting events, where computer vision must operate under tight time constraints and unpredictable visual conditions.

Stadium hoardings, also called pitch-side boards, perimeter signage, or LED ribbons, are a moving target for any recognition system. Cameras pan across the field, zooms shift rapidly, players and officials occlude sections of the signage, and the boards themselves change content every few seconds in some leagues. For Australian broadcasters covering the AFL, NRL, or Big Bash League, where fixtures are spread across multiple venues in a single weekend, automating the cataloguing of on-screen sponsor exposure delivers both commercial accountability and editorial value.

ReCAP's approach treats each visible advertising panel as an object to be detected, tracked, classified, and described over time. The system combines object detection with optical flow analysis to maintain identity through partial occlusion. Once a board region has been isolated, classification models match its visual signature against a reference library of sponsor logos, with optical character recognition used to read textual brand names and campaign slogans. The result is a time-stamped, machine-readable log of every brand appearance.

The economic stakes around perimeter signage are highest during live coverage, when exposure is hardest to quantify. Rights holders, sponsors, and agencies still rely on post-match reports assembled manually from recordings. ReCAP's tooling is designed to close that gap by delivering near real-time visibility into brand exposure, allowing broadcast partners to verify contractual obligations and produce richer analytics for advertisers across multiple markets.

Computer vision workflow for perimeter advertising recognition

ReCAP's recognition engine for live match footage is built around a layered computer vision stack. The first layer performs spatial detection, segmenting the regions of the frame that contain LED ribbons, static hoardings, and rotating digital panels. Boards often sit at oblique angles relative to the main camera, and their content is designed to attract attention rather than remain uniform. The detection model is trained on a wide corpus of European football, rugby, and basketball footage, with adaptation modules for cricket and Australian rules football added through transfer learning.

Once regions are detected, a tracking module maintains identity across frames, even when players block portions of the board or the camera cuts between angles. The tracker relies on appearance embeddings and motion estimation, so a sponsor panel partially hidden behind a defender is still matched to its previous identity when it reappears. This continuity is essential for accurate duration logging, because a fragmented record would miscount sponsor visibility.

The third layer handles recognition itself, distinguishing between two complementary tasks. For branded panels, a convolutional classifier compares the panel image against a curated set of sponsor templates, returning a confidence score and a brand identifier. For text-heavy boards displaying campaign taglines or bookmaker odds, an OCR pass extracts the string, normalises it, and resolves it against a campaign database. Both outputs are aligned on a unified timeline, producing a continuous record of advertising exposure for every camera angle selected by the director.

Latency is a constant constraint. The recognition pipeline is engineered to produce tagged output within a few seconds of the broadcast frame, fast enough for live overlay decisions, automated clipping, and real-time compliance alerts. Where edge compute is available at the venue, processing can happen on-premise; otherwise, lightweight streams are forwarded to a cloud node for inference, with results returned to the production gallery through a low-overhead messaging interface.

Detection, classification, and OCR in practice

The technical core of the system can be summarised by the kinds of panels it has to interpret during a single match. A useful way to think about the challenge is to compare the dominant board categories and the recognition strategies applied to each.

Board category Visual behaviour Primary recognition method Typical challenges
Static perimeter hoarding Fixed content, long exposure Template logo matching, OCR Camera angle distortion, partial occlusion
LED ribbon Animated, content rotation Frame differencing, periodic OCR Rapid content change, motion blur
Digital rotating panel Multiple creatives per match Multi-class classifier Sudden transitions, lighting variance
Virtual overlay region Rendered graphics, region-based Geometric analysis, template matching Depth ambiguity, scene complexity

The grid illustrates how ReCAP's pipeline adapts its strategy to the visual character of each board type. Static hoardings reward template-based matching because the content stays consistent, while LED ribbons require temporal analysis to read rotating messages. Virtual overlay regions, increasingly common in modern broadcasts, are detected through geometric and rendering cues rather than pixel content alone.

Common recognition difficulties encountered during live matches:

Tagging output and metadata for media asset management

The output of the recognition stage is only useful if it can be ingested by the production and rights management systems around it. ReCAP exposes its tagging results as structured metadata aligned with widely used broadcast schemas, including EBUCore extensions developed within the broader media community. Each detected brand appearance is tagged with a start timecode, an end timecode, the camera identifier, a confidence score, and the sponsor's canonical identifier in the project's reference taxonomy. This metadata is written back into the media asset management platform alongside the broadcast record.

Tagging is deliberately fine-grained. Rather than logging a single "Sponsor X was on screen" record per match, the system captures every visible instance, including those that last only a few seconds. This granularity matters because sponsor contracts often distinguish between premium slots and incidental exposure, and because agencies increasingly measure value through cumulative in-broadcast time rather than placement alone. The system also flags duplicate detections, collapsing redundant matches when the same panel is observed across multiple camera cuts.

For media monitoring partners, the metadata stream opens the door to automated report generation. Instead of waiting for a human reviewer to assemble a deck of screenshots, monitoring teams can query the timecoded log directly, pulling every frame in which a particular logo appeared, along with duration totals broken down by match phase. The same log feeds rights management dashboards, helping rights holders verify that contractual obligations around category exclusivity have been honoured during the live window.

A practical operational element of the project involves coordinating processing windows around fixture schedules, where scheduling reminder tools help the consortium's engineers manage processing windows so that no venue feed is left unanalysed across a congested weekend of fixtures.

Australian broadcast and sport applications

Australia presents a rich environment for testing stadium advertising recognition, because the country runs several major codes across overlapping seasons. The AFL season stretches from March through September, with matches spread across the Melbourne Cricket Ground, Docklands Stadium, Adelaide Oval, the Gabba in Brisbane, and Optus Stadium in Perth. Each of these venues features a mix of static perimeter signage and rotating LED ribbons, and the league's central sponsor arrangements mean the same brands appear repeatedly across the country. For a national rights holder, automated recognition across every round delivers a level of consistency that manual logging cannot match.

The NRL, played across Sydney, Brisbane, Melbourne, and other centres, adds another layer of complexity with its denser fixture calendar and its reliance on Accor Stadium configurations during the early season. Big Bash League cricket, played in the southern summer, features shorter matches and faster camera cuts, which stress the tracking module in different ways. The A-League, while smaller in broadcast terms, often features international sponsors that change from round to round, demanding a continuously updated reference library.

Local broadcasters such as the Seven Network and Foxtel, along with streaming services like Stan Sport, are already accustomed to advanced graphics overlays and in-stream sponsor integrations. Adding automated board recognition to their production pipelines gives those overlays an additional layer of accountability, while freeing editorial staff from the manual logging that has long accompanied live coverage. For sponsors, the same data feeds back into planning, helping agencies evaluate which venues deliver the strongest brand recall across a season.

Australian venues and competitions where the system has been benchmarked:

Sponsor verification and virtual replacement

One of the more practical applications of accurate advertising recognition is the verification of in-match sponsor exposure. Sponsorship contracts routinely specify minimum durations, category exclusivity, and placement within certain camera angles. A recognition log tied to the broadcast record allows both sides of a deal to audit performance after the fact, without relying on subjective interpretation of match footage. ReCAP's confidence scoring is exposed to the verification dashboard, so borderline detections can be flagged for human review rather than silently counted.

Virtual replacement, in which a different advertisement is composited into a designated region of the frame for different broadcast markets, is closely related. To replace an overlay cleanly, the system first has to identify the precise geometry of the target region across the match. ReCAP's geometric analysis module produces the masks that drive virtual replacement workflows, and the same masks double as analytical artefacts for post-match reporting. For international tournaments broadcast into the Australian market, virtual replacement also helps tailor sponsor messaging to local audiences.

The combination of recognition and replacement is particularly relevant for governing bodies that sell regional rights packages. By automatically identifying the panels reserved for each market, the system can route the correct creative into the relevant feed, with the recognition log providing the audit trail that accompanies every substitution. Over time, the aggregated data also reveals which panel positions carry the most broadcast value, informing future sponsorship pricing for rights holders across the Australian market and beyond.

Integration with production and rights workflows

ReCAP's stadium advertising recognition is designed to slot into existing broadcast and rights management workflows without requiring wholesale replacement of established systems. Recognition jobs can be triggered automatically as camera feeds are ingested, with results returned to the production gallery through standard metadata sidecars. For cloud-native broadcasters, the same pipeline runs as a managed service, with the consortium providing reference templates and configuration support. Edge deployments at the venue allow latency-sensitive uses, such as live overlay decisions, while cloud inference supports batch analytics and post-match reporting.

Quality monitoring remains a continuous background task. The system flags degraded frames, missed detections, and recognition failures, so that operators can intervene where needed. The feedback loop between human review and model retraining helps the recognition engine adapt to new sponsor templates and venue-specific quirks. Integration with the project's broader face and logo recognition modules also means that incidental sponsor exposure, including logos on player jerseys or staff apparel, can be captured in the same metadata record.

For media asset management platforms, the integration is straightforward. Recognised sponsor appearances become searchable tags within the library, allowing editors to assemble sponsor reels, compliance reports, and highlight packages without leaving the MAM interface. The same tags support downstream uses, including ad insertion decisions, content recommendation, and rights clearance, all of which benefit from a richer, more accurate description of what appears in each frame.

A secondary use case for the project's broader research base, including work documented on the casino-terugval-voorkomen-gokken page, has informed how the consortium models exposure duration and cumulative brand recall across the visible gaming perimeter category, a topic of growing regulatory interest in several European markets.

Stadium advertising recognition, once a manual and error-prone exercise, is becoming a real-time, machine-driven layer of the modern broadcast stack. ReCAP's contribution is to make that layer reliable enough for production use, accurate enough for contractual audit, and flexible enough to support both live overlay workflows and post-event analytics across multiple sports, including the Australian competitions that test the system against some of the most varied board layouts in world sport.