On-screen QR and data matrix recognition with ReCAP's real-time workflow

Broadcast graphics have quietly become one of the richest metadata sources in modern television. From station identifiers flashed in the corner of a news bulletin to promotional QR codes that invite viewers to scan for a recipe, a single frame of broadcast-quality video can carry dozens of machine-readable symbols competing for attention. The ReCAP project treats these symbols as a structured data layer rather than visual noise, building tools that automatically detect, decode and log 2D codes as part of broader real-time content analysis pipelines.

This matters because manual logging of on-screen QR codes and data matrices is impractical at broadcast scale. A regional Australian network producing dozens of live feeds each day, with continuous streams flowing out of Sydney, Melbourne and Brisbane studios, simply cannot rely on a human operator to catch every code that flashes on screen. Automated recognition changes that equation by capturing each code, decoding its payload and stamping a verifiable entry into the media asset management system within seconds of transmission.

The ReCAP project website describes the consortium behind the initiative and the technical goals that guide each phase of development. Understanding how the recognition pipeline fits together, however, requires a closer look at the workflow itself.

How ReCAP detects codes inside video frames

Recognition begins the moment a frame leaves the vision mixer. ReCAP's pipeline runs a fast pre-filter that scans each frame for regions with the high-contrast, geometrically regular patterns characteristic of QR codes, data matrices and the related 2D symbologies such as Micro QR and rectangular MicroPDF417. Candidate regions are flagged rather than fully decoded at this stage, keeping the computational cost manageable even when streams are processed live.

Decoding itself relies on the standard error-correction algorithms defined in each symbology family. ReCAP's workflow decodes QR payloads using the Reed-Solomon correction layer built into the symbol, allowing partial recovery even when a frame is blurred, partially occluded by a presenter or compressed by the broadcast encoder. Data matrices use a similar but distinct error model, and the pipeline keeps separate decoders for each family to avoid cross-talk between symbol types.

Once a code is decoded, the system extracts the payload, normalises it and writes a structured log entry. The log includes the timestamp, the frame number, the decoded payload, the symbology type and any partial-decoding confidence scores. This record is the foundation for downstream tasks such as content verification, broadcast monitoring and rights tracking across the production chain.

The blog post on on-screen QR code recognition walks through a demonstration in which a studio logo and a sliding QR overlay are both captured from a live feed, decoded and indexed alongside the program's master metadata.

Where Australian broadcasters can use the workflow

Australia's free-to-air networks, including the ABC and SBS alongside the commercial metropolitan broadcasters in Sydney and Melbourne, have long used on-air bugs as branding devices. Adding machine-readable codes to those graphics opens a new layer of interactivity without requiring viewers to download a separate app. A cooking segment on a Brisbane-based lifestyle program, for instance, can flash a QR code that links to the full recipe, with the broadcaster using the same scan data to measure engagement after the fact.

Newsroom workflows benefit from the pipeline as well. During a rolling news bulletin, producers regularly flash source URLs, social handles or document references as on-screen graphics for a few seconds each. ReCAP's workflow captures each of these short-lived visual artefacts automatically, creating a verifiable record of what was shown and when. That record is increasingly valuable for fact-checking and compliance reviews, particularly when a story is later contested or its sourcing is challenged.

Sports broadcasts present another natural application. Australian rules football and cricket coverage routinely show team logos, sponsor identifiers and competition marks across the screen, and live betting integrations already rely on overlays that include scannable codes. Automatic detection and logging allows rights holders to confirm that sponsor marks appeared for the contracted duration and that no unauthorised logos entered the frame.

Concrete deployments tend to cluster around a few predictable use cases:

Australian privacy and regulatory context

Capturing and storing every on-screen QR code raises legitimate questions about data handling. Australia's Privacy Act 1988, together with the Australian Privacy Principles (APPs), governs how organisations collect, store and disclose personal information, and QR codes that link to identifiable accounts or carry personal details fall squarely within that framework. Broadcasters running ReCAP's pipeline need to ensure that decoded payloads containing personal identifiers are handled according to their existing APP-compliant retention policies.

The Australian Communications and Media Authority (ACMA) also plays a role, particularly where codes are used as part of regulated content such as gambling advertising or political broadcasts. Real-time logging of codes linked to wagering offers, for example, can serve as proof of compliance with ACMA's rules on responsible placement of betting content during live sport. A consistent, timestamped record of every code shown during a broadcast is far more defensible than relying on a post-production review to reconstruct what appeared on screen.

For organisations processing this data, network security is a related concern. Decoded payloads often flow through studio infrastructure that may touch guest networks, satellite uplinks or third-party cloud services. A clear-eyed look at public and private Wi-Fi differences explains why broadcasters should isolate their recognition workloads on secured private infrastructure rather than relying on consumer-grade wireless. This is especially important when live crews are filing from outside broadcast vans at venues in Perth, Adelaide or regional centres, where wireless connectivity is often the weakest link in the chain.

Technical challenges of on-screen code recognition

Recognising codes on a static printout is straightforward, but broadcast video introduces several complications. Frames are encoded, decoded and re-encoded multiple times between camera and viewer, and each pass introduces compression artefacts that can blur or distort the modules inside a QR code. ReCAP's pipeline compensates by training its detectors on heavily compressed footage, recognising that the symbol captured by a viewer watching a 7pm drama in suburban Melbourne may look nothing like the pristine version rendered in the studio. Rolling shutter, motion blur and the rolling ticker bands that sit across the bottom of many news bulletins all add further noise that a robust decoder must learn to ignore.

Backgrounds are another hurdle. QR codes overlaid on busy footage can blend into the scene, particularly when colours clash or the symbol sits over similarly patterned clothing, water or grass. Outdoor sport coverage, common across Australia's summer cricket season, frequently places sponsor codes over the green of the field, and the decoder must learn to separate symbol from substrate reliably. Studios in Brisbane can experience bright midday sun flooding through windows, while winter broadcasts in Hobart may rely on much lower ambient light.

The recognition model is designed to generalise across these lighting extremes so that detection rates remain consistent regardless of the conditions under which the footage was captured. Consistency across conditions is what separates a research prototype from a tool that broadcasters are willing to trust with their compliance records.

Logging, metadata and downstream use

The structured log produced by the recognition pipeline feeds directly into media asset management systems. Each entry carries enough detail for downstream tools to reconstruct exactly what was shown, where in the frame it appeared and how reliable the decoding was. This metadata layer makes it straightforward to search archives for every broadcast that included a particular URL or sponsor mark, which is useful for both commercial reporting and rights audits.

Integration with existing broadcast tools is a deliberate design goal, and the ReCAP consortium has prioritised compatibility with widely used MAM and playout platforms. A small but representative set of items illustrates what the logging layer records for each detected code:

This combination of attributes lets production teams reuse the data for many purposes, from post-broadcast analytics to compliance reporting. The same log can drive a dashboard that shows how often a sponsor's QR code appeared, fuel a content verification tool that flags unauthorised reuse of brand marks, or feed search interfaces that let archivists locate a specific broadcast from years earlier by the code that appeared in it.

Bringing the workflow into a live production

Deploying ReCAP's recognition tools in a live environment requires careful staging. The pipeline is typically deployed as a sidecar service that consumes a low-latency feed from the playout chain, allowing it to operate without becoming a critical path for the broadcast itself. If the recogniser drops a frame or fails to decode a symbol, the broadcast continues unaffected and the missed entry is simply absent from the log.

Tuning the detector for a particular channel's house look is part of the rollout. Stations with consistent graphic templates can configure the recogniser to focus on the regions where codes typically appear, reducing false positives and improving throughput. Tuning is also where local context matters: a broadcaster in Adelaide may use different overlay placements than one in Perth, and the detector should be calibrated accordingly so that house style does not become a source of false negatives.

After deployment, ongoing monitoring keeps the system honest. Recognition accuracy is tracked over time, with operators alerted when confidence scores drop or when an unusual number of undecodable frames appear. This kind of operational discipline tends to determine whether a recognition system survives contact with the realities of 24-hour broadcasting.

Treat on-screen codes as a first-class metadata layer rather than disposable overlays, design the logging pipeline to capture every detected symbol with full provenance, and ground the deployment in the legal and network realities of the jurisdiction where the broadcaster operates.