How ReCAP decodes embedded timecode and user data in broadcast video
Modern broadcast video is far more than just pictures and sound. Each frame carries metadata tucked into line ranges the eye cannot see, and ReCAP is built to read that metadata reliably. For Australian broadcasters like the ABC, SBS, Seven, and Nine, the integrity of these hidden signals can decide whether a live cross to a Melbourne newsroom or a Sydney sports ground lands cleanly or falls apart on air. The platform's analysis pipeline treats every ancillary packet and timecode pair as first-class content, not as incidental noise.
Timecode and user data streams have been part of the broadcast chain since the analogue era, when VITC sat invisibly between the active picture lines. Digital standards pushed these signals deeper into the SDI and SMPTE ST 2110 envelopes, where they now travel as structured ancillary data, KLV metadata, or IP packet payloads. ReCAP ingests all of these forms at line rate, preserving the relationship between the picture and the data that describes it. For teams operating across Brisbane studios and remote Tasmanian sites, that consistency keeps logs aligned even when sources arrive over different paths.
The challenge is that legacy equipment still produces timecode on the vertical interval, while newer IP workflows push everything through ST 2110 essences. ReCAP bridges those worlds without forcing operators to normalise inputs by hand. It decodes LTC, VITC, and ATC variants in parallel, and pulls user data from ANC packets, KLV groups, and SCTE messages as it goes. That breadth matters when an Australian production house is mixing archive tapes from the 2000s with today's IP-native cameras covering an AFL match at the MCG.
The pages that follow walk through how ReCAP handles these embedded signals, what formats it supports, and how its outputs feed back into production and asset management workflows. The article also covers the quality checks ReCAP performs on those streams, and where the project documentation goes deeper on related topics such as colour-space monitoring and artifact detection.
| Stream family | Physical location in the signal | Common standards | ReCAP's role |
|---|---|---|---|
| Linear timecode (LTC) | Audio channel alongside the picture | SMPTE ST 12 | Decoded, sample-rate corrected, and frame-aligned |
| Vertical interval timecode (VITC) | Lines in the vertical blanking interval | SMPTE ST 12, RP 188 | Read line-by-line, cross-checked against LTC |
| Ancillary timecode (ATC) | HANC packets in SDI or RTP payloads in ST 2110 | SMPTE ST 12, ST 2059 | Extracted from ANC and used for PTP relationship checks |
| Closed captions and subtitles | ANC packets (ST 334) or IP multicast groups | ST 334-1, ST 334-2, TTML in ST 2052 | Parsed, language-tagged, and validated for compliance |
| Active Format Description (AFD) | HANC packets | SMPTE ST 2016 | Read for aspect-ratio signalling, logged with frame index |
| SCTE-104 and SCTE-35 | ANC packets in SDI, PID in MPEG-TS | ANSI/SCTE 104, ANSI/SCTE 35 | Captured for ad insertion, blackout, and program boundaries |
| KLV metadata | ST 2110 data track or separate essence | SMPTE ST 336, MISB ST 0601 | Decoded into structured key-value records per frame |
| Audio loudness metadata | Audio metadata block in ST 2110 or embedded in AAC | EBU R128, ATSC A/85 | Read during the loudness pass, cross-checked against the audio essence |
SMPTE timecode variants and how ReCAP picks them apart
ReCAP's decoder distinguishes between LTC, VITC, and ATC at the earliest stage of ingest. LTC arrives on a dedicated audio pair and is sample-rate corrected so the time-of-day or program time values remain stable even when the source drifts. The pipeline reads 80-bit timecode words, validates the binary group flags, and flags drop-frame versus non-drop-frame so a producer in Adelaide editing for a national feed sees the same count as a director in Perth.
VITC is trickier because it lives on two lines of vertical blanking, traditionally lines 14 and 16 in 525-line systems, or equivalent positions in 625-line material. ReCAP reads both lines, compares the values, and reports a confidence score when they disagree. That disagreement often points to a faulty VTR or a dubbing error in a regional edit suite, both of which the ABC's Brisbane master control has flagged as recurring issues over the past year.
ATC ties timecode to a packet structure inside SDI ancillary space or inside SMPTE ST 2110 RTP flows. ReCAP parses the SMPTE ST 12 payload inside the ANC packet header, then matches it against the ST 2059 PTP timestamp taken from the IP wrapper. The relationship between the two is logged continuously, so a misalignment of even a single frame produces a documented event rather than a silent drift that would only surface in post.
Closed captions, subtitles, and accessibility metadata
Captions and subtitles in Australia move through a mix of CEA-608, CEA-708, and TTML-over-IP, depending on the broadcaster and the distribution target. ReCAP treats all three as structured user data and pulls them out of the same ANC or IP track that carries the timecode. The pipeline preserves the language code and the service number, so SBS's multilingual feeds and the ABC's English captions remain distinguishable in the metadata index.
The system validates caption frames against the underlying video, flagging missing entries, late deliveries, and overlapping text boxes that can occlude a scoreboard graphic. For a live event such as the State of Origin broadcast out of Sydney, these checks run at line rate and feed directly into a compliance dashboard.
TTML carried in ST 2052 documents is parsed into an internal model that exposes begin, end, and region attributes. ReCAP can then correlate the caption timeline with the program timecode, which simplifies downstream tasks like re-versioning for catch-up players or generating transcripts for the broadcaster's archive.
Active Format Description and aspect-ratio signalling
AFD travels in an ANC packet defined by SMPTE ST 2016, and it tells downstream devices whether a 16:9 frame should be presented as full-frame, letterboxed, or pillar-boxed. ReCAP reads the AFD code, pairs it with the coded frame number, and stores the pair as part of the per-frame metadata record. This is essential when a station like Seven must intercut 4:3 archive material with 16:9 field footage without manual adjustments.
The platform also handles bar data, VANC signalling for safe action, and the related RP 186 descriptors. These flags are particularly important for Australian drama productions that shoot for both domestic and international release, where the same master needs to satisfy Freeview Australia's safe-area guidelines as well as European delivery specs.
When AFD values change mid-program, ReCAP logs the transition and emits a structured event that an automation system can react to. In a Sydney post house, that event might trigger a graphics repaint or a switch to a different layout for a sports promo cut.
SCTE-104 and SCTE-35 for live switching and ad insertion
SCTE-104 messages live in HANC packets on SDI and represent the in-facility cue language for ad breaks, blackouts, and program boundaries. ReCAP captures every splice request, then maps it onto a parallel SCTE-35 timeline when the signal exits as MPEG-TS. The combined view tells the operator exactly when a regional Foxtel affiliate needs to insert a local ad.
The decoder is aware of pre-roll, mid-roll, and post-roll tiers, and tags them with the matching splice event type. For rights holders covering cricket or rugby league, this granularity is what makes sure the regional blackout window opens and closes on schedule across all distribution paths, from the Sydney head-end to the regional transmitter at Tamworth.
When SCTE messages arrive late or arrive out of order, ReCAP raises a quality event. The same logic applies to private commands that may carry affiliate identifiers or dynamic ad-decisioning payloads, which the platform logs for later reconciliation against the playout system.
KLV metadata from outside broadcast trucks and drone feeds
KLV is the universal language for MISB-compliant metadata, and it shows up whenever a camera or sensor platform speaks the ST 0601 dialect common in defence, emergency services, and high-end sports coverage. ReCAP ingests KLV from a dedicated ST 2110 data track or from a sidecar file, and decodes each local set into structured fields.
For an outside broadcast truck covering the Bathurst 1000, the KLV stream typically carries GPS coordinates, platform heading, sensor aim, and timestamps. ReCAP synchronises these against the picture timecode so that every frame has a precise geographic tag. The platform also handles dynamic KLV, where the local set changes mid-clip to add or drop metadata items as the platform configuration changes.
The decoded KLV records are exposed through a queryable interface, which is useful for compliance, post-production search, and rights management. A producer searching for footage captured within a council boundary, for example, can filter by latitude and longitude without leaving the asset management system.
Audio metadata, language tags, and loudness descriptors
Audio carries its own hidden metadata, and ReCAP reads it alongside the picture. Dialnorm values, language tags, and content classifier flags travel in the audio metadata block of ST 2110 essences or in the user_data field of an AAC stream. The pipeline decodes these fields and surfaces them as part of the per-frame record.
Loudness descriptors based on EBU R128 or ATSC A/85 are checked against the actual audio essence. ReCAP measures integrated loudness, true peak, and momentary range, then compares the result with the metadata claim. Disagreements are flagged, which is particularly useful for catch-up services that have to respect the ACMA's loudness rules for Australian audiences.
Multichannel layouts such as 5.1 or immersive formats like Atmos are described by channel count and object metadata, and ReCAP records the configuration as part of the audio fingerprint. When a sports broadcast mixes match commentary with crowd ambience, this fingerprint helps the asset management system rebuild the original listening experience for re-versioning.
Real-time validation and quality reporting
ReCAP validates every embedded stream against the relevant SMPTE, ATSC, or SCTE standard as the data flows through. The validation engine checks structural integrity, continuity, and consistency across frames, and emits structured events when any rule is broken. The events feed a dashboard that operators in Melbourne's master control rooms consult during live shifts.
The platform also correlates timecode, AFD, and caption streams against each other. A caption that arrives after the corresponding frame, or an AFD change that happens without a matching splice cue, generates a composite quality event that points to the root cause. ReCAP's related work on monitoring colour-space conversions follows the same pattern for colour metadata, applying the same continuity logic across a related field.
For artifacts introduced by upstream processing, ReCAP's quality module also covers ghost and ringing detection, which often appear together with broken user data after a faulty upconversion. The combined checks give an Australian broadcast engineer a single picture of the signal's hidden layer alongside its visible one, all without leaving the live monitoring console.
Reports from ReCAP can be exported as JSON or CSV, and integrated into the broadcaster's existing logging infrastructure. The result is a workflow in which embedded timecode and user data streams are not just preserved, but actively validated and made searchable from ingest through archive.
ReCAP's next release will extend the KLV decoder to cover emerging MISB local sets used by unmanned aerial platforms operating under Australian Civil Aviation Safety Authority rules. Engineers at the consortium partners are already testing the updated parser against footage captured during trials over regional New South Wales, and the project's public demo page will carry the first recordings once the validation cycle closes in the spring.