Algorithmic delay and synchronization in MPEG audio codecs

In designing time-critical systems like conferencing, broadcast transcoding systems or be it in designing any audio and video play-out system, the knowledge of the delay encountered while audio encoding or decoding becomes critical.

Figure 1 tries to capture the various stages at which audio data encounters delay in different applications and systems. The delay encountered in audio systems can be broadly classified as follows -

Processing Delay: Computing any algorithm on a processor consumes a finite amount of time. This delay would be inversely proportional to the CPU speed and the CPU capability to compute complex logic with minimal cycles.

Algorithmic delay: The core processing during audio compression and decompression involves techniques which work on a frame of samples and inherently introduce delay. This delay can be thought of as similar to the delay introduced while digital filtering.

Application Delay: Applications involve various forms of buffering for either smooth play-out or packetization and streaming which introduce delay. IP streaming or transmission over a constant bit-rate channel introduce delays based upon the available bandwidth.

The focus of this article is on the algorithmic delay introduced due to audio compression and decompression. This is dependent primarily on the chosen compression technology and can be quantified in terms of sample lag of output with respect to input.

This article also discusses utilizing this knowledge of algorithmic delay to synchronize parametric processing modules like SBR (Spectral Band Replication), PS (Parametric Stereo) & MPEG Surround when used alongside transform coding technologies like AAC. The article aims to elucidate the algorithmic delays resulting from various modules used in MPEG audio codecs, and compares performance of the codecs with respect to this attribute.

The first few sections of the article examine the various modules which introduce delay like - Filter banks (FB), Modified Discrete Cosine Transform (MDCT) and bit reservoir module used in MPEG audio codecs. The later sections discuss the algorithmic delay encountered in popular MPEG codecs such as MPEG Layer2, MPEG Layer 3, AAC-LC, AAC-LD, HE-AAC, HEAACv2 and MPEG Surround and also explain the synchronization required while using parametric encoding tools. The article assumes that the reader has an overview of MPEG audio compression technology and focuses primarily on the delay introduced during compression and decompression.