In digital processing with a infinite impulse response filter (IIR), precision refers to the number of bits, or numeric resolution, that is used to represent the input signal, coefficients, intermediate calculations, and the resulting output. Two of the consequences of increasing or decreasing precision are to change the filter:

• Response accuracy

• Signal- to- noise performance

• Coefficient quantization

Quantization is a process where a continuous value (a real number) is translated into a fixed precision representation. The difference between the original value and the quantized value is the quantization error.

Quantization error can cause a filter response that is different from the desired filter in amplitude, phase and/or frequency. The difference is the filter response error. In general, the filter response error increases when:

• Filter center frequency decreases

• Sample rate increases

• Q of the filter increases

• Filter gain increases

The effects of coefficient quantization and sampling rate can be visualized as the amplitude response error of four parametric equalization filters that have, say, a gain of 6 dB with a Q of 6.7. A 200 Hz filter using 20 bit coefficients at a 48 kHz sample rate and three 50 Hz filters of 20 bits at 48 kHz, 24 bits at 96 kHz and 28 bits at 192 kHz sample rates yield substantially different responses.

With an audio DSP, the coefficients are encoded using a twos complement binary format with 4 bits assigned for the integer and 12, 16, 20, or 24 bits assigned for the fractional value. The smallest error predictably occurs with the highest bit precision and the widest sampling frequency.

The error magnitude increases when the number of bits used to represent the coefficients decreases and when the sample rate increases. The magnitude of the filter response error produced by 16 and 20 bit coefficient quantization at 48 kHz is actually greater than the 6 dB gain of the respective 200 Hz and 50 Hz filters.

To achieve filter response errors less than 10 percent for a 50 Hz filter at 96 and 192 kHz sample rates, coefficient sizes of 28 bits or larger are required.

Quantization in the filter response calculation also impacts system performance. Quantization occurs in the filter response as a result of the physical constraints of the processing architecture.

An example of this is can be illustrated in theDirect Form 1 filter architecture, where single sample delays are represented as Z-1 blocks. The input signal and the output signal are represented in A bits. The a and b coefficients are represented in B bits. The multiplication of signals with the b and a coefficients is performed by an A bit by B bit multiplier that produces an A+B bit result. An A+B bit wide adder is then used to sum the products and compute the unquantized filter response.

Quantization occurs at the output of the adder, in the highlighted "Q" block, where the A+B bit result is quantized into A bits. This quantization step produces a quantization error. This quantization error is the noise source of the digital filter. The noise characteristics are determined by the magnitude of the quantization error and the quantization type (round-off, twos complement, or signed magnitude truncation).

The filter noise produced by quantization is calculated by replacing the Q by a noise source that is summed with the filter output. The noise output from the filter, f(n), is a function of this noise source, e(n), and the a coefficients of the filter.

f(n)=e(n) - a1f(n-1) - a2f(n-2)

The filter noise is proportional to the quantization error. Increasing the calculation precision reduces the quantization error. Consequently, increasing the calculation precision will reduce filter noise. The additional bits that are added are called noise bits.

How many noise bits?

To illustrate the characteristics of noise in IIR filters, a parametric equalization filter with a gain of 12 dB and a Q of 6.667 is evaluated at 1000 Hz, 100 Hz, and 30 Hz.

In the case of parametic equalization filters, the magnitude of noise response increases when:

• Filter frequency decreases

• Sample rate increases

• Filter Q increases

The noise transfer function (NTF) describles the amplitude of filter noise with respect to frequency. The noise transfer function is computed by collecting the terms of the noise difference equation and taking the z-transform.

NTF= 1/(1+a1z-1 + a2z-2)

For magnitude truncation, the characteritics of the noise source are:

Uniform distribution over +/- 2-2b, where b is the number of noise bits.

If 0 dB line represents the lowest audible signal amplitude, 8 noise bits are sufficent to keep the noise of a 1000 Hz filter below the signal - but this is not sufficient for the 100 or 30 Hz filters. Similarly, 16 noise bits keep 1000 and 100 Hz filters below the lowest signal level, but at 30 Hz, the noise cuts into the signal response by 11 dB.

Consequently, to preserve the SNR of a 24 bit signal with no more than 11 dB of degration at 30 Hz at a 48 kHz sample requires a system with a minimum of 40 bit data precision (24 signal + 16 noise).

At 96 kHz, the noise amplitudes increase by roughly 10 dB acrss the spectrum. As a result, to preserve all of the input signal without degradation would require 20 noise bits. An increase from 96 kHz to 192 kHz sampling rates produces a similar increase in the noise amplitude.

In digital filters, precision determines the filter response accuracy and signal to noise performance. The amount of precision that is required to achieve a particular level of performance is dependent upon the sample rate and filter characteristics. A general purpose, high performance audio signal processing architecture would need on the order of 48 bit precision with 28 bit coefficients.

See related chart