Advantages and Disadvantages of Using DSP Filtering on Oscilloscope Waveforms

All of today's high-speed real-time sampling oscilloscopes use various forms of digital signal processing (DSP) on digitized oscilloscope waveforms. Some engineers are concerned that filtering digitized data with software may alter the true nature of a captured signal. However, the captured waveform is only a representation of the actual input signal, and "raw" digitized data captured by an oscilloscope includes altered/distorted results contributed by the scope's front-end hardware filtering. In a perfect world, real-time oscilloscopes would have infinitely fast sample rates, perfectly flat frequency responses, linear phase responses, no noise, and infinite bandwidth. But in the real world, oscilloscopes have hardware limitations that produce errors. DSP filtering ultimately can correct for hardware-induced errors to improve measurement accuracy and enhance display quality.

There are five different characteristics of DSP filtering commonly employed in today's higher-performance real-time scopes (Table 1).

Each of these filter characteristics can be implemented in a single finite-impulse response (FIR) software filter in real-time sampling oscilloscopes. This article explores the purposes of these different characteristics of DSP filters, and discusses the benefits and possible tradeoffs associated with each one. This article does not provide information about the actual software implementation of the various DSP filters.

The simplest type of waveform reconstruction uses a linear-interpolation filter. Although this type of filter will improve measurement resolution, accuracy, and display quality, a more accurate type of interpolation is sin(x)/x reconstruction filtering, which is a symmetrical filter. Figure 1 shows an example of a 3-GHz sine wave captured and filtered with linear reconstruction (top/blue trace) and sin(x)/x reconstruction (bottom/yellow trace). With linear reconstruction, we can clearly see the discreetly spaced 50-ps sample points generated by this 20-GSa/s oscilloscope.

Figure 1:  Linear vs sin(x)/x reconstruction

Sin(x)/x filtering will almost always provide a more accurate representation of the input signal with a few caveats. First of all, for sin(x)/x reconstruction filtering to be absolutely accurate, the digitized input signal must not possess any frequency components beyond the Nyquist frequency (f_N). The Nyquist frequency is defined to be of the sample frequency (f_S). For a scope that can sample at 20 GSa/s, the Nyquist frequency is 10 GHz. To provide maximum bandwidth while guaranteeing that no frequency components beyond 10 GHz are ever sampled, the oscilloscope theoretically must have a hardware brickwall filter at 10 GHz or lower. Unfortunately, brickwall filters are not physically realizable in hardware. The red trace (top) in Figure 2 represents the characteristics of a brickwall filter; all frequency components below the Nyquist frequency are perfectly passed, and all frequency components above the Nyquist frequency are perfectly eliminated.

Figure 2:  Various hardware filter responses

In the past, lower-bandwidth scopes typically have had Gaussian-type roll-off characteristics, as represented by the green trace (bottom) in Figure 2. If you are digitizing very fast signals using this slow roll-off characteristic, there often will be significant components of the signal above the -3dB bandwidth point. Frequency components beyond the Nyquist frequency (represented by the hashed area in this graph) will be aliased. If a digitized signal is grossly aliased where the fundamental input frequency is beyond the Nyquist frequency, the displayed waveform will appear to be untriggered when you are viewing repetitive real-time acquisitions, and measurements of digitized points may be in error by orders of magnitude. When the input signal's fundamental input frequency is below the Nyquist frequency, but harmonics of the signal are beyond the Nyquist frequency, you may observe a waveform on the oscilloscope's display with edges that "wobble." For this reason, Agilent Technologies traditionally has limited the bandwidth of lower-bandwidth real-time scopes that have Gaussian roll-off characteristics to the sample rate, which is the Nyquist frequency. This significantly limits the captured energy of signals with harmonic-frequency content beyond the Nyquist frequency.

For some of the newer higher-bandwidth, real-time scopes with bandwidths from 2 GHz to 6 GHz, the hardware roll-off characteristic begins to approach a theoretical brickwall filter. In most oscilloscope measurement cases, this is a desirable characteristic. This type of hardware filter, called a high-order maximally flat filter, is illustrated by the blue trace (middle) in Figure 2. With this type of hardware filter, most of the in-band frequencies are passed with minimal attenuation, and most of the out-of-band frequencies are significantly attenuated. With a high-order maximally flat response, the scope's bandwidth can then begin to approach the Nyquist limit. Agilent recommends that for scopes with a high-order maximally flat response, the bandwidth of the scope should be limited to no more than 0.4 times the sample rate. In other words, for waveform reconstruction using sin(x)/x filtering to be effective and accurate, the bandwidth of a scope that samples at 20 GSa/s should not exceed 8 GHz.

What are the tradeoffs in employing a sin(x)/x software reconstruction filter in an oscilloscope? If the input signal is initially band-limited, or if the hardware of the oscilloscope properly limits the sampled frequency components beyond the Nyquist frequency, the tradeoffs are minimal. But if the input signal has significant high-frequency components beyond the system bandwidth, one artifact of sin(x)/x filtering is the possibility of software-created pre-shoot and over-shoot of the reconstructed waveform. This effect is essentially Gibbs phenomena. The software-created over-shoot is often hidden by inherent over-shoot in the actual input signal, as well as over-shoot created by the scope's hardware filtering. Because pre-shoot is usually not actually present in the signal, oscilloscope users often question the validity of sin(x)/x filtering. But software-induced errors such as pre-shoot can pale in comparison to uncorrected hardware-induced errors when you are measuring out-of-band signals.

Remember, measuring an out-of-band signal simply means that you are attempting to capture a signal which has frequency components beyond the specified bandwidth capability of the oscilloscope. This means that measured results can include significant components of error due to hardware limitations. For example, if you attempt to measure an input signal with an edge speed of 20 ps (10% to 90%), a 6-GHz oscilloscope will produce measured edge speeds in the range of 70 ps, which is a 250 percent error. Although pre-shoot and over-shoot produced by software filtering may be intuitively disturbing, these phenomena are minor sources of error compared to hardware-induced over-shoot and edge-speed errors, which are often overlooked.

To reduce software-induced pre-shoot, oscilloscope designers could employ sin(x)/x reconstruction filtering without phase correction to the acquired out-of-band waveform. Although the resultant filtered waveform that exhibits lots of over-shoot with minimal pre-shoot may feel more comfortable, accuracy of amplitude and edge-speed measurements will be degraded. Proper DSP filtering with linear phase correction will produce the most accurate measurements on fast rising and falling edges. (Phase-correction filtering will be discussed in more detail later in this article).

The best approach is to try to ignore the pre-shoot artifact and take this unintuitive "wiggle" at the beginning of fast-edge pulses as a sign that the real-time oscilloscope is employing a DSP filter that most accurately represents the overall characteristics of the out-of-band input signal. You also can take the pre-shoot artifact as a sign that you are pushing the real-time oscilloscope beyond its intended bandwidth measurement capabilities. You may want to consider using a higher-bandwidth sampling oscilloscope, such as Agilent's 86100C, for your measurement application. If repetitive sampling is not a possibility, then you may need to just accept the real-time measurement results as the best that is possible with today's real-time sampling and filtering technology.

As previously mentioned, sin(x)/x DSP filtering will significantly improve measurement resolution and accuracy to well beyond the real-time sample interval (1/sample-rate). With Agilent's 20-GSa/s 54855A oscilloscope, delta-time measurement accuracy can be improved to +/-7 ps (peak) with the use of sin(x)/x filtering on single-shot acquisitions. In some cases there are also throughput tradeoffs when you use sin(x)/x filtering. In other words, the filter causes your scope display to update more slowly. However, the enhanced accuracy advantages of using sin(x)/x filtering typically far outweigh all disadvantages.

All major real-time scope vendors today allow you to decide if you want to use sin(x)/x filtering. This mode of operation is a default selection in Agilent oscilloscopes, but you can override this selection if you choose.

Figure 3:  Magnitude-flattening filter response

The red trace (top) in Figure 3 shows the typical hardware/analog frequency response of Agilent's 54855A real-time 6-GHz oscilloscope. As you can see, the hardware response of this scope meets the -3dB hardware bandwidth criteria of 6 GHz, but the response also shows about +1dB of peaking at approximately 3.5 GHz, and nearly +2dB of peaking at approximately 5.5 GHz. Oscilloscope manufacturers today do not specify the flatness of their scope's frequency response. The only point in the frequency domain scope makers specify is the -3dB bandwidth point. Even if a scope had +6dB of peaking, which would translate into 60 percent amplitude error at a particular in-band frequency, as long as the -3dB point is higher than the specified bandwidth, the scope is considered to be within specification. But just as attenuation at higher frequencies can degrade the accuracy of measurements, so can amplification/peaking degrade measurement accuracy.

The blue trace (bottom) in Figure 3 shows the corrected magnitude frequency response of the 54855A using magnitude-flattening filtering. With this DSP/software filter, the corrected frequency response of the oscilloscope typically does not deviate more than +/- 0.5dB until the response naturally rolls off near the specified 6 GHz bandwidth. This particular characteristic of the scope's FIR filter is not user-selectable - it always runs when you are sampling at the scope's maximum sample rate to correct for always-present hardware filtering errors. The combination of the software and hardware filter produces more accurate data than data produced by the hardware filter alone.

Figure 4:  In-phase harmonics

Figure 5:  5th harmonic delayed

The red trace in Figure 6 shows the typical frequency-dependent phase error induced by the 54855A hardware at higher input frequencies. The blue trace in this graph shows the corrected phase response using phase-correction DSP/software filtering. As you can see, this software filter corrects for all phase errors to well beyond the bandwidth specification of the instrument.

Figure 6:  Corrected and uncorrected phase response

Figure 7 shows a simulation of a digitized fast-edge signal, with and without phase-correction, for a 6-GHz hardware system based on a high-order maximally flat response. The one artifact that you will note in the phase-corrected waveform (left/red trace) is the presence of minimal pre-shoot and over-shoot on the waveform. Again, neither the pre-shoot nor the over-shoot are actually present in the simulated input signal, which had an infinitely fast rise time, but are artifacts of the linear-phase system response and signal content beyond the -3dB frequency. But don't overlook the excessive over-shoot on the non-phase corrected waveform (right/blue trace). With phase-correction, the overall perturbation errors on both the top and base of this waveform have been improved. Most importantly though, timing measurements such as the rise times and fall times are much more accurate when phase correction is applied on either in-band or out-of-band signals. Again, phase-correction filtering is not user-selectable in Agilent's 54855A oscilloscope; it is always running to correct for added hardware phase errors.

Figure 7:  Pulse response with and without phase correction

Figure 8 shows an example of capturing a 1-GHz sine wave using the 6-GHz bandwidth 54855A oscilloscope without the use of noise-reduction filtering. Using the infinite-persistence display mode, after 1000 accumulated acquisitions we see a band of noise on this captured sine wave that is induced by the oscilloscope's hardware noise floor. At this setting, the instrument-induced noise measured approximately 2.8 mV RMS. The upper/yellow trace shows the input signal scaled to near full-scale at 100mV/div. The lower/green trace shows a 10X waveform expansion of this waveform near its peak.

Figure 9 shows the same 1-GHz sine wave, but now captured using 2-GHz bandwidth noise-reduction filtering. After 1000 accumulated acquisitions, we see a much cleaner waveform due to a nearly 2:1 reduction in the system's noise floor. Again, the upper/yellow trace shows the input signal scaled at 100 mV/div and the lower/green trace shows an expansion of the waveform near its peak allowing us to see more clearly the effects of the scope's reduced noise floor using noise-reduction filtering.

When testing lower-bandwidth signals, or signals with relatively slow edge speeds, engaging noise-reduction filtering will often enhance accuracy of both amplitude and timing measurements. One example involves measuring jitter. One of the largest, but often overlooked, components of error in jitter measurements is jitter/timing-error contributed by vertical noise. There is a direct relationship between vertical noise and timing error as a function of the slew rate of the signal. Although it may be counter-intuitive, reducing the bandwidth of your measurement system may actually improve the accuracy of jitter measurements when you are measuring in-band signals. Turning on noise-reduction filtering will automatically reduce instrument-induced jitter due to an excessive instrument noise floor. Because of the obvious trade-offs (bandwidth versus noise), use of noise-reduction filtering is user-selectable in Agilent's 54855A oscilloscope.

Figure 8:  Default 6-GHz bandwidth mode with 2.8 mV RMS scope noise floor

Figure 9:  2-GHz noise-reduction mode with 1.6 mV RMS scope noise floor

Figure 10:  Bandwidth-enhancement filtering

Again, there is one big tradeoff. As we mentioned, an oscilloscope is a broadband instrument, and the noise floor of the instrument can significantly degrade measurement results. With bandwidth-enhancement filtering, the noise floor of the instrument also is selectively amplified. So, there is a signal-to-noise tradeoff when you use these bandwidth-enhancing characteristics of the scope's FIR DSP filter.

Although bandwidth enhancement filtering may be a fairly new capability in some of today's higher-bandwidth real-time oscilloscopes, this is not a new technique in the test-and-measurement industry. Agilent has been using bandwidth-enhancement techniques for years in network analyzers and spectrum analyzers. In fact, Agilent first used this technique in an oscilloscope to simulate faster edge speeds when performing TDR measurements with a 20-GHz sampling oscilloscope. This technique is known as "normalization" in today's sampling oscilloscopes with TDR measurement capability.

Figure 11 shows an example of measuring an out-of-band signal using a 6-GHz oscilloscope. The input signal has an approximate risetime of 50 ps (based on a 10% to 90% measurement criteria). But since the basic hardware of the oscilloscope has a risetime specification of 70 ps, we measure a risetime of just 74 ps. With the use of 7-GHz bandwidth-enhancement filtering, we can now make a more accurate measurement on this signal of approximately 66 ps, as shown in Figure 12. However, you can see that the baseline noise on both the top and base of this waveform has increased. In the standard 6-GHz bandwidth mode, the noise floor of the scope measures approximately 3 mV RMS at the 100-mV/div setting. The noise floor increases to approximately 6 mV RMS when using 7-GHz bandwidth-enhancement filtering.

Figure 11:  Risetime measurement without bandwidth-enhancement

Figure 12:  Risetime measurement with 7-GHz bandwidth-enhancement

Another benefit of DSP filtering with bandwidth-enhancement on Agilent's 54855A oscilloscopes is high-impedance active probing measurements can be performed up to 7-GHz system bandwidth. No longer is the differential active probe the weakest link in the oscilloscope measurement chain.

Some of the filter characteristics discussed in this application note have minimal or no disadvantages, such as magnitude-flattening and phase-correction filtering. For these reasons, these particular filter characteristics are not user-selectable but run as a default operation when the Agilent 54855A oscilloscope is sampling at the maximum specified sample rate (20GSa/s). Because we believe that sin(x)/x waveform-reconstruction filtering also improves measurement accuracy and display quality, the characteristics of this particular filter runs as a default operating mode of the oscilloscope, but you can easily disabled it. The tradeoffs using sin(x)/x filtering are primarily associated with throughput, not accuracy.

Other characteristics of the scope's FIR DSP filter, including noise-reduction and bandwidth-enhancement filtering have very definite tradeoffs in terms of bandwidth and noise. For this reason, neither of these filter characteristics run as a default mode of operation of the oscilloscope; you must turn them on to use them.

As long as you are aware of the tradeoffs inherent in some filtering types, it makes sense to use DSP filtering to improve the accuracy and resolution of today's real-time oscilloscopes.

Johnnie Hancock graduated from the University of South Florida with a degree in electrical engineering and began his career with Hewlett-Packard in 1979 as an embedded hardware designer, and holds a patent for digital oscilloscope amplifier calibration. Today he is a signal integrity applications engineer with Agilent Technologies' Electronic Products and Solutions Group. He is currently responsible for worldwide application support activities that promote Agilent's high-performance digitizing oscilloscopes. In his spare time, he enjoys cross-country running and restoring his 110 year old Victorian home.