First, before making any measurements, determine any requirements for confidence and accuracy. Determine the testing limit for bit error rate (BER): this is the point at which the BER need not be measured any lower for any Eb/No (the ratio of energy per bit to noise density) value. Apply these to the Eb/No test points to determine the test time estimates.

Next, make adjustments to the confidence, accuracy and BER limit as needed to provide acceptable results and to fit into the testing schedule. Use these parameters to calculate the criteria to stop the measurements. This will be the number of bit errors and the error-free bits required for each Eb/No test point.

Once the testing is complete, the confidence and accuracy have to be applied to the measurements to allow reliable analysis of the waterfall curves.

Now, let's take a look at the details.

Evaluating almost any communications systems will require some form of BER testing, whether it is done for engineering qualification or for production testing. With wireless systems, the testing is not focused on how long between bit errors, but at what point the background noise will cause too many bit errors. For that reason, wireless systems typically require an evaluation of BER in response to varying levels of simulated background noise, where the noise is expressed in terms of Eb/No , or bit signal-to-noise ratio.

Simulated background noise is produced by a noise source generating additive white Gaussian noise (AWGN). The AWGN will cause bit errors on the data stream that can mathematically be represented as a Gaussian probability distribution. The relationship between the AWGN and the bit error distribution allows for a much broader analysis of the BER results. Because of the known properties of the error probability, comparisons of test results among similar devices can be made reliably.

Applying AWGN to wireless devices will lead to test results that are normally represented as BER vs. Eb/No curves, commonly referred to as waterfall curves. Waterfall curves express the BER response to decreasing background noise. A waterfall curve plot is typically the end result of the testing and is a significant factor when wireless devices are being compared.

The goal here is to be able to effectively generate the test results so that there can be reliable comparisons using the resultant waterfall curve. By understanding the probability distribution, testing can be performed effectively. Tradeoffs can be made between the amount of test time that is acceptable and the amount that is required to attain the desired measurement confidence level and accuracy. When the testing is complete, the confidence level and accuracy of the measurements can then be applied to the waterfall curve so that these can be used for reliable, quantitative comparisons.

Gaussian probability distribution

Statistically, any measurement would have to be taken over infinite time to know its true BER. Since the measured BER is taken over a shorter, finite, period, the true BER could be significantly higher or lower. By adding stress with known properties, testing can be completed in less time and still be reproducible.

For wireless systems, this stress is typically in the form of AWGN. The occurrence of bit errors due to the AWGN can be represented through a Gaussian probability distribution. Using the properties of this distribution, the individual measurements can easily be characterized with confidence levels and accuracy.

Confidence level specifies how often the accuracy, when applied to the measured BER, will correctly contain the true BER. Accuracy is the difference between the measured BER and the true BER. Both are typically expressed as a percentage.

Let's take a simple example. Assume that a measurement was performed where 100 bit errors were recorded. If the confidence level were one standard deviation (or 68.27 percent), the accuracy of the measurement would be 10 percent. This means that it is 68.27 percent likely that the true BER would be within 10 percent of the measured BER. If this assumed test were done 10,000 times, the true BER would be within 10 percent of the measured BER 6,827 times; the remaining 3,173 times it would be off by at least 10 percent.

Waterfall curves express the bit-error-rate response to decreasing background noise and are a significant factor when comparing wireless devices.

Knowing the Gaussian probability distribution provides a several formulas with which to apply confidence level and accuracy to the BER measurements. The basic one relates confidence level, accuracy and the number of bit errors. The calculation can be expressed to solve for any of those three factors using the remaining two factors. Most often it will be used to solve for accuracy, but it will also be useful to solve for the number of bit errors required to meet a specified accuracy and confidence level. Notice that these calculations are independent of test time and bit rate. The accuracy is based purely on the measured bit errors.

Waterfall curves

Typically, the test results will end up as waterfall curves, displaying the BER response to decreasing background noise. There are also waterfall curves that are the theoretical limits of the lowest attainable BER values for a given modulation scheme. These theoretical waterfall curves will provide a starting point for estimating test times.

The theoretical values are the theoretically lowest attainable BER. The BER that is actually measured is going to be at or above this level. Using real signals, the measured BER may be close to theoretical or significantly higher. After some initial testing, this can be measured and, if necessary, a new curve can be made representing the expected BER.

For each point on the curve that is to be tested, by using the theoretical or expected BER values as the actual incoming BER an accurate time estimate could be made for a measurement of that signal. The estimates can then be examined to determine where most of the test time would be spent. Mostly likely the confidence will be set to a fixed value, but by adjusting the target accuracy and confident upper bound BER, the time estimates can be controlled so that the test time is being spent to the best benefit. Inaccurate estimates of BER will lead to overestimating test times.

When the test is complete, the measurements can then be analyzed. The calculated accuracy can be applied to each BER and the upper BER limit can be found for each error-free measurement. These can then be applied to the test result's waterfall curve, allowing for more reliable and quantitative comparisons.

Let's assume that a measurement is to be performed using BPSK modulation with no error correction and is operating at 60 Mbits/second. The Eb/No test points will be at 8, 10, 12 and 13 dB. The selected confidence level is 99.9 percent. The target accuracy is 5 percent and the maximum test time allowed is 20 minutes.

As the background noise level (Eb/No) goes up, the test times go out. The problem areas are 12 dB at over two hours and 13dB, which pushes test times out to 6.5 days!

But consider the effects of the BER estimate: If the system requirements do not need any BER measurement better than 5.0e-09, ((IS THAT OK?))the test time is sliced from days to hours.

This improves the test time considerably, but still too much time is being spent at 12 dB and 13 dB. To improve the test times enough to satisfy all testing concerns — to, say, 12 minutes — the accuracy at those points could be widened to 25 percent. This provides an acceptable total test time that is under 20 minutes, with acceptable confidence and accuracy for results.

References

1) Statistical Treatment of Experimental Data: An Introduction to Statistical Methods, by Hugh D. Young; McGraw-Hill Book Co., 1962; reissued by Waveland Press Inc., Prospect Heights, Ill., 1996; ISBN 0-88133-913-X.

2) Modern Elementary Statistics, Fourth Edition, by John E. Fruend; Prentice-Hall Inc., Englewood Cliffs, N.J., 1973; ISBN 0-13-593475-3.