The Six Axes of Calibration

Selecting a supplier of calibration services may seem fairly straightforward—it would seem that a calibration is a calibration and, as long as the supplier has the usual badges of office (ISO9000 registration, ISO17025 accreditation, and so on), the service ought to be reasonably similar whichever service or supplier you choose. However, this is not the case and the lack of regulation on calibration allows for a wide variance in the deliverables and value of calibration.

To help demonstrate these variances and help lead the non-technical buyer to appreciate the importance of each, I have split them into six different axes, each of which can have an impact on the usefulness of the calibration or the direct or indirect cost to the organization buying the service. This article discusses each of the following axes:

1. Extent of Testing—How many parameters? How many test points per parameter?
   
   
2. Information Provided—What data is provided? Pre and post adjustment data?
   
   
3. Measurement Uncertainty—How good is the testing?
   
   
4. Periodicity—How often does it need calibrating? Can this vary?
   
   
5. Speed—How long are you without the instrument?
   
   
6. Packages—What financial and service packages are there? What is included in the service?

1. The supplier does not have the equipment to perform the test
   
   
2. The tests are complex and take a long time to perform, or require special skills
   
   
3. The price agreed does not allow for all of the tests to be performed.

Often this means that important parts of the instrument are inadequately tested and the equipment may unwittingly be making erroneous measurements as a result.

This can result in one of two detrimental situations arising when you test your products with the equipment:

1. Good products can appear to fail the test, resulting in expensive rework and increase in NFF (no fault found) occurrences in the rework department (a key measure in manufacturing circles)
   
   
2. Bad products can appear to pass the test, resulting in increased warranty returns and repair costs, along with a damaged reputation.

Example
A customer sent a Scalar Analyzer Detector head to a reputable company for calibration. The company's procedure was to test the device at three spot frequencies at the low, mid, and high end of its range. The device was reported to be performing to specification and a calibration certificate was issued. The customer experienced subsequent strange measurements using this particular device and sent it to Agilent for a second opinion.

Agilent's procedure is to use a swept frequency across the device's entire range. This testing clearly showed that the head had a problem at a frequency not tested by the first company. If it had been calibrated using the swept method originally, the customer would have saved a great deal of time and money.

An analogy might be a sailing ship's look-out up in the crow's net, looking through his telescope at exactly due East, then due North, and then due West, and reporting, "I see no ships."

Exception
It is important to stress that less testing is not necessarily a bad thing when properly controlled. For equipment used in a specific application in which only well defined aspects of the instrument's capability are used, it is certainly acceptable and may well be advantageous to focus the calibration on those particular functions or ranges. This might be especially true if more extensive testing is made in those areas than would have been performed in the normal, full calibration.

However, this approach would be unsuitable for equipment used in multiple applications, or in a general equipment pool.

Consequently, it is desirable for the full data showing the equipment's performance at each test point to be supplied with the calibration. This information is evidence of the extent of testing and can be used to make objective comparisons between suppliers.

But the information is valuable in more important ways. When an instrument is found by the calibration supplier to be outside of its specified tolerance (OOT—out of tolerance) it means that you have possibly been making bad measurements with that equipment for some time. To assess whether this will have compromised the validity of your testing you will need to know where the instrument was OOT and by how much. You are then in a position to decide whether you need to take corrective action such as re-testing or recalling your products. Without this information you would either be unaware that the testing could have been erroneous, or if the calibration simply reported that the equipment was OOT but provided no supporting data, you would not be able to make an informed decision.

The really useful data is a full set of performance test information taken exactly as the equipment performed when received by the calibration supplier. Any attempts to adjust the equipment during this testing will destroy the integrity of tests later in the procedure. A second set of data taken after adjustments confirms that the instrument is now within tolerance, giving confidence in the future testing made with the equipment.

Another key piece of information is measurement uncertainty.

In its simplest form this is a statement that the accuracy of equipment used to test your instrument is at least a factor of four better than the equipment being calibrated. (Four is usually considered the minimum test accuracy ratio, TAR, useful for calibration).

More detailed information comes in the form of a summary table listing the parameters and the associated uncertainties. The figures can either be expressed in relative (such as ±0.5%) or absolute terms in the units of measurement (such as ±0.2 mV).

The most detailed uncertainty information is provided with the measurement data itself, appearing alongside each reported value and, ideally, with the measured value, specification and uncertainty all expressed in the same terms.

The uncertainty gives a clear indication as to whether the supplier has the technical capability to make the measurements. Uncertainties may be self-assessed or accredited. Accredited uncertainties are more valuable as they have been independently assessed by a recognized accrediting body such as Britain's UKAS, Japan's JCSS, Germany's DKD, the United States' A2LA, and so on.

It is possible to review the best measurement uncertainty figures for accredited calibration laboratories as this information is in the public domain, often made available via the organization's Web site. This gives a valuable insight into the technical competence of the laboratory and allows comparison of ability to make certain measurements. It is also possible to determine that a lab may not really have the credentials to undertake some calibrations.

On the subject of accreditation, it is not uncommon for a lab to gain such recognition for a few specific tests and then accept calibration work for equipment requiring competence outside of their scope of accreditation. Gaining accreditation for just one measurement allows them to display a Certificate of Accreditation and advertise that they are an accredited lab, although their capability would obviously be inadequate for most instruments.

Example
An assessment of the Accreditation Schedule or Scope of calibration suppliers in Britain for two fairly fundamental measurements showed interesting observations.

The selected measurements were 10 MHz frequency, and 1-V dc. The information was from the United Kingdom Accreditation Service (UKAS) Web site.

In the case of the 10 MHz, the spread of accredited uncertainties was large. The best being one part in 10¹², the worst being three parts in 10⁶. So when measuring 10 MHz, the best could boast an uncertainty of ±0.00001 Hz and the worst ±30 Hz. Since the specification for most crystal oscillators is far narrower than ±30 Hz, it is clear that a calibration performed by this company for a crystal oscillator or any instrument that measured frequency to any degree of accuracy would be meaningless.

In the case of 1-V dc, the spread was less large but still significant; the best being four parts in 10⁷, the worst being two parts in 10⁵. Many digital multimeters resolve to 6.5 digits. This means it should be able to display a measured 1-V dc as 1.000000 V (in other words, to 1 microvolt). An uncertainty of four parts in 10⁷ would measure accurately to 0.4 microvolts—sufficiently accurate for the task. An uncertainty of two parts in 10⁵ would only measure accurately to 20 microvolts. In relation to its accuracy the last digit of the multimeter would be entirely untrustworthy.

One thing to remember is that the published "best measurement capability" may not be the same as the actual uncertainty assigned to the calibration of a particular product. Comparing example certificates/reports for a few typical models may give a different perspective.

In order to assess whether the manufacturer's recommended periodicity can be overridden, two pieces of information are needed:

1. The calibration history for the particular instrument
   
   
2. The application in which the instrument is used.

The first point refers directly to the Information discussion (Axis 2). One generally accepted algorithm is that if on three consecutive calibrations the instrument has remained within specification, the period may reasonably be increased. Similarly, if it requires adjustment on two consecutive calibrations, it should be reduced. This sequence is ongoing.

The second point is about risk. If the measurements made by the instrument are critical, it may be prudent to reduce the calibration interval to reduce the risk of bad measurements. Alternatively the application may be such that if bad measurements are made it would not matter greatly, and so the interval could reasonably be extended.

Where both risk and history are considered, you can maintain an effective periodicity management solution such that the risk of bad critical measurements is minimized, and costs are also controlled.

Knowing that the equipment will be away for two weeks means that you can plan for that event, arranging for use of the spare or rental replacement accordingly. But if the promised two weeks becomes three of more, the costs escalate and the control is lost. It is therefore very desirable to ensure that the delivery turn-around promises made by the calibration supplier are short, but equally importantly, can be reliably met.

Companies that can provide the service on your site are able to reduce the turn-around times to just a few hours—in fact just the time it takes to do the calibration. This may add to the service price but the additional cost can be insignificant in comparison to the savings gained through the reduction in spare instrument purchases and rentals.

A word of warning here. An on-site delivery may seem like an ideal solution but it is still necessary to be sure that the technical aspects of the calibration meet your needs. It is common for companies offering calibration on-site to deliver a subset of testing, perhaps also with larger measurement uncertainties. This situation may not be apparent until they arrive at your premises with a limited range of test equipment and with reduced capability.

There are also other considerations in this area. Consider shipping costs and minor repairs. If these are included in your package the savings can be significant. Alternatively the extra costs can be a surprise.

Example
A company quoted $15 per calibration for AVO-Meters (general purpose analog multimeters). It was only after some time that the customer spotted that the price charged for his AVO-Meter calibration was always $40.

When this was queried, it transpired that with AVO-Meter calibration, it is appropriate to replace the batteries. Replacement of batteries constituted a repair with this supplier, with an additional cost of $25 for the repair.

This example may be a small amount, but this can add up to a large extra cost over 12 months.

As is usually the case in any purchase decision, you get what you pay for. Understanding what these things are and what they might mean to your company should make that decision more of a science and less of a black art.