Design Article
Record and replay in global navigation satellite system testing
Julian Thomas, managing director, Racelogic (Buckingham, UK)
2/24/2012 8:55 AM EST
Customers at vehicle manufacturers and proving grounds need tests such as brake distance measurement to be accurate to 2cm. Before 2008, there were two options: live sky testing and simulation. Racelogic has always used live testing, which involves taking equipment out in vehicles on the road and around test tracks. This is useful for getting real world data, but the changing nature of satellite constellations means that no test can ever be repeated. The company’s development team also found constant testing in the field to be time and labor intensive.
Therefore, for some tests we used an expensive artificial GPS simulator, which modeled the satellite signals modified with a user generated trajectory. However, the test engineers found that this didn’t sufficiently reproduce the real world environment. The results were not always a good indication of how their GPS products would actually perform in the real world. Neither live sky field testing nor GPS simulation was adequate. The dynamic nature of satellite constellations meant field tests were time intensive and not repeatable, whilst modeling satellite signals was expensive and couldn’t reproduce actual conditions.
Developing record and replay technology
So, we began to research a system that would have the ability to record live GPS signals and then replay these into products under test. The brief was that it would offer realistic data, in the same way as live sky, but that it would also be consistent, with the ability to repeat a test, or ‘scenario’, as many times as necessary. It would also need to be affordable.
With nothing else available, Racelogic decided to design something that could record live global navigation satellite system (GNSS) signals and replay these into devices under test, offering the blend of realism and repeatability that they needed at an affordable price. In 2008, we developed LabSat – see figure 1 - which had the ability to record raw GPS RF signals from a live antenna to be replayed at a later date. Users could now easily record a journey and replay this on the bench with identical results. With this module, Racelogic found it could reproduce the data from a real brake test activated using a switch on the brake pedal, to within 2cm of the physically measured distance (verified by a laser) all from the test bench. Soon five units were in constant use, and from the feedback of our R&D engineers it was clear that LabSat could be helpful to other companies developing GNSS products.
Software simulation
Some testing scenarios are inherently difficult to record, such as a complex route in distant country, a test crossing meridian lines or a time based test involving a GPS year rollover. For these situations, Racelogic developed the SatGen scenario generation software. SatGen allows the user to define a custom route, anywhere in the world, at a predefined time and date. There is even a Google Earth import feature which makes the definition of the profile very simple. This profile information is then converted into a scenario file which can be replayed on LabSat devices. The additional advantage of an artificially created test is the quality of the simulated data; for example, if you need to precisely control the speed, heading, position or height, then this is very simple to do and the output from LabSat will be extremely precise.
LabSat is now used in several industries. For example, Nokia and Blackberry use LabSat for testing smartphones; Mercedes Benz, Bosch and Continental use it to test in-car navigation systems; BAE systems use LabSat to test the latest defense technology; and Road Angel use it to verify that their GPS safety camera and black spot locators are operating correctly.
Figure 2: SatGen artificial scenario creation software
Six thousand hours of testing
For Road Angel, it’s vital that the products work 100 percent of the time, or the company would risk complaints from customers who claim that their device failed to warn them of speed cameras. Previously, Road Angel’s technicians had to individually test units by driving past known speed camera locations and identifying the units which failed to activate, indicating a malfunction in the GPS system. Because this method required a vehicle and driver, it was time consuming, expensive, and only provided subjective reporting of errors. Having heard about record and replay technology, Richard Meechan, Road Angel’s Development Director, went out with a LabSat and recorded a live route incorporating several speed camera types and locations, for later replay into units under test.
“We had LabSat running 24/7 in our lab for over two weeks, replaying a thirty minute test drive into twenty test units at a time.” Meechan continued: “Each block of units had a different version of firmware installed. LabSat was used to evaluate how well each version detected the various speed camera systems recorded, including new types of cameras introduced since the original firmware was released. In two weeks the LabSat record and replay system saved us over six thousand hours of testing of our new Vantage GPRS connected speed camera locator.”
Road Angel is also a user of Racelogic’s simulation software, SatGen. By creating scenarios in the SatGen software to create realistic data in which to playback through units under test, engineers can test the operation of their devices anywhere in the world. They have found it useful in instances where customers claim that their product failed to indicate a camera, or where the device indicated a camera where there was none. Testing engineers can use SatGen in conjunction with Google Earth to recreate a test drive past the area where the device was claimed to malfunction, to see what went wrong and repair it.
Next: Extended simulation
Therefore, for some tests we used an expensive artificial GPS simulator, which modeled the satellite signals modified with a user generated trajectory. However, the test engineers found that this didn’t sufficiently reproduce the real world environment. The results were not always a good indication of how their GPS products would actually perform in the real world. Neither live sky field testing nor GPS simulation was adequate. The dynamic nature of satellite constellations meant field tests were time intensive and not repeatable, whilst modeling satellite signals was expensive and couldn’t reproduce actual conditions.
Developing record and replay technology
So, we began to research a system that would have the ability to record live GPS signals and then replay these into products under test. The brief was that it would offer realistic data, in the same way as live sky, but that it would also be consistent, with the ability to repeat a test, or ‘scenario’, as many times as necessary. It would also need to be affordable.
With nothing else available, Racelogic decided to design something that could record live global navigation satellite system (GNSS) signals and replay these into devices under test, offering the blend of realism and repeatability that they needed at an affordable price. In 2008, we developed LabSat – see figure 1 - which had the ability to record raw GPS RF signals from a live antenna to be replayed at a later date. Users could now easily record a journey and replay this on the bench with identical results. With this module, Racelogic found it could reproduce the data from a real brake test activated using a switch on the brake pedal, to within 2cm of the physically measured distance (verified by a laser) all from the test bench. Soon five units were in constant use, and from the feedback of our R&D engineers it was clear that LabSat could be helpful to other companies developing GNSS products.
Figure 1: The LabSat unit
Software simulation
Some testing scenarios are inherently difficult to record, such as a complex route in distant country, a test crossing meridian lines or a time based test involving a GPS year rollover. For these situations, Racelogic developed the SatGen scenario generation software. SatGen allows the user to define a custom route, anywhere in the world, at a predefined time and date. There is even a Google Earth import feature which makes the definition of the profile very simple. This profile information is then converted into a scenario file which can be replayed on LabSat devices. The additional advantage of an artificially created test is the quality of the simulated data; for example, if you need to precisely control the speed, heading, position or height, then this is very simple to do and the output from LabSat will be extremely precise.
LabSat is now used in several industries. For example, Nokia and Blackberry use LabSat for testing smartphones; Mercedes Benz, Bosch and Continental use it to test in-car navigation systems; BAE systems use LabSat to test the latest defense technology; and Road Angel use it to verify that their GPS safety camera and black spot locators are operating correctly.
Figure 2: SatGen artificial scenario creation software
Six thousand hours of testing
For Road Angel, it’s vital that the products work 100 percent of the time, or the company would risk complaints from customers who claim that their device failed to warn them of speed cameras. Previously, Road Angel’s technicians had to individually test units by driving past known speed camera locations and identifying the units which failed to activate, indicating a malfunction in the GPS system. Because this method required a vehicle and driver, it was time consuming, expensive, and only provided subjective reporting of errors. Having heard about record and replay technology, Richard Meechan, Road Angel’s Development Director, went out with a LabSat and recorded a live route incorporating several speed camera types and locations, for later replay into units under test.
“We had LabSat running 24/7 in our lab for over two weeks, replaying a thirty minute test drive into twenty test units at a time.” Meechan continued: “Each block of units had a different version of firmware installed. LabSat was used to evaluate how well each version detected the various speed camera systems recorded, including new types of cameras introduced since the original firmware was released. In two weeks the LabSat record and replay system saved us over six thousand hours of testing of our new Vantage GPRS connected speed camera locator.”
Road Angel is also a user of Racelogic’s simulation software, SatGen. By creating scenarios in the SatGen software to create realistic data in which to playback through units under test, engineers can test the operation of their devices anywhere in the world. They have found it useful in instances where customers claim that their product failed to indicate a camera, or where the device indicated a camera where there was none. Testing engineers can use SatGen in conjunction with Google Earth to recreate a test drive past the area where the device was claimed to malfunction, to see what went wrong and repair it.
Next: Extended simulation
|
|
|
|
|
Navigate to related information