I once was working at a startup company where we were building a smart antennae SoC for WiFi. This was around 2002-2003 and 802.11n was not present yet. Our aim was to build the MAC and physical layer.
In order to test our system before implementing the ASIC, we had purchased two identical FPGA prototyping boards; one for the transmitter and one for the receiver.
These boards each hold two Xilinx Virtex XCV2000 (2M gates each). The design itself was about 2M gates split over the two FPGAs. I was responsible for the lab support and for the MAC hardware (and later the MAC firmware) so I set up our hardware for the tests.
The boards were used for the verification of the physical layer only. For debugging we had an oscilloscope and a Logic Analyzer, and I had also proposed using the Xilinx ChipScope technology (a JTAG-based virtual logic analyzer that can be embedded in the FPGAs). The test procedure was to run the design with test vectors and see if the response was compliant with the simulation. The logic analyzer we had was not an option; we wanted to capture many internal signals without any physical access to them. In addition, due to low finance our logic analyzer was a low-end device and its probes were interfering with the low-level high-speed signals we had on board.
The FPGA engineer downloaded his design and began the tests. After the initial debugging and setup, we were ready to transmit packets and test the functionality of the VHDL code. When operating the boards, the power consumption of each board was around 4 to 5A at 5V having the operating clock at 80 MHz. During testing, the test vectors did not match with the simulation and the FPGA engineer was pretty busy identifying the problems. Step-by-step, a few timing or other VHDL problems were resolved until the FPGA engineer could go no further. He could not find any other errors in synthesis or in the routing of the design. He asked for help.
We were reluctant to believe there was a hardware problem. I mean a real hardware problem (i.e. Not VHDL). I started probing the board with the oscilloscope. The FPGA engineer insisted on watching the power supplies. Initially I found out that sometimes the power supply had problems. The board had three or so jumpers to pass power input to the internal regulators. And once or twice the jumpers did not have good contact. I used nippers and solder to make them tight.
The FPGA boards with the RF section connected (later stage)
The problems did not disappear though. As the oscilloscope available had a low bandwidth of 100 MHz and the ground clip was too noisy to be able to see cleanly the signals, I made a small setup to have proper grounding on the probe using a very short lead (actually almost without a lead). We could see the 1.8V of the FPGA power to have a pattern during transmission that corresponded to the internal operation of the logic. See below an illustration that is similar to what we had (this is a simulation not the real data of the oscilloscope – the X-axis is the time in ms; the Y-axis is volts).
The power rail had periodic dips. According to the FPGA datasheet the voltage should be above 1.75V (I do not remember the numbers exactly). In some instances the voltage was at about 1.70V or so. The FPGA engineer was convinced that that was the problem. I promised to fix the problem, although I was still in disbelief that this was the resolution of the intermittent problems.
Checking the schematics of the boards (fortunately we had them) and inspecting the boards themselves, I could not believe that these beasts had very poor decoupling of only 10nF per power pin! In the old days, the rule of thumb was to put 100nF on each IC plus an electrolytic or tantalum capacitor every few chips. However, experienced high-speed designers know that this will not work when speeds are climbing. There are two types of capacitors you need to place on a digital IC. Usually there is a filter capacitor, which is related with operating frequency used for bypassing. Then you can (or should) place more capacitance to compensate for the power hungry switching circuits requiring huge inrush currents instantaneously. Otherwise you will get voltage drops on the power planes (yes they exist!) and you need only a few millivolts to fall below the minimum limits.
That was our case. I could measure the switching voltage drop. Of course things were made more difficult because our 100 MHz oscilloscope was only marginally able to capture clean signals due to bandwidth and grounding. Not to mention the FM radio frequency (87 to 108 MHz) interfering with our measurements. The radio antennas were in a nearby mountain, providing a good SNR for radio….and bad SNR for us. This was a true nightmare.
If the current consumption was 4 to 5A in total for each board, which means around 2A mean current per FPGA, how much switching current did we have? And how much energy could the 10nF capacitors hold?
It was natural to see the small dips. I wasn’t sure how much capacitance to add, so I decided to add a set of additional capacitors to each existing 10nF capacitor: tantalums of 4.7uF and 1uF, along with two ceramic 100nF. The next problem was how to attach these new capacitors as the current capacitors were presented in tiny 0402 packages. The tantalum capacitors were leaded parts (easy to find them in the local market) while the 100nF were 1206 parts. I ended up having to make a small construction where the end of the leads matched the 0402 size package.
Decoupling capacitors construction on the FPGAs.
I had to place many such constructions around each FPGA. These were very sensitive to mechanical force; the small PCB pads were destroyed if you applied even minor pressure to the capacitors. To my surprise, after this “surgery” the design worked!
Yes. The small mV range dip was enough to disturb the logic timing and make the design failing in intermittent ways.
This exercise provided me with a much better understanding of power planes and what is important in each case. I also discovered that even small details of non-compliance can cause a big difference with regard to success or failure. Needless to say, we never told the designer of the development board our findings…About the author
Fascinated by computers seen in advertisements in the start of 80’s Ilias Alexopoulos knew just what he wanted to do. He started playing with early computers for fun and both the hardware and software was his everyday joy.
Later, at the university, Ilias entered the world of embedded computing, DSP, and FPGAs. His passion for high-end systems has lead him to solve many challenges from production testing equipment to high-tech startups.
Commitment to engineering excellence and enjoying the engineering journey is always his aim. Recently, Ilias has started to contribute to the open source community with articles, code, etc. You can find more information at www.ilialex.grClick Here
to see other articles in this "That Was Tricky..."
series...Editor's Note: It would be great if you took the time to write down short stories of your own. I can help in the copy editing department, so you don’t need to worry about being “word perfect”. All you have to do is to email your offering to me at max@CliveMaxfield.com with
“How it was” in the subject line.I can post your article as “anonymous” if you wish. On the other hand, what would be really cool would be if you wanted to add a few words about yourself – and maybe even provide a couple of
“Then and Now” pictures showing yourself as a young engineer
("Then") and as the hero you've grown into
If you found this article to be of interest, visit EDA Designline
where – in addition to blogs on all sorts of "stuff" – you will find the latest and greatest design, technology, product, and news articles with regard to all aspects of Electronic Design Automation (EDA).
Also, you can obtain a highlights update delivered directly to your inbox by signing up for the EDA Designline weekly newsletter – just Click Here
to request this newsletter using the Manage Newsletters tab (if you aren't already a member you'll be asked to register, but it's free and painless so don't let that stop you [grin]).