Another story from my SITA days. Have a look at my last story for a potted account of SITA (the airlines data network) and how it works.
I was based in Harare, Zimbabwe, as Regional Network Manager for SITA in Zimbabwe and eight surrounding countries, but one day my technical boss in Head Office in Paris asked me if I’d go to the Comores Islands -- way out of my area -- to look at a thorny problem with an Air France printer. His reasons were twofold: firstly, I spoke passable French; secondly I had once boasted to him that there was no comms problem I could not solve. Ahh, the arrogance of youth!
The Comores are between the top end of Madagascar and mainland Africa. Three islands form the Republic of Comores, while the fourth, Mayotte (see map, nearby), is a dependency of France. Getting there from Harare required a night stopover in Nairobi, and our representative there, somewhat of a bon-viveur, treated me to dinner and plied me with Calvados, a very fine apple brandy of which I was inordinately fond. So it was with somewhat of a heavy head (and the gift of another bottle of Calvados!) that I boarded the flight to Moroni, capital of the Comores Republic, the next morning.
Learn something new every day
Our problem was in Mayotte, the French island. So I had to catch another flight on a small plane which bounced around on the tropical winds. The cabin attendant handed out entry forms and I dutifully filled mine in. As soon as I finished, however, I was besieged by most of the other passengers who handed me their forms and passports and asked me by hand gestures if I would fill their forms in. I quickly came to the realization that they could not read or write!
I was met in Mayotte by our Madagascar rep who was responsible for this area. Our hotel and the office with the problem were on the main island, and our comms centre and the PTT were on another small island, so my next new experience was having to take a ferry to work in the morning, along with a motley assortment of locals, dogs, goats and chickens.
The problem at the Air France office was that sometimes when they ordered a printout, the printer would not work. Mostly it did.
My boss had sent from Paris a fancy line analyzer that could pick up anything that would upset a modem signal, and also a protocol analyzer. He was sure the problem was a noisy line, so my first line of attack was to thoroughly check the international and local lines for any noise or distortion that could cause data loss.
Our comms centre was in the PTT building so we went there and took the lines for testing at a time that would not disrupt our users too much. Now I said I spoke reasonable French, but my knowledge didn’t include too many specialist technical terms. However the local PTT techs were very helpful and soon I was bandying around terms like Gigue de Phase (Phase Jitter) and Sauts d’Impulsion (Impulse Hits) like a seasoned pro. Not that I found much of either; my testing confirmed that both our international and local circuits were just about perfect. So scratch that theory.
I then set up the protocol analyzer and looked at the data on both links. Again, all seemed perfect. There did not seem to be any data corruption. So the next step was to go to the Air France office and see what I could find there. I had two days before my flight back, and I didn’t think my boss would be impressed if I did not live up to my boast. (Nor would I !!) But I was doing a lot of head scratching on this one.
So I set up my analyzer at the Air France office and got the staff there to do some prints. I monitored all day and managed to capture a couple of instances where the problem occurred; but when it did, there just did not seem to be a block of data addressed to the printer. I began to think that the problem was with Air France. Frustrated, I retired for the day, but took the protocol analyzer with me so I could go over some of my captures. This was no easy task; the analyzer was based on the old Compaq “luggable” with built in 7-inch CRT screen and a 5-1/4 inch floppy disk drive.
A little acknowledgement
That night, going over the data, I noticed something. When the problem occurred, we were sending Air France a block containing two messages, one an acknowledge from the host for the terminal input, and the other the data for the printer. The block was not corrupted but the printer data did not seem to get through. When the printer data came in a bit later (most of the time) it was sent in a separate block, and printed fine.
The next morning I verified my theory. Sure enough, every time the problem came up, the terminal input acknowledge and the printer data were sent in one block. I had seen this before and didn’t think it was a problem, but obviously the Air France terminal equipment (which I wasn’t familiar with) didn’t like it. So I messaged my boss, and with some trepidation lest I had been chasing a red herring, returned to Harare (with no overnights and Calvados, unfortunately!).
The next week I got in touch with my boss. “Well done David” he said. “Well done what?” I asked. “Was it the Air France equipment?”
“No”, he replied, “there’s a setting on our concentrator that forces it to send only one message at a time, and that’s done it!” I didn’t know about that setting and never had to use it again, but with my protocol knowledge and his knowledge of the network equipment we solved the problem. That’s what I call teamwork!