SAN JOSE, Calif. Artificial-intelligence techniques derived from military applications have been incorporated into automated inspection systems from Photon Dynamics Inc. Tricks from neural networks and expert systems are used in a smart-vision engine called AIMS an automatic inspection and management system for finished printed-circuit boards, specifically printed-wiring assembly and high-density interconnect.
"What we did was to start from the ground level and build up a set of algorithms that gave us what we liked from each of the AI approaches, and got rid of what we didn't like," said Mark Deyong, vice president of research at Photon Dynamics, based here.
AIMS was developed under government contract at Intelligent Reasoning Systems Inc., a company founded by Deyong and acquired by Photon Dynamics this year. The military's original charter was to create a smart-vision technology that could spot enemy planes, tanks and the like. Intelligent Reasoning then commercialized the application for automated inspection, in the process snagging major customers such as Celestica, Flextronics, IBM, Lucent, Sanmina and others.
"We were late coming into this game there were several other companies attempting to do this, so we could see what their track record was and where they had fallen down," Deyong said. "For the most part they had the common downfall of relying on the user to program the system."
The machine-learning capabilities of neural networks were harnessed to solve the problem. Instead of user programming, AIMS enforces a simple training-by-example regime where the user "shows" the inspection system examples of good pc boards. Neural networks "fuzzify" among the examples they are shown, so that the inspection system can still make smart decisions even though it has not been specifically trained to recognize every possible defect a pc board can have.
Tracking the audit trail
"The good things about a neural net are training by example and graceful degradation the fact that it can still make smart decisions even though it hasn't been given every example out there," said Deyong. "But the problem with neural nets is that they don't tell you how close you are to failure." By contrast, "a traditional expert system can always justify its decisions by telling you exactly which steps it took to come to a conclusion."
Therefore, Deyong and associates incorporated the explicit rule-based approach of expert systems. But instead of requiring the user to write the rules, AIMS keeps track of each neural-like training step as it learns from each new example. From that audit trail, AIMS can recall each step in its decision process by showing from which images it learned each rule. The technique also enables a "confidence level" to be associated with each of the system's decisions, thereby combining the advantages of an explicit rule base with the fuzzy-generalization ability of neural networks.
"In expert systems you can easily add things incrementally you can add rules and you can see all the other rules you already have, with explicit definitions of everything in the knowledge base," Deyong said. "But they are not capable of forming their own rules, or of fuzzifying those rules."
Deyong said the traditional image-processing approach provides the tools, but requires that the user stitch them together into an application that solves a specific inspection problem. When a new board is to be inspected, he said, the vision system must be manually reprogrammed. Maintenance procedures, just normal wear and tear or component-to-component variations can break a user-programmed inspection algorithm, he said, requiring the operator to manually build new models of matching "templates."
But line operators "don't know the image-processing algorithms or the details of the image-processing equipment," said Deyong. "So if you don't bring in a full-time, high-level person to operate the system, then the inspection system is not going to be up and running most of the time."
AIMS should eliminate the need for highly skilled assembly line operators, by programming the system by example and providing the means for the user to interrogate the automatically generated knowledge base. In this way, knowledge bases of arbitrary complexity can be automatically generated from real examples as the line runs, adaptively increasing in accuracy and reliability during routine use.
"A model that was sophisticated enough to be tolerant of variations that are acceptable, and yet catch all the underlying defects, was too sophisticated for a person to comprehend," said Deyong. "If a user has to program the inspection system they have to rely on something simple enough to understand."
The Photon designers made "the assumption that the patterns we are looking for are too complex for us to see," he said, "and that even if they aren't, they are still very tedious for us to generate and they change over time." Instead, AIMS relies on incremental training.
It is shown good examples to start, and during operation shown "false alarms" examples of boards that were flagged by the system as defective, but which human inspectors passed and "escapes," or boards that were passed by the automatic inspection system but were subsequently found to be defective by in-circuit testers. From these three types of examples, the system adaptively increases in accuracy and reliability during normal operations, according to Deyong.
"During the basic training mode you show the system as many good examples as the operator becomes comfortable with. If you don't show it enough examples it will be overly sensitive to defects, but it won't miss any," he said. "We have 100 percent defect coverage right out of the gate."
Traditional systems, Deyong said, have to be programmed to catch each kind of possible defect. Otherwise you can have escapes, which the operator fixes by tweaking the parameters to catch them. Escapes are said to be a rare occurrence with AIMS, however, because of the system's inherent generalization capabilities. Usually, Deyong said, an AIMS system starts out catching all defects plus a few false alarms.
"Typically we have to train false-alarm levels down, rather than do incremental training with negative examples escapes," he said. "By showing it more examples during basic training or by incrementally training during the subsequent operation, you can desensitize it to normal process variations." As you show AIMS more examples, "false alarms reduce over time," said Deyong.
When errors do creep into the knowledge base, standard procedures during incremental training will show the operator exactly which images contributed. If during basic training, for example, one of the boards had a missing part, the knowledge base will contain examples both with the part present and without it. During operation, the system will generate a false alarm by showing a 50 percent confidence level reflecting the two contradictory training examples. But when the operator asks "why," the system will immediately display the two images one with and one without the part allowing the error to be identified.
"All you do is click on a part, and AIMS shows you all the images it used for that pattern-recognition class, labeled by the order and time at which they were entered into the knowledge base," Deyong said. "The operator just grabs the image with the missing part and reclassifies it as an example of 'missing' instead 'present.' Now when you run the system on good boards it will give a high confidence that the part is present when it's present, and a high confidence it's missing when it's missing."
The adaptivity of these automatically generated models also lends portability to the knowledge bases AIMS builds, Deyong said. Depending on a factory's uniformity, operators can train a knowledge base on one line and move it to other lines making that same board. Likewise, models for individual components can be imported into new knowledge bases. And even if the processes are slightly different, AIMS can adapt to them during normal operation. Tweaking a knowledge base with incremental training will "make it insensitive to process variations across lines," he said.
By loosening its constraints, the same AIMS applications can run on all the different lines on a factory floor. Conversely, constraints can be adaptively tightened during normal operation to prevent escapes. The operator just takes the bad board that was let through and shows it to the system as a negative example; AIMS will automatically push the constraints a little tighter so that similar boards will fail next time.
An audio recording of reporter R. Colin Johnson's full interview with Mark Deyong of Photon Dynamics can be found online at AmpCast.com/RColinJohnson.