Akin to the promptings that newscasters get from their producers via earplugs during a broadcast, mentoring (as the technique is called) has already proved valuable to military teams in achieving mission goals. But instead of a human mentor, the Sandia group is looking to devise a software mentor that could offer advice like "take a deep breath and relax your upper body" or "pay attention to what Private Smith is about to say, his excitement level indicates it could be important."

The researchers recently used a neural network to learn the signatures of abstract desirable traits like "leadership," as well as warning signs such as "nervous," "afraid" or "daydreaming," by analyzing the pulse, respiration, perspiration, facial expressions, head movements and other biometric data streams coming from sensors attached to group members. Goals could be achieved in a stressful virtual environment only if the group cooperated effectively. By perfecting their approach in virtual reality, the scientists hope to someday enable automatic advice and counsel from a virtual mentor.

"What we have demonstrated so far is that we can monitor people's vital signs and use a self-organizing map to perceive in the data qualities like leadership," said project manager Peter Merkle, of the labs' Advanced Concepts Group. "For instance, twice in our experiment leaders emerged, and each time their pulse rate increased whenever one of their team was in jeopardy."

The Sandia researchers believe people react in predictable ways to stress, and that a real-time virtual mentor could use "human" interpretations of these signatures to offer appropriate advice. For instance, if fear can be discerned from shallow breathing and increased pulse rate, then a mentor could advise a soldier to "slow your breathing and relax your muscles." "You can imagine all sorts of advice, such as 'take a deep breath' when people start overreacting to stress," said Merkle.

Sandia's Advanced Concepts Group (ACG) is a think tank that implements its concepts as prototypes. Engineers Dave Warner, Steve Birch and Tim Murphy at subcontractor MindTel LLC (Syracuse, N.Y.) developed the MentorPal prototype using off-the-shelf components and custom-written software based on networked PCs.

Another group at Sandia is researching the best means of providing real-time mentoring, for now using real people. Eventually, Merkle's ACG project will incorporate the knowledge from the other program into its neural net.

ACG's neural network is a self-organizing map, of the type successfully used in scores of applications for almost 20 years. Self-organizing maps are multidimensional, but can be visualized as a flat-sheet neural-network array where the nodes automatically tune themselves to the categories into which the data is naturally clustered. This learning process is automatic, or unsupervised in the jargon, and does not require human intervention. It's also "competitive," thereby resulting in a map of all the natural data clusters that can be used to recognize these categories within real-time data streams.

For leadership, the natural groupings of data in the Sandia experiment revealed that sympathy-expressed as elevated pulse rate in response to team members being at risk-was easily discerned from the categories formed automatically by the self-organizing map, as were other, more-subtle indications. "Right now we just want to learn how to identify traits we think are important, such as leadership," said Merkle. "We also think we may be able to identify trust too," so that the virtual mentor can provide advice that builds, rather than degrades, trust.

MentorPal's hardware architecture begins with a dozen or more miniature sensors and transmitters connected into a wearable network called the Personal Assistance Link. PAL monitors a soldier's vital signs in real-time, providing both a chart-recorder style moving trace for each sensor and an analytic output from the self-organizing map indicating learned categories, such as "drowsy" or "afraid." Then the software links the generic maps of individual performance to the rest of the work group.

Besides the usual array of pulse, respiration and similar physical sensors, PAL also monitors facial expressions, tone of voice and head motions via the same kind of accelerometers that trigger airbags to inflate. Frowning a head shake, a gesture that a human would immediately interpret as "disapproval," can thus be easily discerned and broadcast to other participants.

"There are all sorts of conditions we could watch for, from boredom to hyperactivity," said Merkle. "We could advise a speaker that his audience is becoming confused and 'please provide an example.' Or 'Tom is falling asleep, replace him with Jim, who is refreshed and ready.' The possibilities are endless."

Merkle claims real-time metrics indicate MentorPal could improve teamwork, identify leadership, require less energy to achieve goals and deal more effectively with continuing threats. In general, the researchers hope to map the characteristics that enable each participant in a group to provide "personal best" performance.

Merkle's group plans to add a 128-channel electroencephalogram to correlate brain events with social interactions, along with an electromyograph to measure muscle activity, an electrocardiogram to measure heartbeat, blood volume pulse oximetry to measure oxygen saturation and a respiration monitor to measure breathing depth and rapidity. Sandia also plans to cooperate with the University of New Mexico and the California Institute of Technology in MentorPal development projects.