The only current benchmark for measuring the speed of Java programs in hardware and software platforms — the caffeine mark overseen by Pendragon Software Corp. (Libertyville, Ill.) — should be reworked or discarded, silicon vendors say. Designers focused on getting products out the door don't often have the tools, resources or time to evaluate Java performance, and so lean on their vendors for information, observers said.

"People are playing a lot of games with caffeine marks," said Ron Stein, senior marketing manager with Nazomi Communications Inc. (Santa Clara, Calif.), a Java hardware accelerator provider. "All customers know is bigger is better. They can be snowed very easily."

Roland Samuelson, vice president of business development for Aurora VLSI Inc. (Santa Clara, Calif.), another seller of Java hardware accelerators, said it's standard practice to show prospective customers his products' high caffeine marks, but he admitted it can be difficult to put the numbers into context for them.

"They often ask me how much is enough, and I don't have an answer for them," Samuelson said.

Silicon vendors are quick to say that a competitor is making exaggerated performance claims, but admit they also play the benchmark game. "Benchmarks sell processors," said Stein.

The call for a better Java performance benchmark comes at a pivotal time for many embedded designers, especially those working on wireless phones and PDAs, hotbeds of Java activity. Many are looking to Java as one way to reenergize the stagnating cellular phone industry, which in many regions has been hurt by consumers' unwillingness to upgrade to new phones.

Critics of the caffeine marks benchmark say it doesn't measure real-world applications and lacks a certification process. At this week's Embedded Systems Conference here, representatives from processor vendors ARC Cores, ARM, Aurora VLSI, Chicory Systems, IBM, Infineon, InSilicon, Nazomi, TriMedia and Vulcan met to discuss the preliminary steps needed to create a new processor-oriented benchmark under the stewardship of the EDN Embedded Microprocessor Benchmark Consortium.

Though it would take more than six months to develop such a benchmark, the representatives said they wouldn't hesitate to use a better benchmark once it became available.

Selfish motives

Processor vendors have some selfish interest in pushing for Java benchmarks, not only to show up competitors but also to promote the use of hardware accelerators rather than software-only solutions in the embedded space. Also in the mix are Java Virtual Machines such as those from Insignia Solutions (Fremont, Calif), which offers a real-time compiler that accelerates Java on a number of general purpose processors, and Microsoft Corp.'s forthcoming Common Language Run-Time (CLR), a Java alternative.

Hardware Java acceleration, however, is under serious consideration at a number of embedded systems OEMs. Motorola Inc., for example, is working to integrate a Java hardware accelerator with an ARM core for use in future cellular phones. The company's cellular handset division has introduced two products that support Java, and two more are on the way this quarter. None of these, however, use a hardware Java accelerator. "But in our architecture for future products there will be," said Ed Valdez, director of marketing for the wireless communications division at Motorola's Semiconductor Products Sector (Austin, Texas).

ARM Ltd. itself rolled out ARM7 and ARM9 processor cores this week equipped with Jazelle, its Java accelerator. "There's broad market support for Java out there, particularly in wireless," said Jerry Ardizzone, director of U.S. sales for ARM.

There may also be a limit to how much a pure software Java solution can accomplish in the wireless space. The so-called J2 Micro Edition (J2ME) specification for mobile appliances is divided into applications that use less than 512 kbytes of Java code, and those that use more. Insignia, for one, can work in the PDA and smart-phone space where memory footprints can be as high as 4 Mbytes, but today it doesn't have Java Virtual Machine for mobile applications with the lower memory thresholds.

Hardware accelerators must also meet severe constraints related to memory size, gate count and power consumption, particularly in mobile and wireless systems. "It's a pennies and microwatts game," said Nazomi's Stein.

At a higher level, customers need tools to evaluate, for example, how well a cellular phone runs a Java game during a phone call, said Aurora VLSI's Samuelson.

Why so much interest in lightweight Java in cellular phones? Motorola's Valdez said it could be a means to jump start so-called mobile commerce. It could also give phone vendors a convenient way to upgrade their platforms without switching to a more expensive operating system like Windows CE, he said.

In Japan, mobile telecom giant NTT Docomo has been a lightning rod for Java in the cellular phones, and European makers are not far behind, observers said. And while the United States is lagging, it's a hotbed of intellectual property development and design activity, due largely to the presence of Sun Microsystems Inc., Java's inventor.