Design Article
Cellular Handset Audio Evolution pt 2
Ken Boyce, National Semiconductor
8/21/2005 11:49 AM EDT
Every imaginable game genre is covered on mobile handsets and most are good enough to actively engage the player. The games come in every form imaginable, from simple to quite complex, from strategy to action, and everything in between. Name a game and there is most likely a mobile platform version available.
All the major PC game makers have targeted cell phones for versions of their games. Many have already signed deals with cell phone operators and/or internet aggregators of gaming content for mobile platforms. Versions of popular standalone game console games are also being ported to the cell phone platform.
The trend towards playing games on cell phones has created a business opportunity for cell phone companies. They are now pushing their customers to get high end cell phones so that customers gain access to more and better games and entertainment. Cell phone companies want to compete against traditional gaming devices, primary the hand held game devices.
Cell phone games are big business. Total global revenues from mobile games are forecast to increase from $2.6 billion (about 6% of mobile subscribers) this year to $11.2 billion (about 15% of mobile subscribers) by 2010.
Game Sounds & Vibration
Sound and vibration (haptic feedback) within games add to the quality of the overall game-play, enhance the user experience, and satisfy expectations that most consumers have of a game: quality audio, color graphics presentation, and interaction via input/output devices.
Audio is a major differentiator and important part of the game.
The Function of Sound in Games
Music in games enhances the mood, atmosphere, and pace of the game.
Game music and sounds exist in three types: Linear, Non-linear and Source. Linear includes Title sequences, credits, cut scenes, and Full Motion Video (FMV) are normally time linear and non-interactive. During the game’s linear segments, music can be played and timed to what's happening on screen, using techniques employed by film composers. This type of music is generally played from some digital format data in memory storage (CD, Flash, RAM/ROM) on the game platform.
Non-linear sounds are primarily associated with game sections that are clearly interactive, non-linear music and sounds are used. Non-linear music and sounds are event driven by the inter-action between the player and the action of the game, i.e. something that happens on screen triggers a piece of music to be played. Non-linear music and sounds in games are generally achieved using synthesizers (with either analog or digital output) in the cell phone, in combination with digital audio files in RAM/ROM or Flash memory.
Source Music is defined as music or sounds coming from an object within the game, such as jukebox, radio, lift etc. Source music and sounds have the following sub-catagories:
Sound Effects (Sound FX), or Spot FX.
Ambience
Foley
Dialogue
Sound Effects (Sound FX), or Spot FX - Sound which incorporates all character sounds (vocal sounds, impact sounds etc.) and any other entity sounds (e.g. airplanes, bombs etc.) These sounds are almost always resident in RAM as they are synchronized with entities controlled by the game engine.
Ambience - sounds present to help set the atmosphere of a particular game segment, such as a cacophony of tropical birds to convey a jungle, or the wail of car horns, police sirens and crowds to depict a busy city. Ambient sound effects can be resident in RAM (as with Spot FX) or pre-rendered ambient streams can be read directly from CD, Flash memory, or hard drive (as with Linear music).
Foley - the miscellaneous sounds which enhance the overall aural enjoyment or realism of the game. Examples would be footsteps, or the sound of a lighted match, among many others. Foley audio quality is much closer to film quality sound, matching the graphics for games which are approaching film quality. It is generally pre-rendered, Linear sound used for cut scenes and Full Motion Video.
Dialogue - brings life and character to the “actors” in the game and (except for FMV) is generally event driven and non-linear. Dialogue is generally played back from memory storage in the cell phone.
Haptic Feedback
Haptic Feedback devices have been common on PC platforms for quite some time, but are just beginning to appear on cell phones. Haptic Feedback devices give the game player some sort of “force feedback” in response to actions in a game, as well as to actions which the game player may take. Generally, the action response is triggered by Sound FX and Spot FX events (audio events such as explosion or machinery sounds), while the game player action response is triggered by the game program depending upon the game player input command (such as a movement of some kind).
Gambling
Gambling can be looked at as part of the Gaming market, but from the system perspective it is really significantly different. However, the audio content is similar to that for gaming.
The market for mobile gambling content is predicted to rise from US$1.2B of annual revenues in 2005 to US$7.6B by 2010, assuming operators and the companies who run mobile gambling services companies win legislative and regulatory approvals.
Television
Some wireless industry experts think TV (small screen version) on cell phones will be the next big growth segment. That will require people to essentially stop whatever else they are doing and actually watch, which could slow the adoption of cell phone TV and related services. Informa Telecom and Media concluded in a recent study that 5 percent (about 125M) of all cell phone owners - will be watching TV on handsets by 2010. The cell phone will become a new “outlet” for entertainment and advertising, with as many as 100 channels available.
The programming trend is for shorter (about 1 minute) highly condensed versions of popular programs - called mobisodes (“mobile episodes”) - produced especially for cell phones which begin playing when you select them. They may also be recorded on the phone for later viewing. Several companies have emerged to prepare specially formatted audio and video content for the “smaller screen”.
Cell phone TV is made available in two forms:
Over the cellular network downloaded subscription or free content at about 6 FPS, and requiring special Java or BREW software in the phone
Free, and subscription, over-the-air TV received directly by an antenna and hardware inside the phone (standard 25-30 FPS).
Two over-the-air services are DVB-H (Digital Video Broadcasting - Handheld), and ISDB-T (Integrated Services Digital Broadcasting - Terrestrial). The DVB-H and ISDB-T are open and non-proprietary standards to deliver mobile digital TV to mobile phones sold in Europe, the U.S., parts of Asia and Japan.
Another over-the-air service is satellite-based DMB (Digital Multimedia Broadcasting) already launched in Korea with 7 TV channels and 24 audio channels which uses frequencies around 2.6GHz, relatively close to 3G telephony frequencies.
Some operators and cellular technology providers are building special networks for delivering video and audio content on cell phones which will not load down the cellular networks.
Depending upon the mechanism for delivery of the TV service, the audio content could be either analog or digital and is likely to be independent of baseband and applications processors.
RADIO
Radio is how many people hear different music genres, and is expected to increase related sales of ring sounds and music downloads. Market researcher IDC estimates cell-phone radio to generate over $70 million in sales in 2005, with increases in following years as more companies and content providers launch radio services. Cell phone radio services are expected to be larger revenue generators than cell phone TV, primarily because you don’t have to watch it to get the benefits.
Different ways are being considered for delivery of radio broadcasts to mobile phones. FM or AM/FM radio modules are currently available in cell phones, while some makers are considering satellite radio receivers. Others are considering terrestrial based DAB (Digital Audio Broadcasting). DAB has the capabilities of also carrying text, pictures, video, and data.
Streaming radio over the cellular wireless network has the potential problems of network overload and reception dead spots. Therefore, some cellular operators are building special networks for combining audio (music and radio) and video services that will not load the standard cellular network.
Depending upon how the radio service is offered, the audio content could be either analog or digital and may also be independent of baseband and applications processors.
PODCASTING
Podcasting is not restricted to MP3 players. Podcasting is growing at a phenomenal rate. Podcasting capabilities are being added to cellular phone services to take advantage of the ever increasing range of MP3 content Podcasts. The audio will be played back through the MP3 player HW or SW path within the cell phone.
FUN LIGHTING
Lighting effects - fun lighting - has become a very popular feature on cell phones, serving as both entertainment and as visual silent ring indicators. This function is a natural outgrowth of LED backlighting and key lighting capabilities already in the phone. Add colored LED’s, and you can get a simple version of the “color organ” hobby electronics kit.
Most fun lighting effects are programmed to create various patterns. However a growing number of them are either pseudo-synchronized, or actually synchronized to ring sounds and music that the cell phone is capable of playing.
SILENT RING (Vibrator)
Practically all cell phones have a silent alert vibrator. Most vibrations are made by a motor with off center weight on the shaft. Synthesizer makers provide a simple means of partially synchronizing the silent vibrator with normal ring sounds or music. Cell phone makers want the option of choosing any sound source as a possible trigger for the silent alert vibrator. Increasingly, demand is for very close synchronization to music, but this is hard to achieve as the motor and weight represent an inertial time constant generally slower than typical musical passages. Generally, it is possible to follow the slower percussion patterns of music.
LOCATION SERVICES
User Emergency and general purpose location services are readily available using cell tower information such as the cell phone’s signal strength. Now, Global Positioning Satellite capabilities are beginning to be added. The technology is similar to what GPS devices now in many vehicles do. Most GPS devices now have audio annunciation capabilities. The audio can be in analog or digital format and be provided by either a HW GPS device or SW GPS program on the baseband or applications processor.
MOBILE COMMERCE (M-COMMERCE)
“Cha-Ching” takes on new meaning as audio in your cell phone – your “authorization device” - assists you to make secure purchase transactions using personal ID numbers. Literally, the cell phone sounds like a cash register – to you as well as the product supplier. Cellular activated transaction machines would most likely use a BlueTooth audio link.
SUMMARY
Audio usage in cell phones has expanded as the dizzying array of audio related services roll out on the (now) multi-media cell phone platform. Market research supports the continued increase in consumer demand, as well as revenue expected from such services. Audio quality and performance specifications on cell phones will continue to improve, as audio becomes a major differentiating feature of the services and of the cell phone itself.
All the major PC game makers have targeted cell phones for versions of their games. Many have already signed deals with cell phone operators and/or internet aggregators of gaming content for mobile platforms. Versions of popular standalone game console games are also being ported to the cell phone platform.
The trend towards playing games on cell phones has created a business opportunity for cell phone companies. They are now pushing their customers to get high end cell phones so that customers gain access to more and better games and entertainment. Cell phone companies want to compete against traditional gaming devices, primary the hand held game devices.
Cell phone games are big business. Total global revenues from mobile games are forecast to increase from $2.6 billion (about 6% of mobile subscribers) this year to $11.2 billion (about 15% of mobile subscribers) by 2010.
Game Sounds & Vibration
Sound and vibration (haptic feedback) within games add to the quality of the overall game-play, enhance the user experience, and satisfy expectations that most consumers have of a game: quality audio, color graphics presentation, and interaction via input/output devices.
Audio is a major differentiator and important part of the game.
The Function of Sound in Games
Music in games enhances the mood, atmosphere, and pace of the game.
Game music and sounds exist in three types: Linear, Non-linear and Source. Linear includes Title sequences, credits, cut scenes, and Full Motion Video (FMV) are normally time linear and non-interactive. During the game’s linear segments, music can be played and timed to what's happening on screen, using techniques employed by film composers. This type of music is generally played from some digital format data in memory storage (CD, Flash, RAM/ROM) on the game platform.
Non-linear sounds are primarily associated with game sections that are clearly interactive, non-linear music and sounds are used. Non-linear music and sounds are event driven by the inter-action between the player and the action of the game, i.e. something that happens on screen triggers a piece of music to be played. Non-linear music and sounds in games are generally achieved using synthesizers (with either analog or digital output) in the cell phone, in combination with digital audio files in RAM/ROM or Flash memory.
Source Music is defined as music or sounds coming from an object within the game, such as jukebox, radio, lift etc. Source music and sounds have the following sub-catagories:
Sound Effects (Sound FX), or Spot FX - Sound which incorporates all character sounds (vocal sounds, impact sounds etc.) and any other entity sounds (e.g. airplanes, bombs etc.) These sounds are almost always resident in RAM as they are synchronized with entities controlled by the game engine.
Ambience - sounds present to help set the atmosphere of a particular game segment, such as a cacophony of tropical birds to convey a jungle, or the wail of car horns, police sirens and crowds to depict a busy city. Ambient sound effects can be resident in RAM (as with Spot FX) or pre-rendered ambient streams can be read directly from CD, Flash memory, or hard drive (as with Linear music).
Foley - the miscellaneous sounds which enhance the overall aural enjoyment or realism of the game. Examples would be footsteps, or the sound of a lighted match, among many others. Foley audio quality is much closer to film quality sound, matching the graphics for games which are approaching film quality. It is generally pre-rendered, Linear sound used for cut scenes and Full Motion Video.
Dialogue - brings life and character to the “actors” in the game and (except for FMV) is generally event driven and non-linear. Dialogue is generally played back from memory storage in the cell phone.
Haptic Feedback
Haptic Feedback devices have been common on PC platforms for quite some time, but are just beginning to appear on cell phones. Haptic Feedback devices give the game player some sort of “force feedback” in response to actions in a game, as well as to actions which the game player may take. Generally, the action response is triggered by Sound FX and Spot FX events (audio events such as explosion or machinery sounds), while the game player action response is triggered by the game program depending upon the game player input command (such as a movement of some kind).
Gambling
Gambling can be looked at as part of the Gaming market, but from the system perspective it is really significantly different. However, the audio content is similar to that for gaming.
The market for mobile gambling content is predicted to rise from US$1.2B of annual revenues in 2005 to US$7.6B by 2010, assuming operators and the companies who run mobile gambling services companies win legislative and regulatory approvals.
Television
Some wireless industry experts think TV (small screen version) on cell phones will be the next big growth segment. That will require people to essentially stop whatever else they are doing and actually watch, which could slow the adoption of cell phone TV and related services. Informa Telecom and Media concluded in a recent study that 5 percent (about 125M) of all cell phone owners - will be watching TV on handsets by 2010. The cell phone will become a new “outlet” for entertainment and advertising, with as many as 100 channels available.
The programming trend is for shorter (about 1 minute) highly condensed versions of popular programs - called mobisodes (“mobile episodes”) - produced especially for cell phones which begin playing when you select them. They may also be recorded on the phone for later viewing. Several companies have emerged to prepare specially formatted audio and video content for the “smaller screen”.
Cell phone TV is made available in two forms:
Two over-the-air services are DVB-H (Digital Video Broadcasting - Handheld), and ISDB-T (Integrated Services Digital Broadcasting - Terrestrial). The DVB-H and ISDB-T are open and non-proprietary standards to deliver mobile digital TV to mobile phones sold in Europe, the U.S., parts of Asia and Japan.
Another over-the-air service is satellite-based DMB (Digital Multimedia Broadcasting) already launched in Korea with 7 TV channels and 24 audio channels which uses frequencies around 2.6GHz, relatively close to 3G telephony frequencies.
Some operators and cellular technology providers are building special networks for delivering video and audio content on cell phones which will not load down the cellular networks.
Depending upon the mechanism for delivery of the TV service, the audio content could be either analog or digital and is likely to be independent of baseband and applications processors.
RADIO
Radio is how many people hear different music genres, and is expected to increase related sales of ring sounds and music downloads. Market researcher IDC estimates cell-phone radio to generate over $70 million in sales in 2005, with increases in following years as more companies and content providers launch radio services. Cell phone radio services are expected to be larger revenue generators than cell phone TV, primarily because you don’t have to watch it to get the benefits.
Different ways are being considered for delivery of radio broadcasts to mobile phones. FM or AM/FM radio modules are currently available in cell phones, while some makers are considering satellite radio receivers. Others are considering terrestrial based DAB (Digital Audio Broadcasting). DAB has the capabilities of also carrying text, pictures, video, and data.
Streaming radio over the cellular wireless network has the potential problems of network overload and reception dead spots. Therefore, some cellular operators are building special networks for combining audio (music and radio) and video services that will not load the standard cellular network.
Depending upon how the radio service is offered, the audio content could be either analog or digital and may also be independent of baseband and applications processors.
PODCASTING
Podcasting is not restricted to MP3 players. Podcasting is growing at a phenomenal rate. Podcasting capabilities are being added to cellular phone services to take advantage of the ever increasing range of MP3 content Podcasts. The audio will be played back through the MP3 player HW or SW path within the cell phone.
FUN LIGHTING
Lighting effects - fun lighting - has become a very popular feature on cell phones, serving as both entertainment and as visual silent ring indicators. This function is a natural outgrowth of LED backlighting and key lighting capabilities already in the phone. Add colored LED’s, and you can get a simple version of the “color organ” hobby electronics kit.
Most fun lighting effects are programmed to create various patterns. However a growing number of them are either pseudo-synchronized, or actually synchronized to ring sounds and music that the cell phone is capable of playing.
SILENT RING (Vibrator)
Practically all cell phones have a silent alert vibrator. Most vibrations are made by a motor with off center weight on the shaft. Synthesizer makers provide a simple means of partially synchronizing the silent vibrator with normal ring sounds or music. Cell phone makers want the option of choosing any sound source as a possible trigger for the silent alert vibrator. Increasingly, demand is for very close synchronization to music, but this is hard to achieve as the motor and weight represent an inertial time constant generally slower than typical musical passages. Generally, it is possible to follow the slower percussion patterns of music.
LOCATION SERVICES
User Emergency and general purpose location services are readily available using cell tower information such as the cell phone’s signal strength. Now, Global Positioning Satellite capabilities are beginning to be added. The technology is similar to what GPS devices now in many vehicles do. Most GPS devices now have audio annunciation capabilities. The audio can be in analog or digital format and be provided by either a HW GPS device or SW GPS program on the baseband or applications processor.
MOBILE COMMERCE (M-COMMERCE)
“Cha-Ching” takes on new meaning as audio in your cell phone – your “authorization device” - assists you to make secure purchase transactions using personal ID numbers. Literally, the cell phone sounds like a cash register – to you as well as the product supplier. Cellular activated transaction machines would most likely use a BlueTooth audio link.
SUMMARY
Audio usage in cell phones has expanded as the dizzying array of audio related services roll out on the (now) multi-media cell phone platform. Market research supports the continued increase in consumer demand, as well as revenue expected from such services. Audio quality and performance specifications on cell phones will continue to improve, as audio becomes a major differentiating feature of the services and of the cell phone itself.
Next Week: CURRENT AUDIO DEVICES IN CELLULAR HANDSETS pt 3
Kenneth Boyce is the Staff Strategic Technologist for National Semiconductor's Audio products group. Prior to joining National, Boyce served as director of the Audio and Communications Division at Oak Technology. He holds a bachelor of science degree in electronics from West Virginia University.
Navigate to related information
