In the emerging wireless communication applications, multimedia data will be streamed over a variety of access networks to a multitude of appliances with different resources or capabilities in terms of display size, processing power, hardware support, memory and so on. Hence, multimedia data can be accessed by a large number of users and clients from anywhere, at any time, in a networking-access paradigm often referred to as universal multimedia access.

To accommodate so large a variety of requirements and network characteristics, layered video-coding technologies-of a type that are scalable in terms of bandwidth and complexity-are essential to enable adaptive video transmission. Examples of this kind of video- coding technology are MPEG's spatial-temporal scalability, data partitioning, wavelet coding and MPEG-4 fine-granular scalability (FGS). Of these, FGS strikes a good balance between flexibility and complexity.

FGS has been adopted into the MPEG-4 video-coding standard for the efficient and flexible distribution of video over heterogeneous networks, because the approach has the ability to perform temporal-signal/noise ratio trade-offs at transmission time, depending on the channel conditions or the user's preference.

In addition, the complexity scalability of FGS can be easily established by scaling nonbit-rate-scalable processing tasks (such as inverse-discrete cosine transform) in the decoding pipeline, providing a compression scheme that makes it possible to generate multiple levels of complexity.

Analysis shows that the complexity of decoding FGS-coded content with such scaling is proportional to the receiving bit rate. Hence, to implement complexity-driven multicast, the entire overcoded FGS enhancement layer is further divided evenly into multiple sublayers such that the overall complexity level required to process or decode these layers is proportional to the number of layers the receiver acquires. The base layer and the FGS sublayers form a hierarchical layered structure with descending priority.

At system level, the bit streams generated by source-coding tools such as MPEG-4 are referred to as elementary bit streams. Layered source-coding tools normally generate multiple elementary bit streams. Before any one of them can be sent out to a wired or wireless network, a transport protocol has to be designed to carry the bit stream onto the underlying network.

The Real-time Transport Protocol (RTP), defined by the Internet Engineering Task Force, provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. Functions such as payload type identification, sequence numbering, time stamping and delivery monitoring are sufficient to transport layered content when multiple, distinct RTP transports (or RTP sessions) are used simultaneously.

These RTP sessions form the multiple channels (or more precisely, multicast-group channels) to deliver FGS-encoded video. Each FGS sublayer is carried in one RTP session, forming a virtual channel. In multicast, these channels are mapped directly to distinct Internet Protocol Multicast groups, thereby allowing the receivers to adjust their reception rate on the fly by controlling the number of groups they receive.

Multichannel model

The sender simply transmits the packets to the "group address" associated with each channel. The receivers subscribe to a number of groups that they are capable of decoding based on their capabilities, available computational resources and experienced network condition. The receiver dynamically joins or leaves a multicast group via Internet Group Management Protocol (IGMP) to apply this type of channel control. This rate-adaptation strategy is referred to as a multichannel streaming model.

This multichannel architecture provides such a flexible framework for performing complexity- and bandwidth- adaptive streaming that different types of application-layer congestion- and rate-control protocols can be accommodated. Among them are TCP-Friendly Rate Control and Receiver-Driven Layered Multicast (RLM).

Packetization is performed in a so-called hinting process that generates "hint tracks" for media tracks containing media elementary bit streams (ES), as defined in MPEG-4 systems.

Hint tracks contain instructions-or hints-for packaging media ES tracks into transport packets for transmission. The transport headers and transport payload pointers of hint tracks point to the transport payload contained in the media ES tracks.

Dropping hints

Layered video coding normally generates one media ES track that can be divided in sublayers that have different priorities and complexity levels. We propose a hinting method that can generate multiple hint tracks for one media ES track, so that the media ES track to which these hint tracks point can be delivered over the network using multiple RTP connections. Such a scheme endows the streaming system with the flexibility to adapt to network conditions and complexity requirements by adjusting the number of transmitted scalable layers.

The multi-to-one relationship between the hint tracks and the media ES track breaks the original one-to-one relationship employed for hinting nonscalable streams, thereby providing the flexibility necessary for scalable video transmission. The Session Description Protocol is used as the description format to convey the layered structure of the content as well as the progressive complexity levels in terms of bit rate.

In our system, a hybrid packetization scheme is devised to achieve a good balance of robustness and efficiency for layered content. To take advantage of MPEG-4 error-resiliency tools such as video packet in order to achieve higher robustness, each video packet in the base layer is packaged into one RTP packet, if the video packet is smaller than the path maximum transmission unit (MTU).

This facilitates error concealment in the decoding stage to improve base layer robustness.

For the FGS enhancement layer, multiple video packets are packaged into one RTP packet, if the total size of these video packets is smaller than the path MTU. This reduces the RTP/IP header overhead to achieve higher efficiency. Other general principles for packetization of MPEG-4 audio-video content, as described in the relevant IETF specifications, are also preserved in our method.

In multicast-streaming applications using the multichannel-streaming model, the receiver dynamically joins or leaves a multicast group via IGMP depending on two parameters: experienced network bandwidth, and device capability and available computation resources.

The receiver starts with the most important channel-namely, the base layer-and then performs the following channel control algorithm via the RLM protocol, incrementing its subscription to the next layer in descending order of priority until it reaches either its computational or network capability, whichever is lower.

Step by step

The principle of the adaptation can be summarized as follows:

• Upon network congestion or resource contention, leave the most insignificant active channel.

• In the case of spare bandwidth and spare resources, if the back-off timer expires, join the next inactive channel.

• If the attempt to join the next channel fails, increase back-off timer by a factor of two.

The channel loss rate and complexity level can be obtained implicitly at the receiver by monitoring the gaps in RTP sequence number in the receiving queues and the status of the rendering queue. To achieve better performance in adaptation and multicast scalability, IGMP v.3 is preferred over earlier versions.

This architecture provides an efficient framework for the transmission of numerous simultaneous multimedia-streaming sessions to devices with very different capabilities. The proposed architecture is very generic and can be widely used together with any other layered video-coding scheme capable of prioritizing content.

http://www.eet.com