Design Article
Advanced video frame rate conversion via object-based motion-compensated interpolation
Demin Wang, Senior Scientist and Team Leader, Video Coding and Processing, and<br>André Vincent, Manager of the Advanced Video Systems Group, Communications Research Centre, Canada
2/23/2007 3:35 AM EST
The convergence between broadcasting, telecommunication, and Internet has created a proliferation of the number of video formats and increased the need for high quality frame rate conversions. Frame rate represents the number of frames (images) per second of a video or film material. For example, 35mm film has a frame rate of 24 frames per second (fps), television has a frame rate of 30 fps in North America and 25 fps in Europe, while computer monitors work at 60, 75, or even higher frame rates. With the convergence, users require that a single display device be able to display high quality video at various frame rates coming from different sources.
Thanks to advances in IC technology, video display devices can include a variety of image quality enhancement and format conversion capabilities. Digital noise reducers, coding artefact reducers, and de-interlacers have been built in high-end display devices and significantly improve the video quality. Frame rate conversion may be the next technology to be integrated into modern video displays.
High quality frame rate conversion is, however, one of the most challenged processes in image and video processing. Simple techniques, such as frame repetition and temporal filtering, often result in motion judder or blurred images.
A more complicated technology, called motion-compensated frame interpolation (MCFI), estimates motion trajectories and interpolates new images along the motion trajectories. This may yield high quality conversion if the true motion trajectories are accurately estimated and the occlusion areas caused by motion are properly processed. Unfortunately, it is very difficult to accurately estimate the true motion and to properly process motion-occluded areas.
We have developed an advanced frame rate converter, called CRC-FRC, for high quality video and film frame rate conversion. At the core of this converter are an object-based motion-compensated frame interpolation (object-based MCFI) method and a mechanism to measure motion field reliability. This converter is able to accurately estimate true motion trajectories, process well motion occlusion, and generate high quality images and smooth motion.
In this article, we first briefly describe the CRC-FRC algorithms. Then we report the performance evaluation of CRC-FRC, compared with four existing frame rate converters: ReTimer from REALVIZ, Twixtor from Re:Vision, frame repetition, and frame averaging. The comparison was conducted in terms of both objective and subjective quality of the video generated by these converters.
NEXT: The Advanced Frame Rate Converter Algorithm
Thanks to advances in IC technology, video display devices can include a variety of image quality enhancement and format conversion capabilities. Digital noise reducers, coding artefact reducers, and de-interlacers have been built in high-end display devices and significantly improve the video quality. Frame rate conversion may be the next technology to be integrated into modern video displays.
High quality frame rate conversion is, however, one of the most challenged processes in image and video processing. Simple techniques, such as frame repetition and temporal filtering, often result in motion judder or blurred images.
A more complicated technology, called motion-compensated frame interpolation (MCFI), estimates motion trajectories and interpolates new images along the motion trajectories. This may yield high quality conversion if the true motion trajectories are accurately estimated and the occlusion areas caused by motion are properly processed. Unfortunately, it is very difficult to accurately estimate the true motion and to properly process motion-occluded areas.
We have developed an advanced frame rate converter, called CRC-FRC, for high quality video and film frame rate conversion. At the core of this converter are an object-based motion-compensated frame interpolation (object-based MCFI) method and a mechanism to measure motion field reliability. This converter is able to accurately estimate true motion trajectories, process well motion occlusion, and generate high quality images and smooth motion.
In this article, we first briefly describe the CRC-FRC algorithms. Then we report the performance evaluation of CRC-FRC, compared with four existing frame rate converters: ReTimer from REALVIZ, Twixtor from Re:Vision, frame repetition, and frame averaging. The comparison was conducted in terms of both objective and subjective quality of the video generated by these converters.
NEXT: The Advanced Frame Rate Converter Algorithm
|
|
|
|
|
Navigate to related information
