Design Article
Background subtraction, part 1: MATLAB models
Seth Benton
8/10/2008 12:00 PM EDT
Approximate median
In median filtering, the previous N frames of video are buffered, and the background is calculated as the median of buffered frames. Then (as with frame difference), the background is subtracted from the current frame and thresholded to determine the foreground pixels.
Median filtering has been shown to be very robust and to have performance comparable to higher complexity methods. However, storing and processing many frames of video (as is often required to track slower moving objects) requires an often prohibitively large amount of memory. This can be alleviated somewhat by storing and processing frames at a rate lower than the frame rate— thereby lowering storage and computation requirements at the expense of a slower adapting background.
A more efficient compromise was devised back in 1995 by UK researchers N.J.B. McFarlane and C.P. Schofield. While doing government funded research on piglet tracking in large commercial farms, they came up with an efficient recursive approximation of the median filter. Their 'approximate median' method, presented in their seminal paper, 'Segmentation and tracking of piglets in images', has since seen wide implementation in the background subtraction literature, and been applied to a wide range of background subtraction scenarios.
The approximate median method works as such: if a pixel in the current frame has a value larger than the corresponding background pixel, the background pixel is incremented by 1. Likewise, if the current pixel is less than the background pixel, the background is decremented by one. In this way, the background eventually converges to an estimate where half the input pixels are greater than the background, and half are less than the background—approximately the median (convergence time will vary based on frame rate and amount movement in the scene.) Figure 4 shows the approximate median method at work on the test video.
As you can see, the approximate median method does a much better job at separating the entire object from the background. This is because the more slowly adapting background incorporates a longer history of the visual scene, achieving about the same result as if we had buffered and processed N frames.
We do see some trails behind the larger objects (the cars). This is due to updating the background at a relatively high rate (30 fps). In a real application, the frame rate would likely be lower (say, 15 fps). If you'd like to tinker with the update rate and eliminate the trails, the m-file can be downloaded here.
To get a feel for how the background model works, sometimes it's useful to visualize it. Below is a video of the background model. Rather ghostlike if you ask me.
This method is a very good compromise. It offers performance near what you can achieve with higher-complexity methods (according to my research and the academic literature), and it costs not much more in computation and storage than frame differencing.
Next: Mixture of Gaussians
|
|
|
|
|
tlee@sp
8/12/2008 10:46 PM EDT
I found the article informative in that different approaches were compared and videos were available. The code was a definite plus.
Please keep up the good work Seth!
Sign in to Reply
MDN
11/13/2008 1:23 AM EST
Hello sir,
I found your article very useful.Can you please suggest me how would I improve the perfomance of a video using approximate median filter when there is a large change in the background.
Thanks,
Santosh
Sign in to Reply
ximulka
5/5/2009 7:38 AM EDT
Thanks very much, it was veeeery useful for my research!
Sign in to Reply
ximulka
5/19/2009 6:11 AM EDT
Could you write the name of the paper that you based the MoG algorithm on? I'm trying to adopt it for audio background detection, and it would be very useful for me. Thanks
Sign in to Reply
sairam33
6/19/2009 11:36 AM EDT
hello sir,
The article was very useful for my studies.how can we improve the speed of mixture of gaussians
Sign in to Reply
Laplace Jr.
11/13/2009 7:48 PM EST
Very nice article, it is so helpful for my research.
Thanks so much for sharing this ...
Laplace Jr.
Sign in to Reply
Thierry17
3/24/2010 8:39 AM EDT
A complete survey on background subtraction using GMM is available here:
http://hal.archives-ouvertes.fr/hal-00338206_v1/
and fuzzy version here:
http://hal.archives-ouvertes.fr/hal-00336512_v1/
best regards!
Thierry
Sign in to Reply
Thierry17
3/29/2010 2:41 AM EDT
Very nice article and more information are available in the following publication:
http://hal.archives-ouvertes.fr/docs/00/33/82/.../RPCS_2008.pdf
and fuzzy version of the GMM:
http://hal.archives-ouvertes.fr/hal-00336512_v1/
Best regards!
Thierry
Sign in to Reply
Thierry17
3/29/2010 2:45 AM EDT
A complete survey on background subtraction using GMM is available here:
T. Bouwmans, F. El Baf, Modeling of Dynamic Backgrounds by Type-2 Fuzzy Gaussians Mixture Models, MASAUM Journal of Basic and Applied Sciences, Volume 1, Issue 2, pages 265-277, September 2009.
T. Bouwmans, F. El Baf, B. Vachon, Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey, Recent Patents on Computer Science, Volume 1, No 3, pages 219-237, November 2008.
Sign in to Reply
Thierry17
3/29/2010 2:47 AM EDT
A complete survey on background subtraction using GMM is available here:
T. Bouwmans, F. El Baf, B. Vachon, Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey, Recent Patents on Computer Science, Volume 1, No 3, pages 219-237, November 2008.
and fuzzy version of the GMM:
T. Bouwmans, F. El Baf, Modeling of Dynamic Backgrounds by Type-2 Fuzzy Gaussians Mixture Models, MASAUM Journal of Basic and Applied Sciences, Volume 1, Issue 2, pages 265-277, September 2009.
Best regards!
Thierry
Sign in to Reply
Ahmad52
6/4/2010 12:48 PM EDT
Thank you Sir
It is very much informative for me
But sir I did not understood in approximate median filter method that why the background is incremented and decremented by 1
and where is the role of median filtering in that code
And sir kindly can you tell me that can it be extended so that we can track a specific moving object not all
Thaks
Sign in to Reply
mdabibi
9/2/2010 8:56 AM EDT
i think it's because of the Video Clip in that site is not AVI Standard Form.you must convert it to AVI Standard Form.
mdabibi@gmail.com
Mohammad Dabibi
Sign in to Reply
Ahmad52
6/6/2010 10:52 AM EDT
Thank you Sir
It is very much informative for me
But sir I did not understood in approximate median filter method that why the background is incremented and decremented by 1
and where is the role of median filtering in that code
And sir kindly can you tell me that can it be extended so that we can track a specific moving object not all
Thanks
Sign in to Reply
mdabibi
9/1/2010 8:27 PM EDT
dear sirs,
I couldn't download Test video used for background subtraction .would you please send it to my email.
i need it very soon.
thx for all
email: mdabibi@gmail.com
mohammad dabibi
Sign in to Reply
satish kumarn
10/27/2010 5:05 AM EDT
i can't view and download the test video. please send me test video through mail
email: satish.rmgm@gmail.com
advance thanks
Sign in to Reply
jacky007ex
12/22/2010 6:07 AM EST
Dear Seth Benton,
You did a great job on related topic.
May I use your m-code of Mixture of Gaussians for final year project reference? Just for academic use. Would mind sending me a confirmation letter to me in using it? or you can ask me for further details, thanks a lot.
email: gn00378304@hotmail.com
Thanks!
Sign in to Reply
WORKSTATION
4/28/2011 9:34 AM EDT
Just today I discover this article; I found your code can permit me to begin my research work.
In addition, I would like have some explication about its applications in multiple planes contexts lookdown by multiple cameras.
I want use MoG algorithm (MATLAB codes)to determine the reference background.
Thanks for all.
Sign in to Reply
WORKSTATION
4/28/2011 9:46 AM EDT
In other hand: How to use MoG to determine the reference plane(background)on a scene lookdown by three cameras; we suppose they are three or four parallel planes to the ground.
The frames are simultaneously acquired from these three cameras and already stored in jpg format.
e-mail: ibmacire@yahoo.fr
Sign in to Reply
meganega
5/7/2011 9:50 AM EDT
Mr. Seth Benton
Thanks for writing and sharing this article,
very nice article, and the code give something even more!
this really help my research, thanks again sir :)
Sign in to Reply
WORKSTATION
5/26/2011 8:17 AM EDT
I solving my last request; but the avi file obtained using the folowing function is empty.
mov=im2frame(uint8(fg), gray);
movie2avi(mov,'fileName','fps',15);
The file is creat but empty!
Please, I need your help.
E-mail: ibmacire@yahoo.fr
Sign in to Reply
Jervans
1/4/2012 8:09 AM EST
The files seem to be down. Can they be uploaded again?
Sign in to Reply
Acer4321
5/22/2012 12:11 AM EDT
hey,thanks for useful info.
Can anyone tell me how to convert any video into graysacle video in matlab?????????
Sign in to Reply
jacaheyo
7/26/2012 11:33 AM EDT
Thank you very much for the detailed article!
Some images and the m-files are no longer available.. is it possible to re-upload them again?
Or does anyone have the files?
Thank you very much!
Sign in to Reply
beckzhan
7/30/2012 10:49 PM EDT
Hello.
I need help video substraction and video tracking. Frame differencing and Gaussian method.
alen.almaty@mail.ru
Sign in to Reply
beckzhan
8/2/2012 10:41 AM EDT
Hello Guys.
I have done Frame differencing.It is so easy. and I am doing Gaussian method. I almost finished, but could I ask you about some mistakes?
Best regards,
Bek.
Sign in to Reply
wncbe
8/4/2012 8:25 PM EDT
I am not able to download the codes. How to do it?
Sign in to Reply
mohammedawney84
1/12/2013 9:04 PM EST
please reupload the code again,thanks sir
Sign in to Reply
sailendra123
2/15/2013 5:45 AM EST
please re upload the m files as they are missing.
Sign in to Reply
mani.kandan_#14
2/19/2013 3:39 AM EST
Hai sir,
Thanks for useful information,
But i need matlab code for foreground extraction in dynamic background and algorithms used for that.
Sign in to Reply
supriyam
5/9/2013 7:13 AM EDT
hello sir,
i m not able to find matlab code for background subtrcation.plz tell from where to download it..
Sign in to Reply