Design Article
Background subtraction, part 1: MATLAB models
Seth Benton
8/10/2008 12:00 PM EDT
Frame Difference
Frame difference is arguably the simplest form of background subtraction. The current frame is simply subtracted from the previous frame, and if the difference in pixel values for a given pixel is greater than a threshold Ts, the pixel is considered part of the foreground.
Figure 2 shows the frame difference method applied to the test video. Non-black pixels are foreground pixels.
As can be seen, a major (perhaps fatal) flaw of this method is that for objects with uniformly distributed intensity values (such as the side of a car), the interior pixels are interpreted as part of the background. Another problem is that objects must be continuously moving. If an object stays still for more than a frame period (1/fps), it becomes part of the background.
This method does have two major advantages. One obvious advantage is the modest computational load. Another is that the background model is highly adaptive. Since the background is based solely on the previous frame, it can adapt to changes in the background faster than any other method (at 1/fps to be precise). As we'll see later on, the frame difference method subtracts out extraneous background noise (such as waving trees), much better than the more complex approximate median and mixture of Gaussians methods.
A challenge with this method is determining the threshold value. (This is also a problem for the other methods.) The threshold is typically found empirically, which can be tricky. This is where MATLAB's powerful visualization tools come in handy.
Instead of running the m-file for a series of values and examining the output (a possibly time-consuming exercise), I decided to whip up a simple MATLAB gui (m-file). As shown in Figure 3, the GUI allows you to change the threshold value and see the resulting output. Using the GUI, I was able to quickly zero in on the threshold that gave the best looking output.
(Click to enlarge)
Figure 3. MATLAB GUI for determining threshold value.
For this example, a GUI probably didn't save too much time. But when more variables are involved (as is the case with other methods), doing quick and dirty optimization with a GUI (at least for a first order approximation) can save considerable time. In addition, a GUI can be a good way to "visually debug". In my personal experience, looking at what's happening to the pixels can often tell you where your code is malfunctioning quicker than looking through the code
If you'd like to experiment further with the frame difference method, the m-file can be found here.
Next: Approximate median
|
|
|
|
|
tlee@sp
8/12/2008 10:46 PM EDT
I found the article informative in that different approaches were compared and videos were available. The code was a definite plus.
Please keep up the good work Seth!
Sign in to Reply
MDN
11/13/2008 1:23 AM EST
Hello sir,
I found your article very useful.Can you please suggest me how would I improve the perfomance of a video using approximate median filter when there is a large change in the background.
Thanks,
Santosh
Sign in to Reply
ximulka
5/5/2009 7:38 AM EDT
Thanks very much, it was veeeery useful for my research!
Sign in to Reply
ximulka
5/19/2009 6:11 AM EDT
Could you write the name of the paper that you based the MoG algorithm on? I'm trying to adopt it for audio background detection, and it would be very useful for me. Thanks
Sign in to Reply
sairam33
6/19/2009 11:36 AM EDT
hello sir,
The article was very useful for my studies.how can we improve the speed of mixture of gaussians
Sign in to Reply
Laplace Jr.
11/13/2009 7:48 PM EST
Very nice article, it is so helpful for my research.
Thanks so much for sharing this ...
Laplace Jr.
Sign in to Reply
Thierry17
3/24/2010 8:39 AM EDT
A complete survey on background subtraction using GMM is available here:
http://hal.archives-ouvertes.fr/hal-00338206_v1/
and fuzzy version here:
http://hal.archives-ouvertes.fr/hal-00336512_v1/
best regards!
Thierry
Sign in to Reply
Thierry17
3/29/2010 2:41 AM EDT
Very nice article and more information are available in the following publication:
http://hal.archives-ouvertes.fr/docs/00/33/82/.../RPCS_2008.pdf
and fuzzy version of the GMM:
http://hal.archives-ouvertes.fr/hal-00336512_v1/
Best regards!
Thierry
Sign in to Reply
Thierry17
3/29/2010 2:45 AM EDT
A complete survey on background subtraction using GMM is available here:
T. Bouwmans, F. El Baf, Modeling of Dynamic Backgrounds by Type-2 Fuzzy Gaussians Mixture Models, MASAUM Journal of Basic and Applied Sciences, Volume 1, Issue 2, pages 265-277, September 2009.
T. Bouwmans, F. El Baf, B. Vachon, Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey, Recent Patents on Computer Science, Volume 1, No 3, pages 219-237, November 2008.
Sign in to Reply
Thierry17
3/29/2010 2:47 AM EDT
A complete survey on background subtraction using GMM is available here:
T. Bouwmans, F. El Baf, B. Vachon, Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey, Recent Patents on Computer Science, Volume 1, No 3, pages 219-237, November 2008.
and fuzzy version of the GMM:
T. Bouwmans, F. El Baf, Modeling of Dynamic Backgrounds by Type-2 Fuzzy Gaussians Mixture Models, MASAUM Journal of Basic and Applied Sciences, Volume 1, Issue 2, pages 265-277, September 2009.
Best regards!
Thierry
Sign in to Reply
Ahmad52
6/4/2010 12:48 PM EDT
Thank you Sir
It is very much informative for me
But sir I did not understood in approximate median filter method that why the background is incremented and decremented by 1
and where is the role of median filtering in that code
And sir kindly can you tell me that can it be extended so that we can track a specific moving object not all
Thaks
Sign in to Reply
mdabibi
9/2/2010 8:56 AM EDT
i think it's because of the Video Clip in that site is not AVI Standard Form.you must convert it to AVI Standard Form.
mdabibi@gmail.com
Mohammad Dabibi
Sign in to Reply
Ahmad52
6/6/2010 10:52 AM EDT
Thank you Sir
It is very much informative for me
But sir I did not understood in approximate median filter method that why the background is incremented and decremented by 1
and where is the role of median filtering in that code
And sir kindly can you tell me that can it be extended so that we can track a specific moving object not all
Thanks
Sign in to Reply
mdabibi
9/1/2010 8:27 PM EDT
dear sirs,
I couldn't download Test video used for background subtraction .would you please send it to my email.
i need it very soon.
thx for all
email: mdabibi@gmail.com
mohammad dabibi
Sign in to Reply
satish kumarn
10/27/2010 5:05 AM EDT
i can't view and download the test video. please send me test video through mail
email: satish.rmgm@gmail.com
advance thanks
Sign in to Reply
jacky007ex
12/22/2010 6:07 AM EST
Dear Seth Benton,
You did a great job on related topic.
May I use your m-code of Mixture of Gaussians for final year project reference? Just for academic use. Would mind sending me a confirmation letter to me in using it? or you can ask me for further details, thanks a lot.
email: gn00378304@hotmail.com
Thanks!
Sign in to Reply
WORKSTATION
4/28/2011 9:34 AM EDT
Just today I discover this article; I found your code can permit me to begin my research work.
In addition, I would like have some explication about its applications in multiple planes contexts lookdown by multiple cameras.
I want use MoG algorithm (MATLAB codes)to determine the reference background.
Thanks for all.
Sign in to Reply
WORKSTATION
4/28/2011 9:46 AM EDT
In other hand: How to use MoG to determine the reference plane(background)on a scene lookdown by three cameras; we suppose they are three or four parallel planes to the ground.
The frames are simultaneously acquired from these three cameras and already stored in jpg format.
e-mail: ibmacire@yahoo.fr
Sign in to Reply
meganega
5/7/2011 9:50 AM EDT
Mr. Seth Benton
Thanks for writing and sharing this article,
very nice article, and the code give something even more!
this really help my research, thanks again sir :)
Sign in to Reply
WORKSTATION
5/26/2011 8:17 AM EDT
I solving my last request; but the avi file obtained using the folowing function is empty.
mov=im2frame(uint8(fg), gray);
movie2avi(mov,'fileName','fps',15);
The file is creat but empty!
Please, I need your help.
E-mail: ibmacire@yahoo.fr
Sign in to Reply
Jervans
1/4/2012 8:09 AM EST
The files seem to be down. Can they be uploaded again?
Sign in to Reply
Acer4321
5/22/2012 12:11 AM EDT
hey,thanks for useful info.
Can anyone tell me how to convert any video into graysacle video in matlab?????????
Sign in to Reply
jacaheyo
7/26/2012 11:33 AM EDT
Thank you very much for the detailed article!
Some images and the m-files are no longer available.. is it possible to re-upload them again?
Or does anyone have the files?
Thank you very much!
Sign in to Reply
beckzhan
7/30/2012 10:49 PM EDT
Hello.
I need help video substraction and video tracking. Frame differencing and Gaussian method.
alen.almaty@mail.ru
Sign in to Reply
beckzhan
8/2/2012 10:41 AM EDT
Hello Guys.
I have done Frame differencing.It is so easy. and I am doing Gaussian method. I almost finished, but could I ask you about some mistakes?
Best regards,
Bek.
Sign in to Reply
wncbe
8/4/2012 8:25 PM EDT
I am not able to download the codes. How to do it?
Sign in to Reply
mohammedawney84
1/12/2013 9:04 PM EST
please reupload the code again,thanks sir
Sign in to Reply
sailendra123
2/15/2013 5:45 AM EST
please re upload the m files as they are missing.
Sign in to Reply
mani.kandan_#14
2/19/2013 3:39 AM EST
Hai sir,
Thanks for useful information,
But i need matlab code for foreground extraction in dynamic background and algorithms used for that.
Sign in to Reply
supriyam
5/9/2013 7:13 AM EDT
hello sir,
i m not able to find matlab code for background subtrcation.plz tell from where to download it..
Sign in to Reply