Abstract
This paper presents an algorithm for automatically detectingand segmenting a moving object from a monocularvideo. Detecting and segmenting a moving object from avideo with limited object motion is challenging. Since existingautomatic algorithms rely on motion to detect the movingobject, they cannot work well when the object motion issparse and insufficient. In this paper, we present an unsupervisedalgorithm to learn object color and locality cuesfrom the sparse motion information. We first detect keyframes with reliable motion cues and then estimate movingsub-objects based on these motion cues using a MarkovRandom Field (MRF) framework. From these sub-objects,we learn an appearancemodel as a color Gaussian MixtureModel. To avoid the false classification of background pixelswith similar color to the moving objects, the locationsof these sub-objects are propagated to neighboring framesas locality cues. Finally, robust moving object segmentationis achieved by combining these learned color and localitycues with motion cues in a MRF framework. Experimentson videos with a variety of object and camera motiondemonstrate the effectiveness of this algorithm.
1. Introduction
Automatically detecting and segmenting a moving objectfrom a monocular video is useful in many applicationslike video editing, video summarization, video coding, visualsurveillance, human computer interaction, etc. Manymethods have been presented (c.f. [21, 9, 3, 24, 23]). Manyof them aim at a robust algorithm for extracting a movingobject from a video with rich object and camera motion.However, extracting a moving object from a video with lessobject and camera motion is also challenging. Most previousautomatic methods rely on object and/or camera motionto detect the moving object. Small motion of the objectand/or camera do not provide sufficient information forthese methods.For example, most existing methods use motion to detectmoving objects. They assume if a compact region movesdifferently from the global background motion, it mostlylikely belongs to a moving object. Motion-based methods[8, 12, 21, 9, 3] usually take the detected moving pixels asseeds, and cluster pixels into layers with consistent motions(and consistent color and depth). When motion informationis sparse and incomplete, they cannot work robustly. Forexample, Figure 1 shows an example where a boy sits onthe floor andmoves only in a fewframes. And even in theseframes, he only moves a part of his body. Methods usingobject motion information can only detect an incompletepart of the object. For example, if we segment the objectin a popular Markov Random Field (MRF) framework, asdescribed in § 2.3, only the moving part of the boy’s bodyis detected in frames where the part moves, and no meaningfulregion is found in other frames as shown in Figure 1(b) and ©. This example shows that using object motionalone to infer moving objects is insufficient. Similarly, inthis example, since the camera barely moves, it is also difficultfor a structure from motion (SFM) algorithm as usedin methods like [24] to obtain useful depth information toinfer the moving object.Impressive results have been reported recently for bilayervideo segmentation in the scenario of video chatting[4, 23]. These algorithms can robustly segment a majorforeground object from a video with dynamic background,however, they are not suitable for videos with complex cameramotions.Instead of building a moving object model, some othermethods build a background model to detect and segment amoving object (c.f. [5, 10, 17, 15, 18, 22]). These methodswork well for videos with static cameras. When videos havecomplex camera motions, the background model is hard tobuild.This paper presents a solution that learns amoving objectmodel by collecting the sparse and insufficient motion informationthroughout the video. Specifically, we presentedan unsupervised algorithm to learn the color and localitycues of the moving object

Download full report
http://graphics.cs.wisc.edu/Papers/2009/...PR2009.pdf