Digital video capturing is getting popular with the decreasing price of camcorders and the increasing availability of devices with embedded video cameras such as digital-still cameras, mobile phones and PDAs. While a raw home video is considered as visually non-appealing, having multiple recordings of the same event provides the opportunity to combine audio and video segments from different cameras for improving quality and aesthetics. Mixing content from different recordings requires precise synchronization among the recordings. In most present applications, synchronization is done manually and considered as a very tedious task. In this paper, we propose a novel automated synchronization approach based on detecting and matching audio and video features extracted from the recorded content. We assess experimentally three realizations of this approach on a common data set and make recommendations on the usability of the different realizations in practical use cases. The realizations have no limitations on the number and movement of the cameras. Moreover, they are robust against various ambient noises and audio-visual artifacts occurring during the recordings.

Many people shoot videos at social occasions like parties, concerts, weddings, vacations, etc., using not only camcorders but also mobile phones, PDAs and digital-still cameras. Generally, raw home videos are considered boring because they are long and lack visual appeal. Moreover the recordings captured by multiple cameras in an event contain overlapping content. Considering the total length of all the recordings, it is not only boring but also impractical to watch them all. Multiple cameras provide representations of the event from different viewing angles. These multiple views and camera transitions give an impression of dynamics in video.

Similarly, the aesthetic quality of the combined-videos can be optimized by selecting the best quality audio-video segments from different recordings. In professional productions, the use of multiple cameras is very common. The synchronization of the recordings is ensured by physically connecting the cameras to a device called “jam sync” or by the use of a “clap” at each camera-take, which is a sequence of frames captured by a video camera from the moment the camera is clicked on to the moment it is clicked off. However, in the case of home video recordings, control and coordination among different cameras are not feasible. The cameras are set independently and turned on and off at the will of their users. In current applications involving multiple cameras, the synchronization offset between the recordings is searched manually. The search is considered very time consuming and difficult. It involves finding an instant in all the available recordings containing a distinctive object motion or sound, for example, the sound of a clap or a step in dance.

Then further careful observation in both of the recordings is required to accurately locate the synchronization point. Video editing tools such as Adobe Premiere Pro, Final Cut Pro, and Ulead, which currently require manual synchronization, could offer automatic solutions in their existing products. Another useful application of automated synchronization would be for large video repositories like YouTube, Google, and Yahoo! that contain multiple clips of an event recorded by different people. Presently, the clips are not organized according to an event and each of the clips can only be accessed individually. Automated synchronization of multiple clips available from an event facilitates easy managing and simultaneous watching of the clips. In this paper we propose a novel automated synchronization approach to synchronize multiple camera recordings.

The approach is based on detecting and matching audio and video features extracted from the recorded content. Depending on the features used, we introduce three realizations of the proposed approach using detected flashes from digital-still cameras, audio fingerprints and onsets. We assess them experimentally and recommend their usability based on practical cases.

One of the first problems encountered while combining audiovisual segments from different cameras is that the recordings require precise synchronization, also called time-alignment. Even a slight misalignment between two videos results in discontinuity of the time flow. Similarly, the misalignment between audio and video of different cameras causes lip-sync problems. We assess them carefully and experimentally and recommend their usability based on practical cases.