Automatic synchronization of media clips

Background

Digital graphic design, image editing, audio editing, and video editing applications (hereafter collectively referred to as media content editing applications or media editing applications) provide graphical designers, media artists, and other users with the necessary tools to create a variety of media content. Examples of such applications include Final Cut Pro.RTM. and iMovie.RTM., both sold by Apple.RTM. Inc. These applications give users the ability to edit, combine, transition, overlay, and piece together different media content in a variety of manners to create a resulting media project. The resulting media project specifies a particular sequenced composition of any number of text, audio clips, images, or video content that is used to create a media presentation.

Various media editing applications facilitate such composition through electronic means. Specifically, a computer or other electronic device with a processor and computer readable storage medium executes the media content editing application. In so doing, the computer generates a graphical interface whereby editors digitally manipulate graphical representations of the media content to produce a desired result.

In many cases, editors use media-editing applications to edit a large volume of media clips recorded by several media recording devices. Often times, multiple media recording devices record the same event at the same time to produce multiple angles and effects. Furthermore, the use of multiple media recording devices provides a redundancy of media clips from which editors choose to create a master video. For instance, an editor chooses the audio recording from an auxiliary microphone because it provides the best sound quality for a particular scene, or the editor chooses a particular video recording because of a preferred angle from which the video is shot.

The use of multiple recording devices to record a single scene, however, produces a multitude of media clips to be analyzed. Editing/processing these media clips in an efficient and effective manner requires that the media clips be sorted and synchronized in relation to one another. Manually sorting and synchronizing such vast amounts of media clips even for a day's worth of recordings, as is typically done in the film industry, amounts to a very time consuming process.

Brief summary

Some embodiments of the invention provide a structured arrangement of media content items (e.g., audio clips, video clips, etc.). The arrangement includes a collection of media content items that have been recorded over a period of time. The arrangement includes multi-clips that represent multiple layers of media content items. Each layer of the multi-clip represents a different recording device.

Data reduction is performed on the media content items in some embodiments. Data reduction reduces the number of data samples that require analysis. The data reduction process includes sorting the media content items according to certain metadata included in the media content items. For instance, media content items determined to be from the same media recording device are grouped together. Additionally, media content items are arranged in chronological order according to available temporal data.

Once the media content items from the same sources are grouped together and sorted, one or more algorithms are performed to determine whether media content items recorded by different media recording devices are related to one another (e.g., two media clips recording the same event during an overlapping time period). These algorithms compare sets of audio signals (from video clips and/or audio clips) from different media recording devices in order to determine which media content items are sufficiently related so as to constitute an overlapping pair of media clips. Based on these algorithms, a comparison score is generated. The comparison score is evaluated against a threshold value to determine whether two media content items are sufficiently related.

In some embodiments, such algorithms are used to calculate a time offset to synchronize the related media content items on a common timeline. For example, the algorithms are used to identify how much time to offset a first media clip in order to align it with a second media clip. Examples of algorithms for comparing sets of media content items include cross correlation and phase correlation. Cross correlating or phase correlating two audio signals produces a correlation function that indicates the likelihood of match for all time offsets. The peak value of the correlation function, when it satisfies a threshold value, corresponds to the time offset that produces the most likely alignment of two audio signals.

The time offset values determined from the correlation functions are applied to corresponding media content items to synchronize the group of two or more related media content items. In some embodiments, alignment of pairs of overlapping media content items is further fine-tuned by applying the algorithms on each overlapping pair. The groups of synchronized items are arranged on a common timeline as multi-clips. The multi-clips are placed on the timeline in chronological order where each multi-clip is separated by a certain time gap. The sequence of multi-clips is presented to users as an organized collection of media content items for further editing.

The arrangement of media content items is initiated by a user through a media-editing application in some embodiments. The media-editing application includes a display area for displaying media content items stored in a media library display area. The timeline includes multiple tracks upon which graphical representations of the media clips that are analyzed for synchronization purposes are displayed.

The preceding Summary is intended to serve as a brief introduction to some embodiments of the invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The Detailed Description that follows and the Drawings that are referred to in the Detailed Description will further describe the embodiments described in the Summary as well as other embodiments. Accordingly, to understand all the embodiments described by this document, a full review of the Summary, Detailed Description and the Drawings is needed.

Brief description of the drawings

The novel features of the invention are set forth in the appended claims. However, for purpose of explanation, several embodiments of the invention are set forth in the following figures.

FIG. 1 conceptually illustrates an overview of the synchronization of media clips in some embodiments.

FIG. 2 conceptually illustrates auto synchronization of media clips selected from a media library display area of clips in some embodiments.

FIG. 3 conceptually illustrates synchronization of a folder of media clips in some embodiments.

FIG. 4 conceptually illustrates metadata components of video media clips and audio media clips in some embodiments.

FIG. 5 conceptually illustrates a process for performing the audio synchronization in some embodiments.

FIG. 6 conceptually illustrates sorting media clips into respective source groups in some embodiments.

FIG. 7 conceptually illustrates a process for sorting media clips into source groups in some embodiments.

FIG. 8 conceptually illustrates arranging media clips by time code data in some embodiments.

FIG. 9 conceptually illustrates arranging media clips by file creation time data in some embodiments.

FIG. 10 conceptually illustrates a process of determining time offsets between two sequences of media clips in some embodiments.

FIG. 11 conceptually illustrates comparing media clips of a first sequence to media clips of a second sequence in some embodiments.

FIG. 12 conceptually illustrates applying a time offset to align a first sequence of media clips to a second sequence of media clips in some embodiments.

FIG. 13 conceptually illustrates identifying overlapping media clips of different sequences from different media recording sources in some embodiments.

FIG. 14 conceptually illustrates a process of fine-tuning the overlapping media clips through audio synchronization in some embodiments.

FIG. 15 conceptually illustrates an example of a correlation graph that maps a probability of match for each time offset analyzed in some embodiments.

FIG. 16 conceptually illustrates an example block diagram of a cross correlation pairing detection module that uses cross correlation for determining a match of audio signals in some embodiments.

FIG. 17 conceptually illustrates an example block diagram of a phase correlation pairing detection module that uses phase correlation for determining a match of audio signals in some embodiments.

FIG. 18 conceptually illustrates alignment of media clips by using user markers in some embodiments.

FIG. 19 conceptually illustrates intra-clip adjustment of time drift in some embodiments.

FIG. 20 conceptually illustrates inter-clip adjustment of time drift in some embodiments.

FIG. 21 conceptually illustrates a graphical user interface displaying multiple tracks of media clips to be analyzed and edited in some embodiments.

FIG. 22 conceptually illustrates a graphical user interface displaying a multi-clip generated as a result of the synchronization process in some embodiments.

FIG. 23 conceptually illustrates a process of synchronizing media clips recorded during a specified timeframe in some embodiments.

FIG. 24 conceptually illustrates the software architecture of a media-editing application of some embodiments in some embodiments.

FIG. 25 conceptually illustrates a computer system with which some embodiments of the invention are implemented.

Detailed description

In the following description, numerous details are set forth for purpose of explanation. However, one of ordinary skill in the art will realize that the invention may be practiced without the use of these specific details. For instance, many of the examples illustrate synchronization of two sequences of media clips. One of ordinary skill will realize that this is merely an illustrative example, and that the invention can apply to multiple sequences of media clips representing multiple media recording sources. In other instances, well known structures and devices are shown in block diagram form in order to not obscure the description of the invention with unnecessary detail.

In order to synchronize media clips that include audio data, some embodiments provide a method for determining whether pairs of media clips are related to one another. The method performs algorithms on the audio signals of the pair to calculate a time offset that best aligns the pair of clips. For instance, a phase correlation algorithm in some embodiments produces a correlation function where the peak value of the function corresponds to a time offset, that when applied, produces the best alignment between the two media clips.

In some embodiments of the invention, synchronization is performed by a computing device. Such a computing device can be an electronic device that includes one or more integrated circuits (IC) or a computer executing a program, such as a media editing application.

I. Overview

FIG. 1 conceptually illustrates an example of how media clips are synchronized in some embodiments. As shown, the automatic synchronization has six different stages. In this example, the first stage 105 shows ten media clips from several media recording devices that have been selected to be synchronized. At the second stage 110, media clips are arranged according to source. In this stage, the media recording source of each media clip is determined, and each media clip is sorted into groups according to source. The media recording source is determined by analyzing the recorder identifying data that is stored on the media clips as metadata. In some embodiments, the recorder identifying data includes a serial number of the video media recording device.

In this example, media clips from a first source A are sorted into a first group corresponding to source A, media clips from source B are sorted into a group corresponding to source B, and media clips from source C are sorted into a group corresponding to source C. In some embodiments, the groups into which the media clips are stored are electronic folders or bins of a computing device.

The third stage 115 shows the media clip for each source are arranged according to temporal data. The media clips are arranged based on time code information in some embodiments. Time code information is time data provided by an internal clock of a media recording device that is recorded onto the media clip during recording. Time code information represents time and date of the audio media clip recording as indicated on the clock of the media recording device. In some embodiments, the media clips are arranged according to file creation time information. File creation time information is a time stamp of the start time and/or date of the media clip recording according to the internal clock of the media recording device. The media clips for each source are placed in chronological order in accordance to the available temporal data. The media clips are placed in a sequence with proper offset from each other. That is, the spacing between each media clip of the sequence represents an actual elapsed time between the stoppage of the recording of one clip and the start of the recording of a next clip for that source.

The fourth stage 120 illustrates determining the time offset between two sequences by identifying and analyzing media clips from each sequence that match one another. The audio signal from a selected media clip of the first sequence is compared to the audio signal from several media clips of the second sequence to find a first match that satisfies a threshold value. In some embodiments, the comparison is accomplished by phase correlating the audio signals from the two clips to produce a correlation function. The correlation function provides the probability of match between the two media clips for all time offsets. Accordingly, the peak value of the correlation function corresponds to the time offset that produces the best match between the two clips.

In some embodiments, the peak value is compared against a predetermined threshold value. The threshold value, which is described in further detail by reference to FIG. 15 below, represents a minimum probability required to produce a satisfactory match and is used to filter out false matches. Thus, when the peak value fails to satisfy the threshold value, the two media clips are determined as not matching and a next media clip of the first sequence is compared to the several media clips of the second sequence.

Once a peak value is determined as satisfying the threshold value, no further comparisons are made in some embodiments and the time offset corresponding to that peak value is determined to be the time offset. The time offset represents the difference in actual time between the start of a first sequence and the start of a second sequence when recording the same event. In other words, the time offset, when applied to the first sequence, aligns the first sequence to be in time sync with the second sequence. The details of how time offset is determined will be provided by reference to FIGS. 10 and 11 below.

The fourth stage 120 conceptually illustrates that Clip 1 of source A matches Clip 7 of source B, and that the time offset between the sequences is determined to be +31 seconds. Accordingly, the sequence of source A is offset by +31 seconds. Specifically, the sequence of source A is delayed 31 seconds in relation to the sequence of source B. This delay represents a shift in time between the two clips. The shift in time synchronizes the events recorded on the sequence of source A to events recorded on the sequence of source B, as illustrated in the fifth stage 125.

Once the media clips from the two sequences are properly aligned, overlapping media clips are identified in the sixth stage 130. Overlapping media clips are those pairs of media clips that record the same event during an overlapping time period. For example, after the +31 seconds time offset is applied, Clip 1 from the first sequence and Clip 7 from the second sequence are shown to be overlapping since Clip 1 is recorded during a time period that overlaps with that of Clip 7. Clip 3 of the first sequence and Clip 10 of the second sequence are also identified as overlapping clips for the same reason.

While the fifth stage 125 shows the sequence of source A and the sequence of source B to be aligned after the time offset is applied, this alignment may not be precise for all the overlapping media clips identified in the sixth stage 130 for a variety of reasons (e.g. different clocks used by different media recording devices). Accordingly, the seventh stage 135 illustrates the fine-tuning of the overlapping media clips. Fine-tuning media clips is accomplished by cross correlating a pair of overlapping media clips to produce a correlation function. As described above, the resulting correlation function indicates the likelihood of match for all time offset. Accordingly, the peak value of the correlation function corresponds to a time offset that produces the best alignment of the overlapping media clips. The eighth stage 140 illustrates an alignment of Clip 3 and Clip 10 that is a result of applying a time offset that corresponds to the identified peak value of the correlation function.

FIG. 2 conceptually illustrates a graphical user interface ("GUI") of a media-editing application in some embodiments. Specifically, FIG. 2 illustrates the GUI at three different stages of the auto synchronization process. The GUI of the media-editing application includes a media library display area 220 for displaying several media clips 225, a Sync button 240 for initiating the synchronization process, and a media display area 245 for displaying the media clips after they have been synchronized.

At the first stage 205, the media clips 225 are displayed in a media library display area 220 from which a user makes selections. The user selects media clips from the media library display area by pressing the symbol representing the media clip the user wishes to select. The user deselects a clip in the same fashion. The selection is received through a user selection input such as input received from a cursor controller (e.g., a mouse, touchpad, trackpad, etc.), from a touchscreen (e.g., a user touching a UI item on a touchscreen), from keyboard input (e.g., a hotkey or key sequence), etc. The term user selection input is used throughout this specification to refer to at least one of the preceding ways of making a selection or pressing a button through a user interface. In some embodiments, the user selects at least two media clips from the media library display area 220 of clips to be synchronized. These media clips include video media clips and/or audio media clips. The media clips are recorded by a variety of different media recording devices. The media clips are also recordings of various durations. Media clips that are selected from the media library display area 220 are highlighted (i.e. media clip 230) and remain highlighted unless the clip is deselected.

After the user makes a selection of media clips to synchronize, the user presses the Sync button 240 at a second stage 210 to initiate the synchronization process of the selected media clips. The synchronization process is discussed in detail with regards to FIG. 5 below.

The third stage 215 illustrates the results of the synchronization of the media clips. The groups of overlapping media clips from the three different sources 250, 255, and 260 are shown to be synchronized to one another. Additionally, a common timeline 275 is created in the media display area 245 to more precisely illustrate the alignment of the clips. The groups of overlapping media clips are also arranged in the media display area according to media recording device. For instance, the top set of media clips corresponds to Camera A 250, the middle set of media clips corresponds to Camera B 255, and the bottom set of media clips corresponds to Microphone C 260. In some embodiments, overlapping clips from different sources are grouped into multi-clips 265 and 270. Multi-clips are media structures that include multiple overlapping media clips that are aligned on a common timeline. While this example only specifies media clips from three media recording devices, the auto synchronization is capable of possessing media clips from several media recording devices.

The first multi-clip 265 includes Clip 4 from Camera A, Clip 12 from camera B, and Clip 6 from Microphone C. The three overlapping clips are aligned with one another by the auto synchronization process. Specifically, the three media clips correspond to recordings of the same event and are aligned such that the three media clips will produce synchronized video and audio outputs when played back together. In this example, Clip 12 is positioned to represent a recording from Camera B that started at t=3 of the common timeline 275. Additionally, Clip 4 of Camera A and Clip 6 of Microphone C are both positioned to represent a recording that started recording at t=4 of the common timeline 275. Thus Clip 12 is aligned on the common timeline 275 to be one time unit ahead of Clips 4 and 6. As a result, when the three aligned clips are played back together, the three clips play back the same time of the same recorded event.

The second multi-clip 270 includes Clip 10 from Camera A and Clip 15 from Microphone C. The second multi-clip 270 represents media clips recorded at a later time than the first multi-clip 265. The two overlapping clips of the second multi-clip 270 are also aligned with each other by the auto synchronization process. Specifically, the two media clips correspond to recordings of the same event and are aligned such that both clips will produce synchronized video and audio outputs when played back together. In this example, Clip 10 is positioned to represent a recording from Camera A that started at t=11 of the common timeline 275, and Clip 15 of Microphone C is positioned to represent a recording started at t=12. Thus Clip 11 is aligned on the common timeline 275 one time unit ahead of Clip 15. As a result, when the two aligned clips are played back together, the two clips play back the same time of the same recorded event. Such alignment of overlapping clips from multiple media recording devices into multi-clips provides the user the ability to synchronously view and edit media clips that have recorded the same event.

II. Media Content Synchronizer

FIG. 3 conceptually illustrates a GUI of a media editing application in some embodiments. Specifically, FIG. 3 illustrates the GUI at two different stages of the folder synchronization process. The GUI includes a media library display area 340 that includes a folder 345 containing media clips 315, a Synchronize Folder button 355 for initiating the synchronization process, and a media display area 350 for displaying the media clips when the synchronized media clips are selected from the folder. The GUI imports the media clips from a media storage device 320, which contains media clips recorded by Camera A 325, Camera B, 330, and Microphone C 335. The folder synchronization process performs synchronization on all media clips included in the folder 345 in the same fashion that the selected media clips were shown to be synchronized in FIG. 2. The folder 345 represents media clips that were recorded during a predetermined time period (i.e. a day, week, etc.).

At the first stage 305, the media clips 315 stored on a media storage device 320 are imported into a media library display area 340. Prior to being imported into the media library display area 340, the media clips are stored onto the media storage device 320 by media recording devices. Media recording devices include video recording devices and audio recording devices. As illustrated in FIG. 3, the video media clips are produced by Camera A 325, Camera B 330, and the audio media clips are produced by Microphone C 335. The media clips are digital representation of images and sounds recorded by the media recording devices.

In some embodiments, the media clips 315 are sorted into folders according to the time at which each of the media clips was recorded. The timeframe for which clips are sorted in each folder vary depending on the user's preferences. For example, each folder may represent 12 hours' worth or a week's worth of media clips. FIG. 3 illustrates the folder 345 comprising a day's worth of media clips. The user initiates the synchronization process by pressing the Synchronize Folder button 355. As described above, the selection is received through a user selection input such as input received from a cursor controller (e.g., a mouse, touchpad, trackpad, etc.), from a touchscreen (e.g., a user touching a UI item on a touchscreen), from keyboard input (e.g., a hotkey or key sequence), etc.

The second stage 310 illustrates the results of the synchronization of the media clips 315 in a media display area 350. The second stage 310 also shows the results of the synchronization as multi-clips arranged in a separate folder under the Sync'd Clips folder 395. The Sync'd Clips folder in this example includes a Sync'd Clips 1 folder 396, a Sync'd Clips 2 folder 397, a Sync'd Clips 3 folder 398, and a Sync'd Clips 4 folder 399. Each of the separate folders represents a multi-clip of time-coincident media clips.

When a user selects the folders to be expanded, the synchronized media clip components of each of the multi-clips are displayed in the media display area 350. In this example, the first multi-clip 375 represents the Sync'd Clips 1 folder and includes Clip A1 from Camera A, Clip B1 from camera B, and Clip C1 from Microphone C. The second multi-clip 380 represents the Sync'd Clips 2 folder and includes Clip B2 from camera B and Clip C2 from Microphone C. The third multi-clip 385 represents the Sync'd Clips 3 folder and includes Clip A2 from camera A and Clip C3 from Microphone C. The fourth multi-clip 390 represents the Sync'd Clips 4 folder and includes Clip A3 from Camera A, Clip B3 from camera B, and Clip C4 from Microphone C. Each of the four multi-clips represents two or more overlapping clips that are aligned with one another by the synchronization process. Specifically, the overlapping clips correspond to recordings of the same event and are aligned such that the clips will produce synchronized video and audio outputs during playback. The clips are aligned such that when the clips are played back together, the clips play back the same recorded event.

The media display area 350 of the second stage 310 also shows the media clips arranged by recording device and in chronological order. The second stage 310 further shows overlapping clips from three different media recording devices that are aligned with one another. In this example, the top set of media clips corresponds to Camera A 360, the middle set of media clips corresponds to Camera B 365, and the bottom set of media clips corresponds to Microphone C 370. While this example only specifies media clips from three media recording devices, the synchronization of folders is capable of processing media clips from several media recording devices.

III. Media Clips Structure

FIG. 4 conceptually illustrates examples of media clips processed by the synchronization process in some embodiments. The video media clip 405 includes filename information 415. The filename information 415 includes alpha-numeric and symbol characters. In some embodiments, the filename information is automatically created by a video media recording device at the time of the recording. The filename information created by the video media recording device in some embodiments is created in sequential (e.g. numerical or alphabetical) order. The filename information is input or edited by a user through an input device in some embodiments.

The video media clip also includes video information 420. The video information represents the series of image data that are captured by the video media recording device and used to produce a video. In addition to the video information, the video media clip in some embodiments includes audio information 425 that corresponds to the video information. Audio information of the video media clip is recorded by an on-board audio recorder of the video media recording device. Since both the audio information and the video information are recorded by the same video media recording device, the audio and video information are in sync with one another when recorded. In other words, the audio information corresponds to the video information recorded by the video media recording device. Accordingly, the audio information provides a baseline to which other audio information is compared and synchronized.

In some embodiments, the video media clip includes time code information 430. Time code information is provided by an internal clock of the video media recording device and is recorded onto the video media clip when the video media recording device records an event. In some embodiments, time code information is an actual time and date of the video media clip recording. The time code corresponds to the recorded audio and video information.

In some embodiments, the video media clip includes file creation time information 435. The file creation time information is provided by the internal clock of the video media recording device as a stamp of the start time and/or date that the video media clip was created. Since the file creation time information provides a temporal snapshot, the information is only recorded once per video media clip. Furthermore, no two media clips from the same video media recording device can have identical file creation time information.

In some embodiments, the video media clip includes recorder identifying data 440. The recorder identifying data is unique to each video media recording device (e.g. the data includes a serial number of the video media recording device). In some embodiments, the recorder identifying data is input or edited by a user to identify the video media recording device. The recorder identifying data is provided on the video media clip to indicate the source of the particular clip. Identification of the source of a video media clip expedites the audio synchronization process by providing a data reduction metric.

In some embodiments, the video media clip includes user marker information 445. User marker information is information input by a user to indicate a location on the video media clip for which the user would like to mark. Generally, the user marker corresponds to a specific frame of the video information. In some embodiments, the user marker corresponds to a particular location on the audio information or the actual or elapsed time of the video media clip. The user marker information is utilized to expedite the audio synchronization process by providing a starting point from which to start a synchronization analysis.

In some embodiments, the media clip is an audio media clip 410. The audio media clip includes audio information 455 which is a digital representation of sound recorded from a microphone of an audio recorder. The audio media clip also includes filename information 450, time code information 460, file creation time information 465, recorder identifying data 470 and user marker information 475 similar to the corresponding information included in the video media clips. In some embodiments, the filename information, time code information, file creation time information, recorder identifying data and user marker information are included in the video and audio media clips as metadata.

IV. Auto Synchronization

FIG. 5 conceptually illustrates a process 500 for synchronizing several media clips including video media clips and audio media clips selected by a user from a media library display area of media clips in some embodiments. As shown, process 500 receives (at 505) a user selection of media clips to synchronize. As noted above, the user selects at least two media clips from the media library display area of clips to be synchronized. Next, the process categorizes (at 510) media clips according to source. The process determines the media recording source of each media clip by analyzing the recorder identifying data. Process 500 then sorts each media clip into groups according to the source. Specifically, media clips from a first source are sorted into a group corresponding to the first source, media clips from a second source are sorted into a group corresponding to the second source, etc.

The process then arranges (at 515) each of the media clips for each source group according to temporal data. The temporal data according to which the media clips are arranged is time code information or file creation time information in some embodiments. The media clips for each source are placed in chronological order in accordance to the temporal data of each media clip. The process also arranges the media clips in a sequence with proper offset from each other. That is, the spacing between each media clip of the sequence represents an actual elapsed time between the stoppage of the recording of one clip and the start of the recording of a next clip. The process arranges the media clips in such a sequence for each of the several source groups.

Next, process 500 determines (at 520) time offset between sequences of different sources. The time offset represent the difference in actual elapsed time between the start of a first sequence and the start of a second sequence, the first sequence being from a first source group and the second sequence from a second source group. In other words, the time offset is applied to the sequence of the first source in order to align it with a sequence of the second source so that the media clips of each sequence are roughly synchronized in time. As described in further details below by reference to FIGS. 10 and 11, the process of determining the time offset is an iterative process. For example, the process compares media clips from a first source to media clips from a second source until a certain criteria for the comparison has been satisfied.

After determining the time offsets, process 500 aligns (at 525) the sequences of each of the source groups according to the determined time offsets. The process applies the respective time offsets to shift each sequence into alignment with one another. The alignment of the two sequences provides a hint as to how the media clips from each sequence align with one another. As described above, when two sequences are aligned, the media clips of each sequence are roughly synchronized in time.

Once the sequences are aligned according to time offsets, process 500 identifies (at 530) the media clips from each sequence that overlap with one another on a common timeline. Overlapping clips are identified by process 500 as clips from two separate sequences (i.e. two different media recording devices) that record the same event at the same time. For instance, a video camera and a microphone that concurrently record the same event will produce overlapping clips. After the overlapping clips are identified, process 500 fine-tunes the overlapping clips by making minor adjustments to the alignment of the clips. The process by which the clips are fine-tuned is described in more detail by reference to FIG. 14 below.

Next, the process merges (at 535) each set of overlapping clips that has been fine-tuned into a single multi-clip. In some embodiments, each multi-clip includes at least two clips, where each clip is from a different sequence. Additional clips of the multi-clip must also originate from different sequences. In other words, each clip of a multi-clip must be from a unique sequence (i.e. unique media recording device) since a single media recording device cannot record two different clips at the same time.

After the overlapping clips are merged into several multi-clips, the process arranges (at 540) the merged clips into a master series. The process then exits. The master series includes a series of multi-clips where each multi-clip represents two or more overlapping recordings that are synchronized with one another. In some embodiments, the multi-clips are arranged in chronological order separated by a user specified time duration. For example, a user specifies a five second pause in between each multi-clip. Accordingly, a five second pause is inserted between each multi-clip on the master series. Once the multi-clips are arranged, the master series of all the synchronized clips is available to be viewed by users.

V. Sorting Media Clips by Source

Categorization of media clips according to media recording sources in some embodiments is described by reference to FIGS. 6 and 7. FIG. 6 conceptually illustrates sorting media clips into respective source groups in some embodiments. As shown in FIG. 6, media clip 605 includes filename information 610 and recorder identifying data 615. As described above, the filename information includes alpha-numeric and symbol characters that are automatically created by a media recording device at the time of the recording. Furthermore, the recorder identifying data in some embodiments includes a serial number of the media recording device. In some embodiments, the recorder identifying data is input by the user. The recorder identifying data is provided on the media clip to uniquely identify the source of the particular clip. Identification of the source of a media clip expedites the audio synchronization process. For instance, two media clips recorded by the same media recording device cannot overlap with one another since a single media recording device cannot record two clips at the same time. Thus, sorting media clips according to source ensures that two media clips from the same source will not be compared and synchronized with one another.

FIG. 6 illustrates Clip 1 as including recorder identifying data that identifies the source of the clip as Camera 1/Mic 1. Accordingly, Clip 1 is sorted into the group designated as Camera 1/Mic 1. The remaining clips (e.g. Clips 2-11) are similarly sorted into their respective groups (Camera 1/Mic 1, Camera 2/Mic 2, and Mic 3) according to recorder identifying data. Specifically, Clips 1, 3, and 10 are placed into the Camera 1/Mic 1 group 620; Clips 2, 6, 8, and 9 are placed in the Camera 2/Mic 2 group 625; and Clips 4, 5, 7, and 11 are placed in the Mic 3 group 630. While FIG. 6 includes groups for three separate sources, some embodiments include groups for several additional sources.

FIG. 7 conceptually illustrates a process 700 for sorting the several media clips including video media clips and audio media clips. Process 700 first identifies (at 705) a next media clip to be analyzed. After a media clip is identified, the process determines (at 710) whether the identified media clip contains unique recorder identifying data. When recorder identifying data for the clip is unique, a new group is created (at 715) and the media clip is stored into the newly created group (at 720). However, when the recorder identifying data is not unique, process 700 simply stores (at 720) the media clip into the appropriate group that corresponds to the recorder identifying data.

The description continues in the full USPTO document.