DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is responsive to applicant’s remarks received on August 5, 2026. Claims 1-16 and 18-20 remain pending.
Response to Arguments
Applicant’s arguments filed on August 5, 2026 have been fully considered and are persuasive in part.
Regarding the rejection of claims 1-16 and 18-19 under 35 U.S.C. 112(b), Applicant argues that the terms “singing version” and “alignment” are reasonably understood in view of the Specification, and further notes that “feature similarity” is not recited in the presently pending claims. Upon reconsideration of the claims in light of the Applicant’s remarks and the Specification, Applicant’s arguments are persuasive. Accordingly, the rejection of claims 1-16 and 18-19 under 35 U.S.C. 112(b) is withdrawn.
Regarding the rejection of claims 1, 4, 12, 18 and 19 under 35 U.S.C. 103 over US’670 in view of CN’294, Applicant argues, in substance, that the cited references do not teach of suggest the newly amended limitation requiring that, after alignment, each of the vocal signal in the vocal set, including a vocal signal corresponding to the reference rhythm information, serves as a respective one of the to-be-mixed vocal audios. In particular, Applicant argues that US’670 does not retain both the target vocal and source vocal as respective vocal inputs to the final mix and that CN’294 uses reference audio for alignment/correction audio to be mixed into the resulting output.
Examiner understands Applicant’s argument and agrees that US’670 in view of CN’294, as previously applied, do not expressly teach the newly amended limitation requiring the vocal signal corresponding to the reference rhythm information itself to serve, together with the other aligned vocal signal(s), as a respective to-be-mixed vocal audio.
Accordingly, the rejection of claims 1, 4, 12, 18, and 19 over US’670 in view of CN’294, as set forth in the previous Office action, is withdrawn.
However, Applicant’s amendment necessitates further consideration of the prior art. As discussed in the new ground of rejection below, US20140140536 (Serlectic) teaches retaining an audio input track while manipulating at least one other audio input track based on musical properties of the retained track and thereafter combining the retained track and the manipulated track into a single output audio track (¶¶0356-0361). Serlectic further discloses adding harmonizing vocal tracks to a lead vocal track to produce a multiple-voice audio track (¶0369).
Therefore, while Applicant’s arguments are persuasive with respect to the prior rejection as previously presented, they do not render the amended claims allowable for the reasons set forth in the new rejection below.
Claim Objections
Claims 18 and 19 are objected to because of the following informalities:
In claim 18, the phrase “a respective one of to-be-mixed vocal audio” should be changed to –a respective one of to-be-mixed vocal audios—for grammatical consistency with the subsequent recitation of “the to-be-mixed vocal audios”.
In claim 19, the phrase “a respective one of to-be-mixed vocal” should be changed to --a respective one of to-be-mixed vocal audios—for grammatical consistency with the subsequent recitation of “the to-be-mixed vocal audios”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1,4,12, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over US20130170670 (Casey), hereinafter US’670, in view of CN112216294 (Zheng, provided by the IDS submitted on August 23, 2023), hereinafter CN’294, and in further view of US20140140536 (Serlectic), hereinafter US’536.
Regarding claim 1, US’670 discloses ‘A method for generating a remix (US’670, [Title]: identifies invention as a “System and method for Automatically Remixing Digital Music”; ¶[0019]: “systems and methods that create a transformed digital media, called an automatic matching-mashup”), comprising:
obtaining at least two audios which are different singing versions of a same song (US’670, ¶[0005]: “a method generates a transformed digital media from a target media and one or more source media”, target media and one or more source media correspond to at least two audio inputs, source media corresponds to alternate versions of the same content);
extracting, from each of the at least two audios, a vocal signal and an instrumental signal to obtain a vocal set and an instrumental set, wherein the vocal set comprises the vocal signal of each of the at least two audios, and the instrumental set comprises the instrumental signal of each of the at least two audios (US’670, ¶[0020]:” FIGS. 4-5) of isolating and extracting multiple separate audio components, which correspond approximately to perceptual sound objects ( e.g., a guitar notes, vocal notes, base drum, snare, cymbal, etc.), can be performed on one or more physical transport channels); ¶[0049]: " musical features are extracted from the Time-Frequency Distribution ", separating each input audio signal into multiple distinct sound components, including vocal-related and instrumental-related components);
and mixing the to-be-mixed vocal audios with the to-be-mixed instrumental audio to obtain the remix (US’670, ¶[0008]: “Portions of the source media associated with said matched source features are mixed with the target media to form an augmented target media”, corresponds to remixed transformed media).
US’670 does not expressly disclose ‘aligning tracks of all vocal signals in the vocal set through referring to reference rhythm information, wherein the reference rhythm information is selected from rhythm information of all vocal signals in the vocal set;
determining an instrumental signal, of which a track is aligned with the tracks of the to-be- mixed vocal audios, from the instrumental set as a to-be-mixed instrumental audio
However, CN’294 discloses ‘aligning tracks of all vocal signals in the vocal set through referring to reference rhythm information, wherein the reference rhythm information is selected from rhythm information of all vocal signals in the vocal set (CN’294’, ¶[0129]: " The starting time of the first audio signal and the starting time of the second audio signal are obtained, and the starting time of the first audio signal is adjusted to align with the starting time of the second audio signal so that the rhythm of the first audio signal and the second audio signal are aligned ", time-aligning one singing signal to another singing signal using timing / rhythm information);
determining an instrumental signal, of which a track is aligned with the tracks of the to-be- mixed vocal audios, from the instrumental set as a to-be-mixed instrumental audio (CN’294, ¶[0022]: "Obtain the start time of the first audio signal and the start time of the accompaniment audio signal, and adjust the start time of the first audio signal to align with the start time of the accompaniment audio signal so that the rhythm of the first audio signal and the accompaniment audio signal are aligned ", selection of signals that match the aligned timing framework);
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify US’670 according to CN’294 to align to the vocal audio signals using timing/rhythm information of another vocal signal as reference information, and to align the vocal audio signals with the accompaniment/instrumental audio signal prior to mixing, as taught by CN’294, in order to synchronize the respective audio tracks and improve the quality of the resulting mixed audio.
US’670 (in view of CN’294 & US’536) does not expressly disclose ‘and after the aligning, each of the vocal signals in the vocal set, comprising a vocal signal corresponding to the reference rhythm information, serves as a respective one of to-be-mixed vocal audios
However, US’536 discloses processing a plurality of audio input tracks while retaining one audio input track and manipulating another audio input track based upon musical properties of the retained track. Specifically, US’536 teaches that the audio input tracks may include protected content such as “a single vocal or instrumental track for a song” and describes a process for combining such a restricted audio input track with one or more other audio input tracks while preserving protected aspects of the restricted track (¶0356). US’536 further teaches that “at least one other audio input track of the plurality of audio input tracks based on musical properties of the restricted audio input track are manipulated” (¶0360), and thereafter “the restricted audio input track and the manipulated at least one other audio input track are combined into a single output audio track” (¶0361). Thus, the audio track serving as the basis for manipulating another audio track is itself retained and combined with the manipulated audio track in the resulting output.
It would have been obvious to one of ordinary skill in the prior to the effective filing of the claimed invention to further modify US’670 (in view of CN’294 & US’536) according to US’536 such that the reference vocal signal and the other aligned vocal signal(s) are included in the resulting audio, as US’536 teaches adding harmonizing vocal tracks to a lead vocal track in order to provide a “multiple voice audio track” while preserving the reference/lead vocal.
Regarding Claim 4, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 1, as discussed above.
US’670 (in view of CN’294 & US’536) further discloses ‘wherein the rhythm information is beat information, and aligning the tracks of all vocal signals in the vocal set through referring to the reference rhythm information (CN’294, ¶[0020]: "Using the pitch information of the second audio signal as a standard, the pitch line of the first audio signal is compressed and adjusted so that the pitch information of the first audio signal is time-aligned with the pitch information of the second audio signal."; ¶[0022]:"adjust the start time of the first audio signal to align with the singing start time of the accompaniment audio signal so that the rhythm of the first audio signal and the accompaniment audio signal are aligned"); comprises:
extracting a piece of beat information from each of the at least two audios to obtain a beat set, wherein the beat set comprises the piece of beat information of each of the at least two audios, and the piece of beat information and the vocal signal are mapped to each other for each of the at least two audio to obtain one-to-one correspondence (CN’294, ¶[0124]: "Canonical Time Warping (CTW) is an extension of Canonical Correlation Analysis (CCA) and Dynamic Time Warping (CTW) for spatiotemporal alignment of behavior between two objects. The DTW can calculate the similarity of two time sequences, is particularly suitable for the time sequences with different lengths and different rhythms, automatically distorts the time sequences…the shortest path of the two sequences can be obtained by a recursive algorithm”, ¶[0127]: “audio processing method…performing time-aligned signal correction processing on the pitch of the first audio signal and the pitch of the second audio signal”, obtaining rhythm/time-sequence information for audio signals and using that information in time alignment; corresponds to extracting beat/rhythm information from each audio and associating it with the corresponding vocal signal for alignment);
and in response to a quantity of elements in each piece of beat information being identical throughout the beat set, determining a first piece of beat information in the beat set as the reference rhythm information, for each of one or more second pieces of beat information, calculating a difference between the first piece of beat information and said second piece of beat information, wherein the one or more second pieces of beat information are in the beat set and other than the first piece of beat information (CN’294, ¶[0124]: "Canonical Time Warping (CTW)..and Dynamic Time Warping (CTW) for spatiotemporal alignment, ¶[0127]: “audio processing method…performing time-aligned signal correction processing”, comparing one time sequence against another and determining alignment differences between them through DTW/CTW-style processing);
and mapping the difference to a respective one of one or more second vocal signals based on the one-to-one correspondence to obtain first correspondence, wherein the one or more second vocal signals are in the vocal set and other than a first vocal signal, and the first vocal signal is in the vocal set and mapped to the first piece of beat information (CN’294, ¶[0124]: "Canonical Time Warping (CTW)..and Dynamic Time Warping (CTW) for spatiotemporal alignment, ¶[0127]: “audio processing method…performing time-aligned signal correction processing”, alignment process uses rhythm information from a given signal and applies the resulting difference information back to that corresponding signal for correction; mapping the calculated difference to the respective second vocal signal);
and for each of the one or more second vocal signals, determining a redundant end and a to-be-compensated end based on the corresponding difference, which is determined according to the first correspondence, for adjusting said second vocal signal (CN’294, ¶[0124]: "Canonical Time Warping (CTW)..and Dynamic Time Warping (CTW) for spatiotemporal alignment, ¶[0127]: “audio processing method…performing time-aligned signal correction processing”, automatically distorting or correcting time sequences to align signals with different rhythms; identifying portions of the target vocal that are effectively excess versus portions that require compensation in order to perform the adjustment);
and removing redundant data from the redundant end and adding zero-value data at the to-be- compensated end, wherein the redundant data and the zero-value data each has a data length equal to the difference (CN’294, ¶[0124]: "Canonical Time Warping (CTW)..and Dynamic Time Warping (CTW) for spatiotemporal alignment, ¶[0127]: “audio processing method…performing time-aligned signal correction processing”, signal-correction processing for time alignment based on differences between sequences, implementing the adjustment by trimming excess data and padding the opposite end by the measured amount).
Regarding Claim 12, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 1, as discussed above.
US’670 (in view of CN’294 & US’536) further discloses ‘before all vocal signals having the aligned tracks are determined to serve as the to-be-mixed vocal audios, the method further comprises: determining a vocal signal from the vocal signals having the aligned tracks as a standard vocal signal (CN’294, ¶[0162]: "the root mean square of the signal may be calculated first ..."; ¶[0163]"the overall envelope of the volume may be obtained by calculating a Root Mean Square (RMS) of the signal in frames ..."; "the volume gain of the audio effector is determined according to the difference between the first average volume value and the second average volume value ..."; ¶[0172]"the volume difference between the user and the original singer is calculated... and then the variable quantity is superimposed on the first audio signal to adjust the volume ...", selecting one aligned vocal as a reference and adjusting other aligned vocals based on RMS relationships reads directly and broadly on the claimed loudness-normalization step);
and adjusting loudness of each of one or more to-be-adjusted vocal signals based on B = vocalXx(RMSO/RMSX) (CN’294, ¶[0162]: "the root mean square of the signal may be calculated first ..."; ¶[0163]"the overall envelope of the volume may be obtained by calculating a Root Mean Square (RMS) of the signal in frames ..."; "the volume gain of the audio effector is determined according to the difference between the first average volume value and the second average volume value ..."; ¶[0172]"the volume difference between the user and the original singer is calculated... and then the variable quantity is superimposed on the first audio signal to adjust the volume ...", selecting one aligned vocal as a reference and adjusting other aligned vocals based on RMS relationships reads directly and broadly on the claimed loudness-normalization step), wherein:
the one or more to-be adjusted vocal signals are among the vocal signals having the tracks aligned and other than the standard vocal signal (CN’294, ¶[0162]: "the root mean square of the signal may be calculated first ..."; ¶[0163]"the overall envelope of the volume may be obtained by calculating a Root Mean Square (RMS) of the signal in frames ..."; "the volume gain of the audio effector is determined according to the difference between the first average volume value and the second average volume value ..."; ¶[0172]"the volume difference between the user and the original singer is calculated... and then the variable quantity is superimposed on the first audio signal to adjust the volume ...", selecting one aligned vocal as a reference and adjusting other aligned vocals based on RMS relationships reads directly and broadly on the claimed loudness-normalization step);
B is said to-be-adjusted vocal signal after the adjusting, vocalX is said to-be-adjusted vocal signal before the adjusting, RMSO is a root mean square of the standard vocal signal, and RMSX is a root mean square of vocalX (CN’294, ¶[0162]: "the root mean square of the signal may be calculated first ..."; ¶[0163]"the overall envelope of the volume may be obtained by calculating a Root Mean Square (RMS) of the signal in frames ..."; "the volume gain of the audio effector is determined according to the difference between the first average volume value and the second average volume value ..."; ¶[0172]"the volume difference between the user and the original singer is calculated... and then the variable quantity is superimposed on the first audio signal to adjust the volume ...", selecting one aligned vocal as a reference and adjusting other aligned vocals based on RMS relationships reads directly and broadly on the claimed loudness-normalization step).
Regarding Claim 18; Claim 18 contains substantially the same subject matter as claim 1. Therefore, Claim 1 is rejected on the same grounds as claim 1. However, Claim 19 further discloses an apparatus for generating a remix, electronic device comprising a processor; and a memory storing a computer program, wherein the computer program when loaded and executed by the processor. US’670 discloses ‘An apparatus for generating a remix, electronic device (US’670, ¶[0024]:“System 100 is implemented on a computing device, such as a mobile phone, a portable digital music player, and a desk top computer”), comprising a processor (US’670, ¶[0024]:“ matshup processor 30"); and a memory storing a computer program (US’670, ¶[0024]:"includes a media store 10 for storing digital media, a preprocessor 20"), wherein the computer program when loaded and executed by the processor (US’670, ¶[0024]:"for extracting musical features from digital media, and a matshup processor 30…Media store 10, pre-processor 20 and matshup processor 30 cooperate to produce transformed digital media 40.”) performs:
obtaining at least two audios which are different singing versions of a same song; (corresponds to claim 1)
extracting, from each of the at least two audios, a vocal signal and an instrumental signal to obtain a vocal set and an instrumental set, wherein the vocal set comprises the vocal signal of each of the at least two audios, and the instrumental set comprises the instrumental signal of each of the at least two audios; (corresponds to claim 1)
aligning tracks of all vocal signals in the vocal set through referring to reference rhythm information, wherein the reference rhythm information is selected from rhythm information of all vocal signals in the vocal set, and after the aligning, each of the vocal signals in the vocal set, comprising a vocal signal corresponding to the reference rhythm information, serves as a respective one of to-be-mixed vocal audio; (corresponds to claim 1)
determining an instrumental signal, of which a track is aligned with the tracks of the to-be- mixed vocal audios, from the instrumental set as a to-be-mixed instrumental audio; (corresponds to claim 1)
and mixing the to-be-mixed vocal audios with the to-be-mixed instrumental audio to obtain the remix. (corresponds to claim 1)
Regarding Claim 19; Claim 19 contains substantially the same subject matter as claim 1. Therefore, Claim 19 is rejected on the same grounds as claim 1. However, Claim 19 further discloses a storage medium, storing computer-executable instructions, wherein the computer- executable instructions when loaded and executed by a processor performs a method. US’670 discloses ‘A storage medium, storing computer-executable instructions (US’670, ¶[0024]: “…a computing device, such as a mobile phone, a portable digital music player, and a desk top computer”), wherein: the computer-executable instructions when loaded and executed by a processor (US’670, ¶[0024]:"for extracting musical features from digital media, and a matshup processor 30…Media store 10, pre-processor 20 and matshup processor 30 cooperate to produce transformed digital media 40.”) performs:
obtaining at least two audios which are different singing versions of a same song; extracting, from each of the at least two audios, a vocal signal and an instrumental signal to obtain a vocal set and an instrumental set, wherein the vocal set comprises the vocal signal of each of the at least two audios, and the instrumental set comprises the instrumental signal of each of the at least two audios; (corresponds to claim 1)
aligning tracks of all vocal signals in the vocal set through referring to reference rhythm information, wherein the reference rhythm information is selected from rhythm information of all vocal signals in the vocal set, and after the aligning, each of the vocal signals in the vocal set, comprising a vocal signal corresponding to the reference rhythm information, serves as a respective one of to-be-mixed vocal; (corresponds to claim 1)
determining an instrumental signal, of which a track is aligned with the tracks of the to-be- mixed vocal audios, from the instrumental set as a to-be-mixed instrumental audio; (corresponds to claim 1)
and mixing the to-be-mixed vocal audios with the to-be-mixed instrumental audio to obtain the remix. (corresponds to claim 1)
It would have obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the audio processing/remixing method of US’670 to perform the alignment taught by CN’294 using rhythm/timing information of one of the audio signals as reference information for aligning the other audio signal(s), in order to temporally synchronize the audio signals prior to combination and thereby provide a coherent combined audio output.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over US’670 (in view of CN’294 & US’536), and in further view of and US20100313736 (Lenz), hereinafter US’736 and US20090060207 (Barry), hereinafter US’207.
Regarding Claim 2, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 1, as discussed above.
US’670 (in view of CN’294 & US’536) further discloses ‘wherein extracting the vocal signal from each of the at least two audios comprises: for each of the at least two audios (US’670,¶[0020]:” FIGS. 4-5) of isolating and extracting multiple separate audio components, which correspond approximately to perceptual sound objects ( e.g., a guitar notes, vocal notes, base drum, snare, cymbal, etc.), can be performed on one or more physical transport channels); ¶[0049]: " musical features are extracted from the Time-Frequency Distribution ", separating each input audio signal into multiple distinct sound components, including vocal-related and instrumental-related components),
CN’294 further discloses ‘and extract the vocal signal from the median signal (CN’294, ¶[0156]: "STFT is performed on the first audio signal ... blank parts are removed ... ¶[0156]the spectrogram of each frame is calculated, the average value of the center frequency amplitude of each frame signal is extracted ..." US’670, ¶[0035]: "audio conversion module must detect each new played (or sung) musical note from within the audio stream, and identify its pitch and volume, and the times of its beginning and end", The cited passages start from analyzed frame/spectral data and extract vocal-related information from the analyzed audio stream. That is the cited basis for deriving the vocal signal from the previously obtained representative signal; frame-level spectral extraction, pitch/volume extraction, and removal of blank parts read broadly on extracting a vocal signal from analyzed audio, including channel- or mixture-based vocal-isolation implementations; these references corresponds to frame-/spectral-analysis of a vocal signal and extraction of note-related information from sung audio and signal-extraction processing for deriving vocal content from source audio),
and average the vocals in amplitude or spectral features to obtain the vocal signal (CN’294, ¶[0156]: "STFT is performed on the first audio signal ... blank parts are removed ... ¶[0156]the spectrogram of each frame is calculated, the average value of the central frequency amplitude of each frame signal is extracted ..." (CN’294, ¶[0153]" the average value of the center frequency amplitude of each frame signal in the intermediate audio signal is extracted to obtain the timbre feature vector of the intermediate audio signal.", extracting an average spectral-amplitude value for frame signals; average value of the central-frequency amplitude of each frame signal is extracted; averaging amplitude or spectral features to obtain the vocal signal; frame-level spectral extraction).
US’670 (in view of CN’294 & US’536) does not expressly disclose ‘calculating a median signal of said audio,
or extracting vocals from a left channel and a right channel, respectively, of said audio.
However, US’736 discloses ‘calculating a median signal of said audio (US’736, [0035]: "audio conversion module must detect each new played (or sung) musical note from within the audio stream, and identify its pitch and volume, and the times of its beginning and end", frame/spectral analysis of singing audio; calculating an intermediate representative signal of the audio and extracting the vocal signal from that analyzed signal),
However, US’207 discloses ‘or extracting vocals from a left channel and a right channel, respectively, of said audio, (US’207, ¶[0012]:” "the stereo recording comprises a first channel signal and a second channel signal"; "the azimuth of the desired source is computed...") audio (US’207, Abstract and ¶[0012]: "the stereo recording comprises a first channel signal and a second channel signal ( e.g. LEFT and RIGHT stereo signals)"; "the azimuth of the desired source is computed..."; "Methods of sound source separation in which individual sources are extracted from a multiple source recording... analyzing stereo recordings to facilitate separation of individual musical sound sources from stereo music recordings", stereo first/second channel signals; treating separate portions of an audio input as sources from which vocals can be extracted).
It would have been obvious to incorporate median-based audio-model computation as one routine way to derive a vocal component during source separation of US’736 and to operate on the left and right channels separately isolate sources channel selection of US’207 with the modified mixing method of US’670 (in view of CN’294 & US’536) to enable independent processing of audio components and improve flexibility of remix generation.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over US’670 (in view of CN’294 & US’536), and in further view of and US’207.
Regarding Claim 3, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 1, as discussed above.
and determining the instrumental as the instrumental signal (CN’294, ¶[0036]: "the accompaniment audio signal is the accompaniment audio signal when the user sings a song ... ¶[0091] The first audio signal may be a pure human voice signal emitted by the user without music…", accompaniment content is distinguished from vocals, the separated accompaniment/source can be retained as the instrumental signal).
US’670 (in view of CN’294 & US’536) does not expressly disclose ‘wherein extracting the instrumental signal from each of the at least two audios comprises: for each of the at least two audios, extracting an instrumental from a left channel or a right channel of said audio.
However, US’207 discloses ‘wherein extracting the instrumental signal from each of the at least two audios comprises: for each of the at least two audios, extracting an instrumental from a left channel or a right channel of said audio (US’207, Abstract and ¶[0012]: "the stereo recording comprises a first channel signal and a second channel signal ( e.g. LEFT and RIGHT stereo signals)"; "the azimuth of the desired source is computed..."; "Methods of sound source separation in which individual sources are extracted from a multiple source recording... analyzing stereo recordings to facilitate separation of individual musical sound sources from stereo music recordings", stereo first/second channel signals; teaches first/second channel processing and source extraction from stereo recordings),
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate isolating/retaining (channel selection) technique of US’207 of an accompaniment or instrumental component in the modified multi-track system of the modified mixing method of US’670 (in view of CN’294 & US’536) so it can later be selected as the instrumental portion of the remix, allow finer control over remix generation.
Claims 5-10, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over US’670 (in view of CN’294 & US’536), and in further view of and US 20140133675 (King), hereinafter US’675.
Regarding Claim 5, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 4, as discussed above.
US’670 (in view of CN’294 & US’536) further discloses ‘wherein calculating the difference between the first piece of beat information and said second piece of beat information (US’670 ¶[0008]: “the mixing is based upon a probabilistic mixing algorithm that uses a distance between the matched target feature and source feature to define an amplitude of each said portion... ¶[0038]: a distance function is a normed Euclidean distance function”, ) comprises:
a length of a data frame serving as a data unit (US’670, ¶[0010]: "A segmenter of the system segments the target features into a plurality of temporal target segments..."; ¶[0050]:"segmenting the target features into temporal target segments...", variables M, BeatO, BeatX, numBeats, and L as the difference, reference timing representation, target timing representation, number of beats/elements, and data-frame length).
US’670 (in view of CN’294 & US’536) does not expressly disclose ‘calculating the difference between the first piece of beat information and said second piece of beat information based on M = [sum(BeatO-BeatX)/numBeats]xL, wherein:
M is the difference, BeatO is a vector representation of the first piece of beat information, BeatX is a vector representation of said second piece of beat information, sum (BeatO-BeatX) represents calculating a sum of all elements in a vector obtained by subtracting BeatX from BeatO, numBeats is a quantity of elements in each piece of beat information, and L represents a length of a data frame serving as a data unit
However, US’675 discloses ‘calculating the difference between the first piece of beat information and said second piece of beat information (US’675, ¶[0044]: “A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data”, calculating a stretch value from a reference interval and a target interval) based on M = [sum(BeatO-BeatX)/numBeats]xL(US’675, ¶[0044]: “A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data”, calculating a stretch value from a reference interval and a target interval), wherein:
M is the difference, BeatO is a vector representation of the first piece of beat information, BeatX is a vector representation of said second piece of beat information, sum (BeatO-BeatX) represents calculating a sum of all elements in a vector obtained by subtracting BeatX from BeatO, numBeats is a quantity of elements in each piece of beat information, and L represents a length of a data frame serving as a data unit (US’675, ¶[0044] ("A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data”, calculated difference/stretch value, reference/target interval representations, and temporal data units/segments).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate performing the alignment difference with a formula using the known interval-difference of US’675 with frame-length parameters of the modified mixing method of US’670 (in view of CN’294 & US’536) preventing incorrect alignment and which sound data having a higher quality may be synchronized with sound data having a lower quality to replace the lower quality sound data.
Regarding Claim 6, US’670 (in view of CN’294 and US’675) discloses ‘The method according to claim 4, as discussed above.
US’670 (in view of CN’294 and US’675) further discloses ‘further comprising: in response to a quantity of the at least two audios being two and the quantity of elements in each piece of beat information being different in the beat set, determining a third piece of beat information as the reference rhythm information (CN’294, ¶[0020]: "Using the pitch information of the second audio signal as a standard, the pitch line of the first audio signal is compressed and adjusted so that the pitch information of the first audio signal is time-aligned with the pitch information of the second audio signal."; ¶[0022]:"adjust the start time of the first audio signal to align with the singing start time of the accompaniment audio signal so that the rhythm of the first audio signal and the accompaniment audio signal are aligned", selecting rhythm information for alignment), wherein the third piece of beat information has minimum elements among the beat set; (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", selecting the reference timing sequence corresponds to determining a third piece of beat information as the reference rhythm information when two audio differ in beat element);
reducing the quantity of elements in a fourth piece of beat information to be identical to the quantity of elements in the third piece of beat information, wherein the fourth piece of beat information is in the beat set and other than the third piece of beat information (US’675, ¶[0044]::“A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data”, stretching/compressing a target interval to match the reference interval);
determining groups of adjacent beats based on the third piece of beat information or the fourth piece of beat information after the reducing (CN’294, ¶[0136]:"beat estimation can be divided into three stages ... determine the rhythm of the accompaniment audio signal based on the time difference between the peaks", beat estimation and rhythm determination from peak/time-difference information);
and for each of the groups of adjacent beats, dividing the third vocal signal and the fourth vocal signal based on said group of adjacent beats to obtain a first data segment and a second data segment corresponding to said group of adjacent beats (US’670, ¶[0010]: "A segmenter of the system segments the target features into a plurality of temporal target segments..."; ¶[0050]:"segmenting the target features into temporal target segments...", dividing audio into temporal segments), wherein the third vocal signal is in the vocal set and mapped to the third piece of beat information, and the fourth vocal signal is in the vocal set and other than the third vocal signal); (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", sound data and time intervals between signals; dividing the third and fourth vocal signals based on each beat group to obtain corresponding first and second data segments applies to segmentation into aligned intervals; third vocal signal being mapped to the third/reference beat information applies to signal portion of the alignment );
and adjusting a data length of the first data segment and a data length of the second data segment to be equal (US’675, ¶[0044]: "the stretch value may be calculated to describe an amount of time a time interval is to be stretched or compressed... to match", adjusting the amount/length of one interval relative to another to achieve equality/alignment; stretching/compressing one interval relative to the other).
Regarding Claim 7, US’670 (in view of CN’294 and US’675) discloses ‘The method according to claim 6, as discussed above.
US’670 (in view of CN’294 and US’675) further discloses ‘wherein adjusting the data length of the first data segment and the data length of the second data segment to be equal (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", comparing intervals and stretching/compressing one interval relative to the other) comprises:
in response to the quantity of data frames in the first data segment being not equal to the quantity of data frames in the second data segment, determining a data segment having most data frames among the first data segment and the second data segment as an shortening target (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", choosing the longer interval for shortening);
calculating a shortened length for each data frame in the shortening target (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", calculating a per-frame amount of shortening from the interval difference);
and shortening each data frame in the shortening target based on the shortened length (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", compressing the target interval to match the reference interval).
Regarding Claim 8, US’670 (in view of CN’294 and US’675) discloses ‘The method according to claim 7, as discussed above.
US’670 (in view of CN’294 and US’675) further discloses ‘wherein the calculating the shortened length for each data frame in the shortening target comprises: calculating the shortened length based on P = [(m-n)xL]/m, wherein: P is the shortened length, m is a maximum between the quantity of data frames (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", calculated interval-difference adjustment in terms of the relevant frame counts and data-unit length corresponds to calculating the shortened length P from m, n, and frame length L; sequence-alignment process) in the first data segment and the quantity of data frames in the second data segment, n represent a minimum between the quantity of data frames in the first data segment and the quantity of data frames in the second data segment (US’670, ¶[0010]: "A segmenter of the system segments the target features into a plurality of temporal target segments..."; ¶[0050]:"segmenting the target features into temporal target segments...", temporal data frames/segments),
Regarding Claim 9, US’670 (in view of CN’294 and US’675) discloses ‘The method according to claim 6, as discussed above.
US’670 (in view of CN’294 and US’675) further discloses ‘wherein reducing the quantity of elements in the fourth piece of beat information to be identical to the quantity of elements in the third piece of beat information comprises: sorting elements in the third piece of beat information based on magnitude of timestamps to obtain a target sequence (CN’294, ¶[0131]:“the start time of the first audio signal is detected ... the start time of the accompaniment audio signal is obtained ... ¶[0136]: beat estimation can be divided into three stages ... determine the rhythm of the accompaniment audio signal based on the time difference between the peaks”, beat/rhythm timestamps and ordering by time differences; ordering rhythm events in time corresponds to sorting elements in beat information based on timestamp magnitude to obtain a target sequence);
determining a sequential number of a current iteration (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", progressing through the ordered alignment points corresponds to determining a sequential number of the current iteration);
determining an element, of which a sequential number in the target sequence is equal to the sequential number of the current iteration, in the third piece of beat information as a target element (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", selecting the relevant target interval/alignment point corresponding to the current alignment step corresponds to determining the target element in the third beat information according to the current sequence number);
calculating a distance between a timestamp of the target element and a timestamp of each of one or more comparison elements (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", calculating timing difference/stretch between reference and target intervals) ,
wherein each of the one or more comparison elements is in the fourth piece of beat information and has not been matched to any element in the target sequence (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; one-to-one correspondence between reference and target alignment points)
determining a comparison element corresponding to the minimum distance among the one or more comparison elements to match the target element (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data");
and in response to the sequential number of the current iteration being not less than a maximum quantity of iterations, deleting, from the fourth piece of beat information, the one or more comparison elements that remain (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", once the correspondence process reaches the final iteration, unmatched target intervals remain outside the selected correspondences),
and retaining, in the fourth piece of beat information, the one or more comparison elements that have been matched to any element in the target sequence (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", preserves the intervals/alignment points that have been matched to the reference).
Regarding Claim 10, US’670 (in view of CN’294 and US’675) discloses ‘The method according to claim 9, as discussed above.
US’670 (in view of CN’294 and US’675) further discloses ‘further comprising: in response to the sequential number of the current iteration being less than the maximum quantity of iterations, repeating: updating the sequential number of the current iteration by an increment of one (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data"; repeating correspondence across ordered alignment points; advancing to the next alignment step corresponds to updating the sequential number by an increment of one);
and determining the element, of which the sequential number in the target sequence is equal to the updated sequential number of the current iteration, in the third piece of beat information as a new target element (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", selecting the next reference timing point/alignment point for matching corresponds to determining the new target element with the updated sequence number);
calculating the distance between the timestamp of the new target element and the timestamp of each of the one or more comparison elements (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", repeatedly calculating interval difference for reference and target intervals corresponds to calculating the distance between the new target element and the comparison elements);
determining the comparison element corresponding to the minimum distance among the one or more comparison elements to match the target element (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", choosing the target interval corresponding by the relevant calculated difference);
until the sequential number of the current iteration is less than the maximum quantity of iterations (US’675, ¶[0043]: “a first time interval in a first representation of sound data... corresponds to a second time interval in a second representation of sound data"; (¶[0033]: "alignment point 402... corresponds to alignment point 402′..."; (¶[0044]: "the stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", repeats the alignment process until the relevant alignment points/intervals have been handled; iterative completion of the matching process corresponds to repeating until the current-iteration number reaches the maximum quantity of iterations).
Regarding Claim 16, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 1, as discussed above.
US’670 (in view of CN’294 and US’670) further discloses ‘from the instrumental set as the to-be-mixed instrumental audio (CN’294, ¶[0044]: " … so that the first audio signal and the accompaniment audio signal are rhythmically aligned", ¶[0144]: "the content analysis of the accompaniment audio signal includes at least one of style analysis and rhythm analysis. ¶[0147]: at least one of equalization adjustment ,..and reverberation increase is performed ... according to ... rhythm feature information of the accompaniment audio signal. ... ¶[0148]: the intermediate audio signal and the accompaniment audio signal ... are mixed", corresponds to aligning accompaniment/reference audio by rhythm information and then using accompaniment rhythm characteristics in the mixed result; selecting or aligning an instrumental signal to the reference rhythm information before using it in the remix),
or aligning a track of an instrumental signal in the instrumental set with the reference rhythm information (CN’294, ¶[0044]: " the singing start time of the first audio signal and the singing start time of the accompaniment audio signal, and adjust the start time of the first audio signal to align with the singing start time of the accompaniment audio signal, so that the first audio signal and the accompaniment audio signal are rhythmically aligned", ¶[0144]: "the content analysis of the accompaniment audio signal includes at least one of style analysis and rhythm analysis. Accompaniment features can be style features or rhythm features ... ¶[0147]: at least one of equalization adjustment, dynamic compression, volume adjustment, and reverberation increase is performed ... according to ... rhythm feature information of the accompaniment audio signal. ... ¶[0148]: the intermediate audio signal and the accompaniment audio signal ... are mixed", corresponds to aligning accompaniment/reference audio by rhythm information and then using accompaniment rhythm characteristics in the mixed result; selecting or aligning an instrumental signal to the reference rhythm information before using it in the remix),
and determining instrumental signal (CN’294, ¶[0036]: "the accompaniment audio signal is the accompaniment audio signal when the user sings a song ... ¶[0091] The first audio signal may be a pure human voice signal emitted by the user without music…"¶[0022]:"adjust the start time of the first audio signal to align with the singing start time of the accompaniment audio signal so that the rhythm of the first audio signal and the accompaniment audio signal are aligned", accompaniment/instrumental audio used with the vocal signal and rhythm alignment).
US’670 (in view of US’294) does not expressly disclose ‘wherein determining the instrumental signal, of which the track is aligned with the tracks of the to-be-mixed vocal audios, from the instrumental set as the to-be-mixed instrumental audio comprises: determining an instrumental signal, of which the track is aligned with the reference rhythm information’; ‘,having the aligned track as the to-be-mixed instrumental audio’.
However, US’675 discloses ‘wherein determining the instrumental signal, of which the track is aligned with the tracks of the to-be-mixed vocal audios, from the instrumental set as the to-be-mixed instrumental audio comprises: determining an instrumental signal, of which the track is aligned with the reference rhythm information (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", aligning a target interval to a reference interval),
‘having the aligned track as the to-be-mixed instrumental audio (US’675, ¶[0044]: "A stretch value is calculated based on an amount of time represented in the first and second time intervals... The stretch value may be used to align a time interval in the overdub sound data to a time interval in the reference sound data", aligning a target interval to a reference interval).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate aligning the instrumental track of the modified mixing method of US’670 (in view of CN’294 & US’536) to the time interval alignment technique used for the vocals of US’675 before remixing achieving a cohesive and balanced sound in a mix.
Claims 11 are rejected under 35 U.S.C. 103 as being unpatentable over US’670 (in view of CN’294 & US’536), and in further view of and US’736.
Regarding Claim 11, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 1, as discussed above.
US’670 (in view of CN’294 and US’670) further discloses ‘wherein the rhythm information is a beats per minute (BPM), and aligning the tracks of all vocal signals in the vocal set through referring to the reference rhythm information (CN’294 ¶[0020]: "Using the pitch information of the second audio signal as a standard, the pitch line of the first audio signal is compressed and adjusted so that the pitch information of the first audio signal is time-aligned with the pitch information of the second audio signal."; ¶[0022]:"adjust the start time of the first audio signal to align with the singing start time of the accompaniment audio signal so that the rhythm of the first audio signal and the accompaniment audio signal are aligned", time/pitch alignment of one vocal signal to another signal of the same song) comprises:
for each of the one or more target BPMs, mapping the ratio to a respective one of one or more target vocal signals based on the one-to-one correspondence to obtain second correspondence, wherein the one or more target vocal signals are in the vocal set other than a reference vocal signal, and the reference vocal signal is in the vocal set is mapped to the reference BPM (CN’294 ¶[0020]: "Using the pitch information of the second audio signal as a standard, the pitch line of the first audio signal is compressed and adjusted so that the pitch information of the first audio signal is time-aligned with the pitch information of the second audio signal", applying the time-alignment relationship back to the corresponding first audio signal relative to the second audio signal);
US’670 (in view of CN’294 & US’536) does not expressly disclose ‘determine the BPM of each of the at least two audios to obtain a BPM set, wherein the BPM set comprises the BPM of each of the at least two audios, and the BPM in the BPM set is mapped to the vocal signal in the vocal set for each of the at least two audios to obtain one-to-one correspondence;
determining a BPM from the BPM set as a reference BPM, wherein the reference BPM serves as the reference rhythm;
calculating a ratio of the reference BPM to each of one or more target BPMs, wherein the one or more target BPMs are in the BPM set and other than the reference.
However, US’736 discloses ‘determine the BPM of each of the at least two audios to obtain a BPM set, wherein the BPM set comprises the BPM of each of the at least two audios, and the BPM in the BPM set is mapped to the vocal signal in the vocal set for each of the at least two audios to obtain one-to-one correspondence (US’736, ¶[0032]: "the MIDI reference input 303 data is traversed… at a fixed speed or tempo";¶[0037]: "when changes in tempo are indicated in a MIDI input, the changes are passed..."; ";¶[0040]: "the smoothed time lag is used to calculate the new tempo", tempo determination and tempo-change handling);
determining a BPM from the BPM set as a reference BPM, wherein the reference BPM serves as the reference rhythm information (US’736, ¶[0040]: “the musical distance (in beats) is determined by identifying which note in the score was played and calculating the elapsed beat count, according to the score, from the last played note to the just-played note”, tempo determination and new-tempo calculation using a reference input at a fixed speed/tempo);
calculating a ratio of the reference BPM to each of one or more target BPMs, wherein the one or more target BPMs are in the BPM set and other than the reference BPM (US’736, ¶[0040]: “the musical distance (in beats) is determined by identifying which note in the score was played and calculating the elapsed beat count, according to the score, from the last played note to the just-played note”, s determining tempo and calculating a new tempo, and uses a reference input traversed at a fixed speed/tempo);
and for each of the one or more target vocal signals, altering a tempo of said target vocal signal based on the ratio, which is determined according to the second correspondence, while maintaining a pitch of said target vocal signal (US’736, ¶[0040]: “the musical distance (in beats) is determined by identifying which note in the score was played and calculating the elapsed beat count, according to the score, from the last played note to the just-played note”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the use of BPM/tempo ratios to time-adjust different singing versions of US’736 while preserving pitch content to the modified mixing method of US’670 (in view of CN’294 & US’536), because tempo alignment without undesired pitch distortion in synchronizing vocal tracks it ensures that the vocal performance remains cohesive and natural.
Claims 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over US’670 (in view of CN’294 & US’536), and in further view of US’207.
Regarding Claim 13, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 1, as discussed above.
US’670 further discloses ‘wherein mixing the to-be-mixed vocal audios with the to-be-mixed instrumental audio to obtain the remix (US’670 (corresponds to the final mixing of multiple performances/recording) comprises:
and mixing the stereo signal of each vocal signal in the to-be-mixed vocal audios with the instrumental audio to obtain the remix (US’670, ¶[0008]: “Portions of the source media associated with said matched source features are mixed with the target media to form an augmented target media”, corresponds to remixed transformed media).
US’670 (in view of CN’294 & US’536) does not expressly disclose ‘calculating a gain for a left channel and a gain for a right channel;
determining a stereo signal of each vocal signals in the to-be-mixed vocal audios based on the gain for the left channel and the gain for the right channel’.
However, US’207 disclose ‘calculating a gain for a left channel and a gain for a right channel, (US’207; ¶[0012]: "the stereo recording comprises a first channel signal and a second channel signal"; "the azimuth of the desired source is computed...", stereo first/second channel signals);
determining a stereo signal of each vocal signals in the to-be-mixed vocal audios based on the gain for the left channel and the gain for the right channel (US’207, ¶[0056]: “the portion of the inverted frequency-azimuth plane corresponding to the desired source is re-synthesised to form an output stereo source audio signal"; " The azimuth index, d, (where 0=d=~) may be defined as the position (between Left and Right) from which the source will be extracted”, first and second channel signals in a stereo recording, source azimuth computation from those channels, and re-synthesis of an output stereo source signal).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate using channel/gain allocation of US’207 during the final mixdown of the multiple performances of the modified mixing method of US’670 (in view of CN’294 & US’536) because gain staging and panning choices are routine audio-mixing implementations.
Regarding Claim 14, US’670 (in view of CN’294 and US’207) discloses ‘The method according to claim 13, as discussed above.
US’670 (in view of CN’294 and US’207) further discloses ‘wherein mixing the stereo signal of each vocal signal in the to-be-mixed vocal audios with the instrumental audio to obtain the remix (US’670, [0020]:“multiple separate audio components, which correspond … ( e.g., a guitar notes, vocal notes, … performed on one or more physical transport channels, i.e. a mono, stereo or multi-channel mixture of sound sources. The isolated and extracted audio components can then be combined”, mixing the stereo signals of the vocals with the instrumental) comprises:
mix the stereo signal of each vocal signal in the to-be-mixed vocal audios with the instrumental audio (US’670, ¶[0009]: “First audio portions of the target media corresponding to the target feature segment and second audio portions of the source media corresponding to the source segments of the N closest matches are mixed”, corresponds the remix output) based on SongComb = alphax(vocal1+...+vocalN)+(1-alpha) xsurround, to obtain the remix, (US’670, ¶[0007]: “a mixing coefficient based upon the generated probabilistic distribution is generated and the time windowed portions associated with the N closest matches are mixed with the associated time segmented portion using the associated mixing coefficients.”, a linear channel-weighted audio combination to obtain the remix), wherein:
SongComb is the remix (US’670, ¶[0008]: “Portions of the source media associated with said matched source features are mixed with the target media to form an augmented target media”, corresponds the remix output), vocall to vocalN each is the stereo signal, alpha is a preset adjustment factor, and surround is the instrumental audio (US’207, ¶[0056]: “the portion of the inverted frequency-azimuth plane corresponding to the desired source is re-synthesised to form an output stereo source audio signal"; " The azimuth index, d, (where 0=d=~) may be defined as the position (between Left and Right) from which the source will be extracted”, the remix output, stereo vocal signals, and instrumental/source audio used in the weighted mix).
Regarding Claim 15, US’670 (in view of CN’294 and US’207) discloses ‘The method according to claim 13, as discussed above.
US’670 (in view of CN’294 and US’670) further discloses ‘wherein calculating the gain for the left channel and the gain for the right channel (US’207; ¶[0012]: "the stereo recording comprises a first channel signal and a second channel signal (e.g. LEFT and RIGHT stereo signals) …converting the first channel signal into the frequency domain, converting the second channel signal into the frequency domain, defining a set of scaling factors, and producing a frequency azimuth plane”, stereo first/second channel signals, source azimuth computation, and output stereo source audio generation) comprises:
calculating the gain for the left channel and the gain for the right channel based on a preset angle in a stereo field and an angle of the vocal signal in the stereo field (US’207 , ¶[0056]: "the portion of the inverted frequency-azimuth plane corresponding to the desired source is re-synthesised to form an output stereo source audio signal"..the azimuth of each source signal is computed between Left and Right channel signals, source azimuth computation, and output stereo source audio generation corresponds to calculating the left/right channel gains based on a stereo-field angle and the angle of the vocal signal in the stereo field),
or calculating the gain for the left channel and the gain for the right channel by allocating linear gains (refer to above citation, forming the output stereo source signal from the first and second channel signals).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over US’670 (in view of CN’294 & US’536), and in further view of US20210201863 (Bosch Vicente), hereinafter US’863.
Regarding claim 20, US’670 (in view of CN’294 & US’536) discloses ‘The method according to claim 1, as discussed above.
US’670 (in view of CN’294 & US’536) further discloses ‘wherein, during the aligning, a track of a first vocal signal extracted from a first audio of the at least two audios remains unchanged (US’536, ¶[0356]: protecting restricted track from alteration and identifies the restricted track as potentially “a single vocal … track for a song”; ¶[0358]:a vocal track may have restrictions preventing alteration and that an audio input track may allow “no manipulation of speed, pitch, key or chord”;¶¶[0360]-[0361]).
‘tracks of vocal signals extracted from one or more other audios of the at least two audios are aligned with the track of the first vocal signal, and an instrumental signal extracted from the first audio is determined as the to-be-mixed instrumental audio
tracks of vocal signals extracted from one or more other audios of the at least two audios are aligned with the track of the first vocal signal (US’863, ¶[0145]: ”Alignment in step 304 of procedure 300 involves properly aligning the candidate (e.g., vocal) segment 124 with the segment 122 under consideration from the query track 112…”; [0150]: aligning the vocal starting boundary with the instrumental insertion point such that “the beats will automatically also be aligned”), and an instrumental signal extracted from the first audio is determined as the to-be-mixed instrumental audio (US’863, ¶[0079]:”query track 112 includes separated vocal component/track 116 and separated instrumental component/track 118”; [0004]:”the instrumentals of one song as the foundation, and then the vocals are inserted into the instrumentals one short segment at a time. Any number of the vocal segments can be inserted into the instrumentals…”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to further modify US’670 (in view of CN’294 & US’536) according to US’863 such that the first audio provides separated vocal and instrumental component used as the foundation for the mashup, in order to properly align the vocal and instrumental segments so that the mixed segments sound good together.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US9892758 teaches temporally aligning a cover audio recording with a corresponding track of an original multitrack composition using time-varying audio characteristic/correlation and then substituting the aligned recording into the composition.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICOLE K GILLESPIE whose telephone number is (571)482-4187. The examiner can normally be reached Monday-Friday 7:30-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dedei K Hammond can be reached at (571)270-3819. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICOLE K GILLESPIE/Examiner, Art Unit 2837
/DEDEI K HAMMOND/Supervisory Patent Examiner, Art Unit 2837
