DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 8, 10, 18, and 20 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 4, 6-7, 9, 11-12, 14, 16-17, and 19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nicol et al. (US #2021/0204003).
Regarding Claim 1, Nicol discloses an electronic device (title, abstract, figs. 1-29) providing a sound source edit function (fig. 26: 2614; fig. 28: apply current edit decision, apply new edit decision), the electronic device comprising:
memory configured to store an original sound source (Nicol ¶0076 discloses the reference audio data can include audio content recorded by microphones in a rehearsal and processed by estimator 522. In the rehearsal, the microphones and sound sources are placed in same acoustic arrangement as in live event 100. The microphones then record audio signals when each sound source is played individually. In addition, the microphones can record noise samples, where no sound source is playing. ¶0161 discloses reference audio data) and at least one instruction (Nicol ¶0340 discloses memory 2150 can store video processing instructions that, when executed by processor 2104, can cause processor 2104 to perform various operations described in reference to figs. 25-29); and
at least one processor operatively connected to the memory (Nicol fig. 21: memory interface 2102 connected to processor(s) 2104 and memory 2150),
wherein the at least one instruction (Nicol ¶0350 discloses audio processing manager 2320 can include computer instructions that, when executed, cause processor 2302 to perform various audio estimation and manipulation operations), when executed by the at least one processor individually or collectively, causes the electronic device to:
receive an original sound source (Nicol fig. 5: rehearsal data) including a plurality of sound source elements (Nicol fig.5: Nm signals [one per microphone]; ¶0074 discloses each sound source can correspond to, for example, an instrument, a vocalist, a group of instruments or a group of vocalists),
select a test sound source corresponding to the original sound source from among a plurality of test sound sources that are pre-stored (Nicol ¶0056 discloses the attributes can include, for example, how many decibels (dB) should separate two sound sources [e.g., between the lead vocalist and other sound sources], which sound source should be dominant [e.g., by playing above volume levels of other sound Sources], and by how much [e.g., by X dB]. The one or more user interface items can accept user input for specifying a rehearsal session, in which reference audio data from the sound sources is collected for automated mixing. ¶0057 discloses integrated recorder 200 can optionally perform one or more operations including, for example, synchronization of signals from the main and spot microphones, separation of sound sources from recorded signals, mixing signals of the different sound sources based on reference audio data, and mastering the mixed signals. ¶0076 discloses mixing and mastering units 506 can be software and/or hardware components of server 408 configured to perform mixing operations on channels of individual sound sources based at least partially on reference audio data, and perform mastering operations on mixed audio signals to produce final N channel audio signals [e.g., stereo audio, surround sound]), the plurality of test sound sources each being matched to at least one of a plurality of sound source separators according to a performance indicator (Nicol ¶0272 discloses the server system 2502 is configured to automatically edit live data 2504, e.g., live audio and video of a musical performance or live audio and video of any event, based on rehearsal video data and rehearsal audio data. The audio data 2508 includes N audio signals from N audio capturing devices, e.g., one or more microphones. The number and position of the audio capturing devices can be arbitrary. Accordingly, respective input gains of the audio capturing devices may be unknown. Due to arrangement of the audio capturing devices, levels of audio signals may not directly correlate to natural or perceived levels at which the performers are playing. ¶0294 discloses the AVEE 2518 can boost the prominence of a performer designated as having a lead role, to match the maximum prominence among all performers. The AVEE 2518 can designate one or more performers as having a lead role based on input received from a user interface),
select at least one sound source separator, which corresponds to separation of the test sound source, from among the plurality of sound source separators (Nicol fig. 5: source separator 504; estimator 522 estimates based on rehearsal data [i.e., test sound source]; ¶0076 discloses the reference audio data can include audio content recorded by microphones in a rehearsal and processed by estimator 522. In the rehearsal, the microphones and sound sources are placed in same acoustic arrangement as in live event 100. The microphones then record audio signals when each sound source is played individually),
extract the plurality of sound source elements by separating the original sound source by using the selected at least one sound source separator (Nicol ¶0074 discloses source separator 504 is configured to separate each sound source from the Nm synchronized signals. Each sound source can correspond to, for example, an instrument, a vocalist, a group of instruments or a group of vocalists., and
store an edit sound source including the extracted plurality of sound source elements (Nicol fig. 26: steps 2602 to 2614).
Regarding Claim 2, Nicol discloses the electronic device of claim 1, wherein the at least one instruction, when executed by the at least one processor individually or collectively, further causes the electronic device to:
in connection with the selecting of the at least one sound source separator, determine the at least one sound source separator based on at least one of a degree of distortion of the plurality of sound source separators for at least one test sound source among the plurality of test sound sources (Nicol ¶0077 discloses estimator 522 can process the Nm signals, determine a loudness matrix, derive sound source characteristics [i.e., distortion] and positions, and provide the instrument characteristics and positions to mixing and mastering units 506 for the mixing operations. ¶0162 discloses in the cost function, estimator 522 can penalize estimating instrument loudness in live data that are significantly above those represented in reference audio data. In the cost function, estimator 522 can scale a cost function by an average across microphone difference between the level measured between performance and rehearsal; fig. 5. ¶0176-¶0186; fig. 12), a speed at which the plurality of sound source separators perform sound source separation, or a hardware resource usage amount of the plurality of sound source separators.
Regarding Claim 4, Nicol discloses the electronic device of claim 1, wherein the at least one instruction, when executed by the at least one processor individually or collectively, further causes the electronic device to:
when a plurality of sound source separators are selected in the selecting of the at least one sound source separator, extract sound source elements by using each sound source separator in the extracting of the plurality of sound source elements (Nicol fig. 25: feature extraction tempo, novelty, beat/downbeat 2522; ¶0281-¶0282).
Regarding Claim 6, Nicol discloses the electronic device of claim 1, wherein the at least one instruction, when executed by the at least one processor individually or collectively, further causes the electronic device to:
provide at least one edit sound source list stored (Nicol ¶0398 discloses determining a need value for each source-band pair in a list of source-band pairs, each source-band pair representing a sound source and a frequency band, the need value indicating a relative importance of the sound source represented in the pair being equalized in the frequency band in the pair relative to other sound sources and other frequency bands and a masking level of the sound source represented in the pair by one or more other sound sources),
receive a user input for selecting a specific edit sound source included in the edit sound source list (Nicol ¶0397 discloses mapping, by the equalizer, a respective signal for each sound source to excitation in each frequency band. ¶0399 discloses iteratively equalizing signals of sound sources represented in the source-band pairs in the list having a highest need value and removing equalized source-band pairs from the list until a highest need value of remaining source-band pairs is below a threshold value. ¶0345 discloses mixing service 2240 can provide user interfaces that allow a mixing professional to log in through a remote console to perform mixing operations on live audio data; fig. 22. ¶0056 discloses the GUI can include one or more user interface items configured to accept user input for specifying the attributes of sound sources [e.g., target volume or gain levels of guitar, lead vocalist, bass or drum]. The attributes can include, for example, how many decibels (dB) should separate two sound sources [e.g., between the lead vocalist and other sound sources], which sound source should be dominant [e.g., by playing above volume levels of other sound sources], and by how much [e.g., by X dB]. The one or more user interface items can accept user input for specifying a rehearsal session, in which reference audio data from the sound sources is collected for automated mixing), and
output at least one sound source element included in the selected specific edit sound source (Nicol ¶0400 discloses providing the equalized signals for playback on one or more loudspeakers).
Regarding Claim 7, Nicol discloses the electronic device of claim 6, wherein the at least one instruction, when executed by the at least one processor individually or collectively, further causes the electronic device to:
receive a user input for adjusting a sound field effect of a specific sound source element or a user input for selecting an application program for adjusting a specific sound field effect (Nicol ¶0056 discloses the GUI can include one or more user interface items configured to accept user input for specifying the attributes of sound sources [e.g., target volume or gain levels of guitar, lead vocalist, bass or drum]. The attributes can include, for example, how many decibels (dB) should separate two sound sources [e.g., between the lead vocalist and other sound sources], which sound source should be dominant [e.g., by playing above volume levels of other sound sources], and by how much [e.g., by X dB]),
obtain an original sound source corresponding to the selected specific edit sound source (Nicol ¶0056 discloses the one or more user interface items can accept user input for specifying a rehearsal session, in which reference audio data from the sound sources is collected for automated mixing), and select a new sound source separator (Nicol fig. 5: 504) related to adjusting a sound field effect of the specific sound source element or a new sound source separator corresponding to a type of the selected application program (Nicol ¶0066 discloses for example, server 408 can increase or decrease a volume level of a microphone when clipping is detected, increase or decrease a sample bitrate or bit depth, or select a compression type, based on detected bandwidth limitations. ¶0280 discloses in another example, the data is gathered by direct human input via a customized graphical interface, for example, by showing a still frame of the band as captured by one of the video devices, and prompting the user to tap on each performer to then select which sound sources he plays from a pre-populated menu),
perform sound source separation on the original sound source by using the new sound source separator (Nicol ¶0091 discloses mixing unit 702 is a component of mixing and mastering units 704 configured to perform mixing operations on the Ns signals from source separator 504 or the Nm synchronized signals from synchronizer 502 automatically using reference audio data and input from one or more remote or local mixing consoles),
generate a new edit sound source by applying the sound field effect to a sound source element, which is selected by a user, from among the sound source-separated sound source elements (Nicol ¶0155 discloses a leveling and panning unit can combine the circuits and functions of leveling unit 706 and panner 708. The leveling and panning unit can receive reference audio data. The reference audio data can include a representation of channel signals from multiple channel signal sources recorded in a rehearsal of one or more sound sources. The leveling and panning unit can receive target level data, the target level data specifying a target level of each sound source. The leveling and panning unit can receive live audio data. The live audio data can include recorded or real time signals from the one or sound sources playing at live event 100. The leveling unit can determine a joint cost function for leveling the live audio data and panning the live audio data based on the reference audio data. The leveling and panning unit can apply the gains and panning positions to signals of live audio data of an event to achieve an audio effect of leveling sound sources in the live audio data and placing sound sources in the live audio data between left and right of a stereo sound stage for output to a storage device or to a stereo sound reproduction system), and
output the new edit sound source (Nicol fig. 5: 508; fig. 6: 608. ¶0155 discloses the leveling and panning unit can apply the gains and panning positions to signals of live audio data of an event to achieve an audio effect of leveling sound sources in the live audio data and placing sound sources in the live audio data between left and right of a stereo sound stage for output to a storage device or to a stereo sound reproduction system).
Regarding Claim 9, Nicol discloses the electronic device of claim 6, wherein the at least one instruction, when executed by the at least one processor individually or collectively, further causes the electronic device to:
provide a screen interface capable of adjusting at least one of a volume or a sound field effect of the at least one sound source element (Nicol ¶0056 discloses the GUI can include one or more user interface items configured to accept user input for specifying the attributes of sound sources [e.g., target volume or gain levels of guitar, lead vocalist, bass or drum]. The attributes can include, for example, how many decibels (dB) should separate two sound sources [e.g., between the lead vocalist and other sound sources], which sound source should be dominant [e.g., by playing above volume levels of other sound sources], and by how much [e.g., by X dB]. The one or more user interface items can accept user input for specifying a rehearsal session, in which reference audio data from the sound sources is collected for automated mixing).
Claims 11-12, 14, 16-17, and 19 are rejected for the same reasons as set forth in Claims 1-2, 4, 6-7, and 9.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nicol et al. (US #2021/0204003) in view of Chatterjee et al. (Visual Scene Graphs for Audio Source Separation).
Regarding Claim 3, Nicol discloses the electronic device of claim 1, but may not explicitly disclose wherein the performance indicator includes at least one of a source to distortion ratio (SDR), a sources to artifacts ratio (SAR), a signal to interference ratio (SIR), a signal to noise ratio (SNR), or similarity with a target value.
However, Chatterjee teaches wherein the performance indicator includes at least one of
a source to distortion ratio (SDR), a sources to artifacts ratio (SAR), a signal to interference ratio (SIR), a signal to noise ratio (SNR), or similarity with a target value (Chatterjee section 4.3 discloses we report the model performances in terms of the Signal-to-Distortion Ratio [SDR] [dB], where higher SDR indicates more faithful reproduction of the original signal. We also report two related measures, Signal-to-Interference Ratio [SIR] [which gives an indication of the amount of reduction of interference in the estimated signal] and Signal-to-Artifact Ratio [SAR] [which gives an indication of how much artifacts were introduced]; tables 1 and 3; fig. 2).
Nicol and Chatterjee are analogous art as they pertain to separating sound. Therefore it would have been obvious to someone of ordinary skill in the art before the effective filing date of the invention was made to modify performance indicator (as taught by Nicol, ¶0190) to model performances in terms of SDR, SIT and SAR (as taught by Chatterjee, section 4.3, and tables 1 and 3) to provide a novel algorithm that leverages the power of scene graphs to induce audio source separation (Chatterjee, section 5).
Claim 13 is rejected for the same reasons as set forth in Claim 3.
Claims 5 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nicol et al. (US #2021/0204003) in view of Shin et al. (US #2006/0122841).
Regarding Claim 5, Nicol discloses the electronic device of claim 1, but may not explicitly disclose wherein the at least one instruction, when executed by the at least one processor individually or collectively, further causes the electronic device to: store metadata related to the edit sound source, and wherein the metadata includes at least one of a name of a sound source separator applied to sound source separation of the original sound source, a number of sound source elements, a type of the sound source elements, or a name of a sound field effector applied to the sound source elements.
However, Shin (title, abstract, figs. 1-9) teaches wherein the at least one instruction, when executed by the at least one processor individually or collectively, further causes the electronic device to:
store metadata related to the edit sound source (Shin ¶0033 discloses the sound source management unit 30 stores the sound source data processed as digital data and sampled by the sound source processing unit 20, in the sound source database 100, or reads sound source data stored in the sound source database 100 and senses whether or not there is a corresponding sound source in input sound), and
wherein the metadata includes at least one of a name of a sound source separator applied to sound source separation of the original sound source, a number of sound source elements, a type of the sound source elements, or a name of a sound field effector applied to the sound source elements (Shin ¶0038 discloses if the extracted sound sources are sound sources registered in the sound source database 100 described above in operation 206, then by referring to the mapping information of the image mapping database 120, images corresponding to the sound sources are read in the image database 140, and displayed on a display unit in operation 208. ¶0039 discloses for example, if the sound being reproduced through the digital AV device is orchestral music, then by using the frequency characteristics of each musical instrument, such as violin, viola, cello, and double bass, the sound source for the musical instrument can be extracted from the sound being reproduced. By using the sound source database 100, it is confirmed whether or not the extracted sound source for each musical instrument is a registered sound source. Then, if it is a registered sound source, an image corresponding to the sound source is displayed on the display unit by referring to the mapping information of the sound source/image database 120).
Nicol and Shin are analogous art as they pertain to separating sound. Therefore it would have been obvious to someone of ordinary skill in the art before the effective filing date of the invention was made to modify editing sound source (as taught by Nicol) to extract a registered sound source from other input sound (as taught by Shin, ¶0033) to overcome the conventional digital AV device which cannot provide a function for extracting, managing and displaying a specific sound source included in sound being reproduced, it cannot provide a function for selectively reproducing a specific sound source from the sound being reproduced (Shin, ¶0006).
Claim 15 is rejected for the same reasons as set forth in Claim 5.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YOGESHKUMAR G PATEL whose telephone number is (571)272-3957. The examiner can normally be reached 7:30 AM-4 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at (571) 272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YOGESHKUMAR PATEL/Primary Examiner, Art Unit 2691
Prosecution Insights