DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims Status
Claims 1-32 are currently pending in the application.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-7, 9-24 and 29-32 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 2017/0256271 A1 to Lyon et al.
As to claim 1, Lyon discloses a computer-implemented method for enhancing audio, the method comprising: receiving audio portion data of a recorded video, the audio portion data comprising non- media sounds and media sounds (see page.2, ¶0021,¶0023-¶0024¶ ); determining reference media data for the media sounds in the audio portion data of the recorded video (see page.2,¶0023-¶0024,¶0050); generating synchronized media data based at least on the reference media data and the media sounds in the audio portion data, the synchronized media data being synchronized to the media sounds in the audio portion data (see page.2,¶0025,¶0050); providing, to a device, at least one of the synchronized media data or data based on the synchronized media data for combining the synchronized media data and the audio portion data to obtain an enhanced video (see page.2, ¶0023-¶0024, ¶0050).
As to claim 2, Lyon further discloses wherein determining the reference media data comprises: extracting one or more audio fingerprints from the media sounds; and matching at least one of the one or more audio fingerprints against one or more reference audio fingerprints to identify the reference media data (see page.5, ¶0050).
As to claim 3, Lyon further discloses wherein generating the synchronized media data comprises: identifying a coarse temporal offset for the reference media data when compared to the audio portion data (see page.6, ¶0063,¶0050); identifying a fine-grained temporal offset for the reference media data when compared to the audio portion data, wherein the fine-grained temporal offset search space is based on at least one of the determined reference media data or the coarse temporal offset (see page.7, ¶0069); and generating the synchronized media data based on the reference media data, the media sounds in the audio portion data, and the fine-grained temporal offset (see page.2, ¶0023-¶0050, ¶0063, ¶0069).
As to claim 4, Lyon further discloses wherein identifying the fine-grained temporal offset comprises: segmenting the audio portion data into a plurality of independent sub-intervals(see page.5,¶0052); determining a separate fine-grained temporal offset for each of the plurality of independent sub-intervals(see page.5, ¶0050,¶0052,¶0063); and selecting the fine-grained temporal offset from among the fine-grained temporal offsets of the plurality of independent sub-intervals using a voting mechanism or lowest bit-error criterion (see page.5, ¶0052,¶0050, ¶0063).
As to claim 5, Lyon further discloses wherein identifying the fine-grained temporal offset comprises: segmenting the reference media data and the audio portion data into a plurality of overlapping segments (see page.7,¶0074,¶0050,¶0052,¶0063);
and processing the plurality of overlapping segments in parallel using multithreading to improve synchronization performance and speed (seepage.13,¶0142,¶0050,¶0051,¶0063).
As to claim 6, Lyon further discloses wherein identifying the fine-grained temporal offset comprises matching fine-grained audio features extracted from the audio portion data to pre- extracted fine-grained audio features of the reference media data that have been stored in a feature database(see page.5,¶0049-¶0051,¶0063).
As to claim 7, Lyon further discloses generating reference canceled audio data based on the audio portion data of the recorded video and the synchronized media data (see page.2,¶0021,¶0025,¶0050); and providing the reference canceled audio data to the device for combining the reference canceled audio data and the audio portion data to obtain the enhanced video(see page.2,¶0021,¶0023-¶0025,¶0050).
As to claim 9, Lyon further discloses wherein generating the reference canceled audio data is performed concurrently with video recording (see page.14,¶0155, ¶0021, ¶0023-¶0024,¶0050).
As to claim 10, Lyon further discloses generating reference enhanced audio data based on the audio portion data and the synchronized media data; and providing the reference enhanced audio data to the device for combining the reference enhanced audio data and the original audio portion data to obtain the enhanced video (see page.7, ¶0072, ¶0021,¶0023-¶0025,¶0050).
As to claim 11, Lyon further discloses wherein generating the reference enhanced audio data comprises one or more of: providing the synchronized media data as a reference signal to an adaptive filter configured to enhance the media components of the audio portion data, generating an error signal by subtracting the filter output from the audio portion data, iteratively updating the filter coefficients based on the error signal using an adaptive algorithm, and using the final error signal as the reference enhanced audio data (see page.2, ¶0025, ¶0050,¶00052,¶0063,¶0074); applying one or more room acoustic simulation methods to model the recording environment's acoustics and generate the reference enhanced audio data; or passing the synchronized media data directly as the reference enhanced audio data without further modification(see page.2, ¶0025, ¶0050,¶00052,¶0063,¶0074).
As to claim 12, Lyon further discloses wherein generating the reference enhanced audio data is performed concurrently with video recording(see page.2,¶0021,¶0023-¶0025).
As to claim 13, Lyon further discloses wherein the synchronized media data is synchronized to the media sounds in the audio portion data (see page.2,¶0021,¶0023-¶0025).
As to claim 14, Lyon further discloses wherein the audio portion data is captured by one or more microphones of a smartphone, tablet, laptop, concert sound system, stage sound system, broadcast system, field reporting system, or microphone array(see page.3,¶0029,¶0034,¶0021,¶0074).
As to claim 15, Lyon further discloses wherein determining the reference media data and generating the synchronized media data are performed concurrently (see page.2,¶0021,¶0023-¶0025,¶0050, ¶0142).
As to claim 16, Lyon further discloses wherein one or more of determining the reference media data or generating the synchronized media data is performed concurrently with video recording (see page.2,¶0021,¶0023-¶0025,¶0050, ¶0142).
As to claim 17, Lyon further discloses a non-transitory processor readable medium containing a set of instructions thereon for enhancing audio, wherein when executed by a processor, the instructions cause the processor to perform the method of claim 1 (see page.14,¶0151,¶0153).
As to claim 18, Lyon further discloses an apparatus for enhancing audio, the apparatus comprising: one or more processors; and memory accessible by the one or more processors, the memory storing instructions that when executed by the one or more processors, cause the apparatus to perform the method of claim 1(see page.14,¶0151).
As to claim 19, Lyon discloses a computer-implemented method for enhancing audio, the method comprising: receiving audio stream data, the audio stream data comprising non-media sounds and media sounds (see page.3,¶0029 and ¶0021,¶0023-¶0024); determining reference media data for the media sounds in the audio stream data(see page.2,¶0021,¶0023-¶0024,¶0050); generating synchronized media data based at least on the reference media data and the media sounds in the audio stream data(see page.2,¶0025,¶0050); and providing, to a device, at least one of the synchronized media data or data based on the synchronized media data to obtain an enhanced video(see page.2,¶0023-¶0025,¶0050).
As to claim 20, Lyon further discloses generating reference canceled audio data based on the audio stream data and the synchronized media data; and providing the reference canceled audio data to the device for combining the reference canceled audio data and the audio stream data to obtain the enhanced video (see page.2,¶0021,¶0023-¶0025,¶0050).
As to claim 21, Lyon further discloses generating reference enhanced audio data based on the audio stream data and the synchronized media data; and providing the reference enhanced audio data to the device for combining the reference enhanced audio data and the audio stream data to obtain the enhanced video (see page.2,¶0021,¶0023-¶0025,¶0050).
As to claim 22, Lyon further discloses a non-transitory processor readable medium containing a set of instructions thereon for enhancing audio, wherein when executed by a processor, the instructions cause the processor to perform the method of claim 19 (see page.14,¶0151,¶0153).
As to claim 23, Lyon further discloses an apparatus for enhancing audio, the apparatus comprising: one or more processors; and memory accessible by the one or more processors, the memory storing instructions that when executed by the one or more processors, cause the apparatus to perform the method of claim 19 (see page.14,¶0151).
As to claim 24, Lyon discloses a computer-implemented method for enhancing audio, the method comprising: generating or obtaining a recorded video, the recorded video comprising audio portion data and video portion data, the audio portion data comprising non-media sounds and media sounds (page.2,¶0021,¶0023-¶0024,¶0050); receiving at least one of: reference canceled audio data, the reference canceled audio data based on the media sounds of the audio portion data and synchronized to the audio portion data of the recorded video(see page.2,¶0021,¶0025,¶0050); or reference enhanced audio data, the reference enhanced audio data based on the media sounds of the audio portion data and synchronized to the audio portion data of the recorded video (see page.2,¶0021,¶0023-¶0025,¶0050); adjusting audio of the recorded video based on at least one of the reference canceled audio data or the reference enhanced audio data to obtain enhanced audio (see page.2,¶0021,¶0023-¶0025,¶0050); and generating an enhanced video based on the recorded video and the enhanced audio (see page.2,¶0021,¶0023-¶0025,¶0050).
As to claim 29, Lyon further discloses at least one of saving or sharing the enhanced video(see page.10,¶0114).
As to claim 30, Lyon further discloses wherein the audio portion data of the recorded video was recorded by one or more microphones of a smartphone, tablet, or laptop, wherein the video portion data was recorded by a camera of the smartphone, tablet, or laptop(see page.2,¶0021,¶0029,¶0034,¶0074).
As to claim 31, Lyon further discloses a non-transitory processor readable medium containing a set of instructions thereon for enhancing audio, wherein when executed by a processor, the instructions cause the processor to perform the method of claim 24 (see page.14,¶0151,¶0153).
As to claim 32, Lyon further discloses an apparatus for enhancing audio, the apparatus comprising: one or more processors; and memory accessible by the one or more processors, the memory storing instructions that when executed by the one or more processors, cause the apparatus to perform the method of claim 24 (see page.14,¶0151).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over US 2017/0256271 A1 to Lyon et al in view of US 11,741,934 B1 to Zhang et al.
As to claim 8, Lyon discloses wherein generating the reference canceled audio data comprises providing the synchronized media data as a reference signal to an adaptive filter configured to cancel the media components from the audio portion data (see page.7,¶0074,¶0021,¶0025,¶0050); generating an error signal by subtracting the adaptive filter output from the audio portion data (see page.8,¶0081,¶0021,¶0025,¶0050, ¶0074); and subtracting the filter output at the converged filter coefficients from the audio portion data to yield the reference canceled audio data (see page.2,¶0021,¶0025,¶0050,¶0074, ¶0081,¶0084).
Lyon fails explicitly discloses iteratively updating the filter coefficients based on the error signal using an adaptive algorithm until convergence.
Zhang discloses iteratively updating the filter coefficients based on the error signal using an adaptive algorithm until convergence (see col.11,ll.46-50)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lyon with the teaching as taught by Zhang in order to allow the system to dynamically determine the audio data to filter based on the error signal.
Claims 25-28 is rejected under 35 U.S.C. 103 as being unpatentable over US 2017/0256271 A1 to Lyon et al in view of US 2023/0328359 A1 to Brown et al.
As to claim 25, Lyon fails explicitly discloses displaying a user-selectable icon to generate or obtain the recorded video, wherein generating or obtaining the recorded video is based on receiving a selection of the user-selectable icon.
Brown discloses displaying a user-selectable icon to generate or obtain the recorded video, wherein generating or obtaining the recorded video is based on receiving a selection of the user-selectable icon (see fig.16A-16H; page.5,¶0047,¶0065).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lyon with the teaching as taught by Brown in order to allow the system to output an enhanced video based on user input .
As to claim 26, Brown further discloses displaying, on the video recording screen, a user-selectable icon that enables a user to switch between generating a standard video and generating an enhanced video; and generating the video in the mode selected via the user-selectable icon (fig.16A-16H;page.5,¶0115,¶0460,¶0479 and ¶0047,¶0065).
As to claim 27, Brown further discloses automatically selecting, during video generation, between generating a standard video and generating an enhanced video based on automatic content recognition of background media (see fig.16A-16H; page.5,¶0047,¶0065, ¶0115,¶0460, ¶0479).
As to claim 28, Brown further discloses displaying at least one user-selectable icon to adjust audio of the recorded video, wherein adjusting audio of the recorded video is based on receiving a selection of the at least one user-selectable icon (see fig.16A-16H; page.5,¶0047,¶0065).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MULUGETA MENGESHA whose telephone number is (469)295-9212. The examiner can normally be reached Monday-Friday 9:00AM-5:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benjamin Bruckart can be reached at 571-272-3982. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MULUGETA MENGESHA
Primary Examiner
Art Unit 2424
/Mulugeta Mengesha/ Primary Examiner, Art Unit 2424
Prosecution Insights