Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The Amendments dated 4/14/26 have been entered. In light of the amendments, new grounds of rejections have been found. Please see 102/103 sections below.
With this office action claims 1-20 are rejected.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 8, 12 13, 14, 16, 17 and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhou (US 20170364752 A1 A1).
With respect to claim 1 Zhou teaches (claim 1) An electronic device, comprising:
a processor (Zhou ¶[[0151] For example, FIG. 6 is a block diagram that illustrates a computer system 600 upon which an example embodiment of the invention may be implemented. Computer system 600 includes a bus 602 or other communication mechanism for communicating information, and a hardware processor 604 coupled with bus 602 for processing information. Hardware processor 604 may be, for example, a general purpose microprocessor.) to:
perform a first noise cancellation on a video stream using a first noise canceller characteristic (Zhou ¶[0066] In some embodiments, microphone signals acquired by the microphone system (108) with the omnidirectional microphone pattern or audio beam pattern (302-1) of FIG. 3A can be spatially filtered [first noise canceller characteristic](or applied with audio beam forming techniques) to generate a new audio beam pattern different from the initial audio beam pattern).
detect a change in salience of a visual object within the video stream (Zhou¶[ [0083] In some embodiments, a candidate salient object may comprise a set of data elements, functions, etc., including but not limited to any of: area (e.g., single point, full sphere, triangle, a round shape, a rectangle shape, a regular or irregular shape, etc.), volume (if three dimensional space is to be represented in the output media content), position (e.g., centroid of area, etc.), one or more motion vectors (e.g., change in position or velocity between successive image frames or between successive audio samples, etc.), a viewing angle, a saliency score, etc, ¶[0036] Candidate salient objects can be created from the video objects and/or sound objects. Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations [second noise cancellation] such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects. ); and
perform a second noise cancellation on the video stream using a second noise canceller characteristic based on the change in the object salience (Zhou¶[ [0083] In some embodiments, a candidate salient object may comprise a set of data elements, functions, etc., including but not limited to any of: area (e.g., single point, full sphere, triangle, a round shape, a rectangle shape, a regular or irregular shape, etc.), volume (if three dimensional space is to be represented in the output media content), position (e.g., centroid of area, etc.), one or more motion vectors (e.g., change in position or velocity between successive image frames or between successive audio samples, etc.), a viewing angle, a saliency score, etc, ¶[0036] Candidate salient objects can be created from the video objects and/or sound objects. Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations [second noise cancellation] such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects. )
With respect to claim 8 Zhou teaches A computing device, comprising: a camera (Zhou ¶[0045] In some embodiments, the video object-tracking block (102) is implemented with software, hardware, a combination of software and hardware, etc., to perform object extraction and tracking based at least in part on input image data captured with a camera system 106 deployed in a spatial environment);
and a processor (Zhou ¶[[0151] For example, FIG. 6 is a block diagram that illustrates a computer system 600 upon which an example embodiment of the invention may be implemented. Computer system 600 includes a bus 602 or other communication mechanism for communicating information, and a hardware processor 604 coupled with bus 602 for processing information. Hardware processor 604 may be, for example, a general purpose microprocessor) to:
receive a video stream from the camera (Zhou¶[0034].The media metadata may comprise audio metadata, image metadata, metadata related to both audio and video, etc., that includes positional and motion information related to the sound objects and/or video objects as extracted from source media content, extracted from raw media content, extracted from a video stream generated by a spherical camera, etc. );
modify an acoustic noise canceller based on a change in salience of an object depicted within the video stream (Zhou¶[ [0083]. Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations [second noise cancellation] such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects. ); and
perform acoustic noise cancellation on the video stream using the modified acoustic noise canceller (Zhou¶[ [0083] Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations [second noise cancellation] such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects. )
With respect to claim 12 Zhou teaches wherein modifying the acoustic noise canceller based on the change in the object salience comprises modifying the acoustic noise canceller based on an audible characteristic of an object (Zhou¶ [0029 During the playback, the user can play, stop, pause, forward, and rewind the recorded audio signal and associated video using standard "play/stop", "rewind", and "forward" buttons 410. In addition, during the playback, the user can change the audio mode, for example, to reduce noise, focus on one or more sound sources, and the like. One or more additional control or option buttons 420 are available to enable the user to control the playback and change to a different audio mode or toggle between two or more audio processing modes. For example, there can be one button corresponding to each audio mode.)
With respect to claim 13, Zhou teaches A non-transitory tangible computer-readable medium comprising instructions when executed cause a processor of an electronic device (Zhou¶ [0148] In various example embodiments, an apparatus, a system, an apparatus, or one or more other computing devices performs any or a part of the foregoing methods as described. In an embodiment, a non-transitory computer readable storage medium stores software instructions, which when executed by one or more processors cause performance of a method as described herein.) to:
filter video using a digital filter (Zhou¶[ [0083]. Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations [second noise cancellation] such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects. );
detect a change in a salience of an object depicted in the video -in real time(Zhou¶[ [0083] In some embodiments, a candidate salient object may comprise a set of data elements, functions, etc., including but not limited to any of: area (e.g., single point, full sphere, triangle, a round shape, a rectangle shape, a regular or irregular shape, etc.), volume (if three dimensional space is to be represented in the output media content), position (e.g., centroid of area, etc.), one or more motion vectors (e.g., change in position or velocity between successive image frames or between successive audio samples, etc.), a viewing angle, a saliency score, etc, ¶[0036] Candidate salient objects can be created from the video objects and/or sound objects. Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations [second noise cancellation] such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects. ) ;
modify the digital filter based on an audible characteristic of the object (Zhou¶ 0036] Video objects can be identified and tracked in a video processing chain which incorporates computer vision technologies. The video objects can be merged with sound objects identified and tracked in a corresponding audio processing chain. Candidate salient objects can be created from the video objects and/or sound objects. Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects. .) ; and
filter an audio portion of the video using the modified digital filter (Zhou¶ 0036] Video objects can be identified and tracked in a video processing chain which incorporates computer vision technologies. The video objects can be merged with sound objects identified and tracked in a corresponding audio processing chain. Candidate salient objects can be created from the video objects and/or sound objects. Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects. .) ; ) .
With regards to claim 14 Zhou teaches wherein modifying the digital filter based on the audible characteristic of the object comprises modifying the digital filter to pass the audible characteristic of the object (Zhou¶ [0095] Example adjustments/modifications may include, but are not necessarily limited to only, any of: content compression (e.g., allocate a relatively large bit budget of a rate limited bitstream to a salient video object or a salient sound object, etc.); saliency level adjustment (e.g., adjust a salient video object to be more visible, adjust a salient sound object to be more audible)
With respect to claim 16 Zhou teaches wherein modifying the acoustic noise canceller based on the change in the object salience is performed automatically without a user (Zhou ¶[0112] In some embodiments, media metadata as described herein can be used for object tracking by one or more end users in games, consumer-generated media content, consumer-contributed media content, consumer-mixed media content, etc. The matched sound/video objects can be used, for example, to improve/enhance audio data captured by a consumer device (e.g., an action camera, etc.) in media content such as VR content, AR content, compute games, etc. The audio data captured by the consumer device may represent a real-world person's voice, can be automatically isolated and selected for emphasis/boost relative to other content (e.g., background noise, surround noise, ambient noise, mechanical noises, another real-world person's voice, etc.), can be accurately time-wise and position-wise to the real-world person's visual feature captured by a camera operating in conjunction with the consumer device, etc.).
With respect to claim 17 Zhou teaches wherein the change in the object salience comprises detecting that an object has moved into a camera subject position (Zhou ¶[0036] Video objects can be identified and tracked in a video processing chain which incorporates computer vision technologies. The video objects can be merged with sound objects identified and tracked in a corresponding audio processing chain.).
With respect to claim 19 Zhou teaches modifying the digital filter comprises passing an audible characteristic of an object determined to have salience and canceling an audible characteristic of an object determined not to have salience ([0036] Video objects can be identified and tracked in a video processing chain which incorporates computer vision technologies. The video objects can be merged with sound objects identified and tracked in a corresponding audio processing chain. Candidate salient objects can be created from the video objects and/or sound objects. Salient objects can be selected from the candidate salient objects. The salient objects may be applied with specific manipulations such as video object related manipulations, sound object related manipulations, etc., to enhance their saliency in media content relative to other (e.g., less salient, non-salient, etc.) objects.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-3, 5-6, 9 are rejected under pre-AIA 35 U.S.C. 103 as being unpatentable over Zhou in further view of Kuo (US 20160086345 A1).
With respect to claim 2 Zhou does not explicitly disclose however Kuo teaches wherein detecting the change in the object salience comprises detecting motion relating to an object (Kuo ¶[0016] IP cameras are generally used for surveillance purposes. For example, IP cameras may perform motion detection to detect a moving object in the video, such as a person approaching the front door [motion relating to an object], in order to extract useful information for the user).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the detection of Kuo in order to determine events more accurately ([0015], Kuo);
With respect to claim 3 Zhou does not explicitly disclose however Kuo teaches wherein detecting the change in the object salience comprises detecting a person engaging with an object (Kuo ¶[0016] IP cameras are generally used for surveillance purposes. For example, IP cameras may perform motion detection to detect a moving object in the video, such as a person approaching the front door [person engaging with an object], in order to extract useful information for the user).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the detection of Kuo in order to determine events more accurately ([0015], Kuo);
With respect to claim 5 Zhou does not explicitly disclose however Kuo teaches wherein detecting the change in the object salience comprises identifying an object between a person and a camera (Kuo ¶[0016] IP cameras are generally used for surveillance purposes. For example, IP cameras may perform motion detection to detect a moving object in the video, such as a person approaching the front door [door between camera and person], in order to extract useful information for the user).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the detection of Kuo in order to determine events more accurately ([0015], Kuo);
With respect to claim 6 Zhou does not explicitly disclose however Kuo teaches wherein detecting the change in the object salience comprises detecting a position of a person (Kuo ¶[0016] IP cameras are generally used for surveillance purposes. For example, IP cameras may perform motion detection to detect a moving object in the video, such as a person approaching the front door [approaching= position of a person], in order to extract useful information for the user).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the detection of Kuo in order to determine events more accurately ([0015], Kuo);
With respect to claim 9 Zhou does not explicitly disclose however Kuo teaches wherein the change in the object salience comprises detecting an object at a camera subject position (Kuo ¶[0016] IP cameras are generally used for surveillance purposes. For example, IP cameras may perform motion detection to detect a moving object in the video, such as a person approaching the front door [motion relating to an object], in order to extract useful information for the user).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the detection of Kuo in order to determine events more accurately ([0015], Kuo);
Claims 4 are rejected under pre-AIA 35 U.S.C. 103 as being unpatentable over Zhou in further view of Zhang (US 20220068248 A1).
With respect to claim 4 Zhou does not explicitly disclose however Zhang teaches wherein detecting the change in the object salience comprises determining an object is a musical instrument (Zhang¶[0085] When the preset condition is the musical instrument corresponding to the target music video, the method further includes determining the musical instrument corresponding to the music in the target music video, and rendering the digital music score segment to generate a music score segment image corresponding to the musical instrument. For example, the fingering of the musical instrument is provided to the user as a music score segment image. For example, if it is determined that the musical instrument is a piano, the piano fingering segment image may be provided to the user.).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the musical instrument of Zhang in order to improve user experience ([0007], Zhang)
Claims 7 are rejected under pre-AIA 35 U.S.C. 103 as being unpatentable over Zhou in further view of Yanagihara (US 20150002740 A1).
With respect to claim 7 Zhou does not explicitly disclose however Yanagihara teaches wherein detecting the change in the object salience comprises detecting a face (Yanagihara¶[0056] (1) When the volume of a human voice (voice signal) in a video relatively increases compared with the volume of other sounds (background sound signal), it is determined that a human face is displayed in the scene of the video. Accordingly, a mode is specified such that image quality improvement control data should be output to the image quality improvement engine so as to perform color adjustment to make the color of human skin finely viewable [first noise canceller characteristic].
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the face recognition of Yanagihara in order to improve align audio with the proper user.
Claims 10-11 and 15 are rejected under pre-AIA 35 U.S.C. 103 as being unpatentable over Zhou in further view of Jong (US 20080189613 A1).
With respect to claim 10 Zhou does not explicitly disclose however Jong teaches wherein modifying the acoustic noise canceller based on the change in the object salience comprises modifying the acoustic noise canceller to pass a frequency band of an object (Jong ¶ [0032] For example, if the selected sound source object is a sound source object of a classic tone, the control unit 130 may play the corresponding multimedia file in a classic tone by increasing an output of a low frequency (for example, 125 Hz). If the selected sound source object is a sound source object of a vocal tone, the control unit 130 may play a multimedia file by decreasing an output of a frequency band covering a human voice. Therefore, the user may hear only the melody portion of the multimedia file by eliminating the human voice.).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the frequency band of Jong in order to block unwanted sounds.
With respect to claim 11 Zhou does not explicitly disclose however Jong teaches wherein modifying the acoustic noise canceller based on the change in the object salience comprises modifying the acoustic noise canceller from rejecting a frequency band of an object to passing the frequency band of the object (Jong ¶ [0032] For example, if the selected sound source object is a sound source object of a classic tone, the control unit 130 may play the corresponding multimedia file in a classic tone by increasing an output of a low frequency (for example, 125 Hz). If the selected sound source object is a sound source object of a vocal tone, the control unit 130 may play a multimedia file by decreasing an output of a frequency band covering a human voice. Therefore, the user may hear only the melody portion of the multimedia file by eliminating the human voice.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou with the frequency band of Jong in order to remove unwanted sounds from the audio.
With regards to claim 15, Zhou does not explicitly disclose however Jong teaches wherein modifying the digital filter based on the audible characteristic of the object comprises modifying the digital filter to pass non-human sounds (Jong ¶ [0032] For example, if the selected sound source object is a sound source object of a classic tone, the control unit 130 may play the corresponding multimedia file in a classic tone by increasing an output of a low frequency (for example, 125 Hz). If the selected sound source object is a sound source object of a vocal tone, the control unit 130 may play a multimedia file by decreasing an output of a frequency band covering a human voice. Therefore, the user may hear only the melody portion of the multimedia file by eliminating the human voice.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou to include non-human sounds of Jong in order to remove human sounds from audio.
Claims 18 is rejected under pre-AIA 35 U.S.C. 103 as being unpatentable over Zhou in further view of Thompson (US 20190076741 A1).
With regards to claim 18, Zhou does not explicitly disclose however Thompson teaches wherein modifying the acoustic noise canceller comprises selecting a first set of audible characteristics to pass for a first object determined to have salience and selecting a different second set of audible characteristics to pass for a second object determined to have salience at a later time in the video stream (Thompson¶[0091]The audio may be recorded contemporaneously with the video (i.e., “live” relative to the video) or at a different time prior to or after the video. The audio 604 may include one or more salient audio segments 612 and one or more non-salient audio segments 614. In some cases, the salient and non-salient audio segments may be temporally aligned with the salient and non-salient video segments, respectively [each video segment, temporally, has different audible characteristics ] ).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou to include different audible characteristics of Thompson in order to improve contextual audio/video synchronization.
Claims 20 is rejected under pre-AIA 35 U.S.C. 103 as being unpatentable over Zhou in further view of Frahm (US 20160309081 A1).
With regards to claim 20, Zhou does not explicitly disclose however Frahm teaches wherein detecting the change in the object salience comprises determining that the object is salient based on a combination of detection of a face of a person, detection that the object is between the person and a camera, and detection that the person is engaging with the object (Frahm ¶[0048] FIG. 4 shows an example input (a user looking at a scene) and the example output (automatically autofocusing on a part of the image that the user-facing camera sees based on what the user is actually looking at). For example, in FIG. 4, a gaze vector calculated by gaze estimation module 106 passes through a portion of the scene. The intersection of the gaze vector with the scene is used by subregion selection module 108 to determine the portion of the scene on which the scene facing camera should be focused. A saliency analysis maybe performed to identify the object in the scene that is closest to the intersection of the gaze vector with the scene. In the illustrated example, the gaze vector intersects with a person's head. However if the gaze vector had not intersected exactly with the person's head, the saliency analysis maybe used to identify the person's head as the closest object to the intersection with the gaze vector and the scene facing camera may be focused on that object.).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify noise cancellation of Zhou to include detection of Frahm in order to improve contextual audio/video synchronization.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ATHAR N PASHA whose telephone number is (408)918-7675. The examiner can normally be reached Monday-Thursday Alternate Fridays, 7:30-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached on (571)272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ATHAR N PASHA/Primary Examiner, Art Unit 2657
Prosecution Insights