DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 22, 2026 has been entered.
Allowable Subject Matter
Claim 9 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 8, 11-13, 17, 19, 21 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kim et al. (US 9532140 B2, “Kim”).
As to claims 1, 11, 19, Kim discloses a method comprising:
obtaining first image data obtained using one or more first image sensors of a first device, the first image data representative of one or more first images (via video camera or image sensor of mobile device 100, Fig. 1A; Fig. 8, step S802; col. 8, lines 51-54; col. 15, lines 39-48);
obtaining first data obtained using one or more first sensors of the first device, the first data indicating at least a first position of a first speaker with respect to the first device (position of source of audio communication is obtained using microphones and camera; Fig. 1C; col. 11, lines 12-42; location information comprising the angle from the mobile device to the audio source, col. 2, lines 4-7);
sending, to a second device, the first image data along with first content to cause the second device to present the one or more first images along with the first content indicating the first position of the first speaker with respect to the first device (sending near-side video image and near-side location information identifying a near-side source of audio communication; col. 18, line 40 – col. 19, line 9);
obtaining second image data obtained using one or more second image sensors of the second device, the second image data representative of one or more second images (capturing far-side video image from far side mobile device camera sensor; col. 4, lines 29-39; col. 18, lines 46-52);
obtaining second data obtained using one or more second sensors of the second device, the second data indicating a second position of a second speaker with respect to the second device (obtaining far-side location information associated with the visual identification of the source of the far-side audio communication, col. 4, lines 54-57; col. 18, lines 46-52); and
sending, to the first device, the second image data along with second content to cause the first device to present the one or more second images along with the second content indicating the second position of the second speaker with respect to the second device (near-side mobile device displays far-side video image from far-side mobile device and far-side location information, col. 18, lines 53-64).
As to claim 3, Kim discloses: wherein: the first data includes location data; and the method further comprises determining, based at least on processing the location data, the first position of the first speaker with respect to the first device (position of source of audio communication is obtained using microphones and camera; Fig. 1C; col. 11, lines 12-42; first location information identifying a source of an audio communication, col. 10, lines 43-46; location information comprising the angle from the mobile device to the audio source, col. 2, lines 4-7).
As to claims 8, 17, Kim discloses:
obtaining third image data obtained using the one or more first image sensors of the first device, the third image data representative of one or more third images (identifying a second mouth of a second speaker in the video image, col. 2, lines 22-24; identifying a second person as a second source of the audio communication, col. 3, lines 33-34);
obtaining third data indicating at least a third position of the first speaker with respect to the first device (determining a second angle associated with a second direction from the point associated with the second source of the audio communication, col. 2, lines 24-27; second location information comprising a second angle from the mobile computing device to the second person, col. 3, lines 61-63); and
sending, to the second device, the third image data along with third content to cause the second device to present the one or more third images along with the third content indicating the third position of the first speaker with respect to the first device (multiple areas of emphasis when more than one speaker is talking at the same time, col. 14, lines 14-25).
As to claim 12, Kim discloses: wherein:
the first sensor data includes at least audio data representing speech from the first speaker (multi-microphone device tracks and enhances the voices of individual speakers, col. 9, lines 50-54; first location information identifying a source of an audio communication, col. 10, lines 43-46); and
the one or more processors are further to determine, based at least on analyzing the audio data using one or more acoustic source location processes, the first position of the first speaker with respect to the first device (location information comprising the angle from the mobile device to the audio source, col. 2, lines 4-7; direction of arrival (DOA) is estimated using a microphone array, calculating an observed value of phase delay at a particular frequency component, and filtering sound, col. 11, line 43 - col. 12, line 8).
As to claim 13, Kim discloses: wherein: the one or more first sensors include at least one or more location sensors; and the one or more processors are further to determine, based at least on the first sensor data, the first position indicating at least one of a distance or a direction of the first speaker with respect to the first device (location information comprising the angle from the mobile device to the audio source, col. 2, lines 4-7; direction of arrival (DOA) is estimated using a microphone array, calculating an observed value of phase delay at a particular frequency component, and filtering sound, col. 11, line 43 - col. 12, line 8).
As to claim 21, Kim discloses: further comprising generating, by at least one of the first device or a system that is remote from the first device and the second device, the first content indicating the first position of the first speaker with respect to the first device (Kim: user interface may be created that shows both the visually identifiable audio source and an angle associated with the audio sector information for a direction from the device to the audio source, col. 7, lines 33-36).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 4-5, 7, 14, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim in view of Poore et al. (US 2021/0055367 A1, “Poore”).
Kim discloses: the first data includes audio data representing speech from the first speaker (multi-microphone device tracks and enhances the voices of individual speakers, col. 9, lines 50-54; first location information identifying a source of an audio communication, col. 10, lines 43-46); and
the method further comprises determining, based at least on processing the audio data, the first position of the first speaker with respect to the first device (direction of arrival (DOA) is estimated using a microphone array, calculating an observed value of phase delay at a particular frequency component, and filtering sound, col. 11, line 43 - col. 12, line 8).
but differs from claim 2 in that it does not explicitly disclose: beamforming.
Poore teaches determining the direction of a sound source with respect to a head-mounted device by performing beamforming using a microphone array (para. 0023, 0056, 0060, 0064). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim with the above teaching of Poore in order to locate and indicate a sound source (para. 0054), while isolating the sound from ambient noise and reverberation, as taught by Poore (para. 0023).
As to claims 4, 14, Kim in view of Poore teaches: determining, based on at least one of the first data or the first image data, an identifier associated with the first speaker, wherein the first content further indicates the identifier associated with the first speaker (Poore: indicator 300 can include an identity, name, icon, symbol, graphic, etc. corresponding to the sound source, para. 0061).
As to claim 5, Kim in view of Poore teaches: wherein the identifier associated with the first speaker comprises at least one of: a specific identifier represented by user data associated with the first speaker; or a general identifier that is assigned to the first speaker (Poore: indicator 300 can include an identity, name, icon, symbol, graphic, etc. corresponding to the sound source, para. 0061).
As to claims 7, 16, Kim in view of Poore teaches: wherein the first content includes: a representation of an environment corresponding to the first device; and an indicator of the first position of the first speaker within the environment (Poore: see Figs. 5, 6).
Claims 6, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim in view of Poore, as applied to claim 1 above, and further in view of Lyren (US 2016/0240009 A1).
Kim in view of Poore differs from claim 6 in that although it teaches presenting an arrow to indicate the relative direction of the sound source (Poore: Fig. 6, para. 0062), it does not specifically teach: the content including text indicating at least one of a direction, a distance, or coordinates associated with the position of the speaker with respect to the first device.
Lyren teaches providing an indication that an image is not in the field of view of the device, the indication including text which provides notice or instructions where the object is located, distance to the object, etc. (para. 0179, 0181). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim in view of Poore with the above teaching of Lyren in order to provide a more detailed description regarding a sound source that is completely outside of the FOV of a capturing device.
Claim(s) 18, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim in view of Fleizach et al. (US 2024/0061547 A1, “Fleizach”).
Kim differs from claims 18, 20 in that it does not disclose: wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative Al operations; a system for performing operations using a large language model; a system for performing one or more conversational Al operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
Fleizach teaches a computer system, such as a portable device, tablet computer, notebook computer, etc. (para. 0006) which allows users to interact with a virtual or mixed reality environment (para. 0046) and visually indicates a location of a sound source inside or outside the view of the environment (para. 0013, 0051, 0416, 0468, 0472). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the portable computing device of Kim (col. 8, lines 58-63) with the above teaching of Fleizach in order to expand utility within a mixed or virtual environment.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stella L Woo whose telephone number is (571)272-7512. The examiner can normally be reached Monday - Friday, 8 a.m. to 5 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached at 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
STELLA L. WOO
Primary Examiner
Art Unit 2693
/Stella L. Woo/ Primary Examiner, Art Unit 2693
Prosecution Insights