DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims Filed: 03/21/2025
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/21/2025, 01/11/2026, and 06/11/2026 was filed after the mailing date of the 03/21/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1, 7-10, 16-18, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ota (US Publication) 20170277257 A1 in view of Fullam (US Publication) 20160080874 A1, in further view of Scavezze (US Publication) 20140320389 A1.
Regarding claim 1, Ota teaches An electronic device (Fig 1 [100], Head mounted display (HMD), para 17) comprising: a display (Fig 1 [102], display surface, para 17); a camera (Fig 1 [104], camera array, para 17); a speaker (Fig 9 [918], signal generation device, para 57. Though not shown); a microphone (Fig 1 [106], microphone, para 20) array including a plurality of microphones (In Fig 1, based on the direction of gaze of the user, one or more directional microphones [106] are used to discriminate a source of sound from the corresponding gaze, para 20);
memory (Fig 1 [922], machine-readable medium, para 58) comprising one or more storage media storing instructions (In Fig 9, instructions [924] may also reside, completely or at least partially, within the main memory [904], static memory [906], and/or within the processor [902], see para 58);
and at least one processor (Fig 1 [902], processor, para 58) comprising processing circuitry, wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device to:
identify, based on an image obtained by the camera, an external object viewable through the display (In Fig 6, an object (e.g. person) is identified based on the gaze direction [602], see para 35);
obtain, at least by controlling the microphone array based on a direction of the identified external object, an acoustic signal corresponding to the direction (In Fig 1, the HMD [100] includes multiple directional microphones [106] and based on the direction of the user’s gaze, these microphones are used to discriminate sound sources corresponding from the direction of user, see para 20);
in the display based on a position of the external object identified in the image (In Fig 4, the HMD [302] displays speech recognized text in a dialog box [400], which is positioned proximate to the person speaking; so that the user is given an intuitive user interface showing which person’s speech is being provided, see para 23);
and control to output, in response to received based on the visual object, an audio signal associated with the acoustic signal through the speaker (In Fig 6, the user may select the operation from a popup dialog box that appears in the augmented reality (AR) content or verbalize their selection with a voice command, see para 35).
Ota does not explicitly teach control to display, based on the acoustic signal, a visual object to adjust a volume of the acoustic signal
Scavezze discloses control to display, based on the acoustic signal, a visual object (Fig 3 [302], basketball, para 66) to adjust a volume of the acoustic signal (In Fig 3, the user [304] gazes at the basketball [332], the mixed reality interaction program (Fig 1 [14]) may interpret the user’s gaze to correspond with displaying over the basketball a virtual volume control knob. See para 66.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of control to display, based on the acoustic signal, a visual object to adjust a volume of the acoustic signal as taught by Scavezze in Ota’s invention. The motivation would have been to determine the identified physical object based on one or more aspects of the mixed reality environment. In para 2, in Scavezze.
Ota does not explicitly teach an input to adjust the volume
Fullam discloses an input to adjust the volume (In para 26, beamforming may additionally or alternatively include adjusting the amplitude of one or more signals output by the microphone array; specifically adjusted by an amount determined based on the relative distance and/or direction of the gaze target from the individual mics of the microphone arrays)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of an input to adjust the volume as taught by Fullam in Ota’s invention. The motivation would have been to emphasize sounds that originate near the gaze target determined by the eye tracker. In para 3, in Fullam.
Regarding claim 7, Ota teaches wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device (Fig 1 [100], HMD, para 17) to:
Ota does not explicitly teach control to output, in response to the input for increasing the volume, the audio signal, wherein the audio signal has a phase corresponding to a phase of the acoustic signal.
Fullam discloses control to output, in response to the input for increasing the volume, the audio signal, wherein the audio signal has a phase corresponding to a phase of the acoustic signal (In Fig 3, by adjusting the amplitude, the volume of the signals eventually output via the speakers may be adjusted, relative to each other. The amplitude adjustment may act to amplify or attenuate a particular signal. See para 26).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of control to output, in response to the input for increasing the volume, the audio signal, wherein the audio signal has a phase corresponding to a phase of the acoustic signal as taught by Fullam in Ota’s invention. The motivation would have been to emphasize sounds that originate near the gaze target determined by the eye tracker. In para 3, in Fullam.
Regarding claim 8, Ota teaches wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device (Fig 1 [100], HMD, para 17) to:
Ota does not explicitly teach output, in response to the input for decreasing the volume, the audio signal, which is corresponding to the acoustic signal, having a preset difference with respect to a phase of the acoustic signal.
Fullam discloses output, in response to the input for decreasing the volume, the audio signal, which is corresponding to the acoustic signal, having a preset difference with respect to a phase of the acoustic signal (In Fig 3, Step 314, the beamforming may include adjusting the phase of one or more of the signals output buy the microphone array. By adjusting the phase of the one or more signals, interference may occur, attenuating one or more signals. See para 25).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of output, in response to the input for decreasing the volume, the audio signal, which is corresponding to the acoustic signal, having a preset difference with respect to a phase of the acoustic signal as taught by Fullam in Ota’s invention. The motivation would have been to emphasize sounds that originate near the gaze target determined by the eye tracker. In para 3, in Fullam.
Regarding claim 9, Ota teaches further comprising, communication circuitry, wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device (Fig 1 [100], HMD, para 17) to:
Ota does not explicitly teach output a second audio signal different from the audio signal which is a first audio signal that is associated with a virtual object displayed based on information transmitted through the communication circuitry from an external electronic device different from the device; and control to display a third visual object to control the second audio signal associated with the virtual object that is different from the visual object which is a first visual object
Scavezze discloses output a second audio signal different from the audio signal which is a first audio signal that is associated with a virtual object displayed (In Fig 3, the mixed reality interaction program [14] may generate audio feedback that is perceived by the user [304] as originating from a virtual or physical object in the office [308], para 17) based on information transmitted through the communication circuitry from an external electronic device different from the device device (In Fig 3, the mixed reality interaction program (Fig 1 [14]) may generate audio comprising a broadcast of an Anytown Ants basketball game that is perceived by the user [304] as originating from the virtual website [352] or the basketball [332], see para 60. The external device being a computing device (Fig 1 [22]), that operatively connected to the HMD (Fig 3 [200]), [Para 69]; and the computing device can be a smart phone, laptop, tablet, etc.[Para 13]);
and control to display a third visual object to control the second audio signal associated with the virtual object that is different from the visual object which is a first visual object (In Fig 3, the user [304] gazes at the basketball [332], the mixed reality interaction program (Fig 1 [14]) may interpret the user’s gaze to correspond with displaying over the basketball a virtual volume control knob. See para 66. In para 65, the user may provide a user input to multiple virtual objects associated with multiple physical objects. In para 55, depending on the user’s input, the interaction program (Fig 1 [14]) may interpret the user’s orientation as corresponding to a virtual action. I would be obvious to a person of ordinary skill in the art that through routine experimentation would the designer have the option to control a third or multiple visual objects with their respective audio signals).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of output a second audio signal different from the audio signal which is a first audio signal that is associated with a virtual object displayed based on information transmitted through the communication circuitry from an external electronic device different from the device; and control to display a third visual object to control the second audio signal associated with the virtual object that is different from the visual object which is a first visual object as taught by Scavezze in Ota’s invention. The motivation would have been to determine the identified physical object based on one or more aspects of the mixed reality environment. In para 2, in Scavezze.
Regarding claim 10, Ota teaches A method (Fig 8 [800], para 46) of an electronic device (Fig 1 [100], Head mounted display (HMD), para 17) comprising:
identifying, based on an image obtained by a camera, an external object viewable through a display (In Fig 6, an object (e.g. person) is identified based on the gaze direction [602], see para 35);
obtaining, at least by controlling a microphone array (Fig 5 [512], microphone array, para 24) based on a direction of the identified external object, an acoustic signal corresponding to the direction (In Fig 1, the HMD [100] includes multiple directional microphones [106] and based on the direction of the user’s gaze, these microphones are used to discriminate sound sources corresponding from the direction of user, see para 20);
the display (Fig 1 [102], display surface, para 17) in based on a position of the external object identified in the image (In Fig 4, the HMD [302] displays speech recognized text in a dialog box [400], which is positioned proximate to the person speaking; so that the user is given an intuitive user interface showing which person’s speech is being provided, see para 23);
and outputting, in response to received based on the visual object, an audio signal associated with the acoustic signal through a speaker (In Fig 6, the user may select the operation from a popup dialog box that appears in the augmented reality (AR) content or verbalize their selection with a voice command, see para 35).
Ota does not explicitly teach displaying, based on the acoustic signal, a visual object to adjust a volume of the acoustic signal
Scavezze discloses displaying, based on the acoustic signal, a visual object (Fig 3 [302], basketball, para 66)to adjust a volume of the acoustic signal (In Fig 3, the user [304] gazes at the basketball [332], the mixed reality interaction program (Fig 1 [14]) may interpret the user’s gaze to correspond with displaying over the basketball a virtual volume control knob. See para 66.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of displaying, based on the acoustic signal, a visual object to adjust a volume of the acoustic signal as taught by Scavezze in Ota’s invention. The motivation would have been to determine the identified physical object based on one or more aspects of the mixed reality environment. In para 2, in Scavezze.
Ota does not explicitly teach an input to adjust the volume
Fullam discloses an input to adjust the volume (In para 26, beamforming may additionally or alternatively include adjusting the amplitude of one or more signals output by the microphone array; specifically adjusted by an amount determined based on the relative distance and/or direction of the gaze target from the individual mics of the microphone arrays)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of an input to adjust the volume as taught by Fullam in Ota’s invention. The motivation would have been to emphasize sounds that originate near the gaze target determined by the eye tracker. In para 3, in Fullam.
Regarding claim 16, Ota does not explicitly teach comprising: outputting, in response to the input for increasing the volume, the audio signal, the audio signal having a phase corresponding to a phase of the acoustic signal.
Fullam discloses comprising: outputting, in response to the input for increasing the volume, the audio signal, the audio signal having a phase corresponding to a phase of the acoustic signal (In Fig 3, by adjusting the amplitude, the volume of the signals eventually output via the speakers may be adjusted, relative to each other. The amplitude adjustment may act to amplify or attenuate a particular signal. See para 26).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of comprising: outputting, in response to the input for increasing the volume, the audio signal, the audio signal having a phase corresponding to a phase of the acoustic signal as taught by Fullam in Ota’s invention. The motivation would have been to emphasize sounds that originate near the gaze target determined by the eye tracker. In para 3, in Fullam.
Regarding claim 17, Ota does not explicitly teach comprising: outputting, in response to the input for decreasing the volume, the audio signal, which is corresponding to the acoustic signal, having a preset difference with respect to a phase of the acoustic signal.
Fullam discloses comprising: outputting, in response to the input for decreasing the volume, the audio signal, which is corresponding to the acoustic signal, having a preset difference with respect to a phase of the acoustic signal (In Fig 3, Step 314, the beamforming may include adjusting the phase of one or more of the signals output buy the microphone array. By adjusting the phase of the one or more signals, interference may occur, attenuating one or more signals. See para 25).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of comprising: outputting, in response to the input for decreasing the volume, the audio signal, which is corresponding to the acoustic signal, having a preset difference with respect to a phase of the acoustic signal as taught by Fullam in Ota’s invention. The motivation would have been to emphasize sounds that originate near the gaze target determined by the eye tracker. In para 3, in Fullam.
Regarding claim 18, Ota does not explicitly teach comprising: outputting a second audio signal different from the audio signal which is a first audio signal that is associated with a virtual object displayed based on information transmitted through communication circuitry from an external electronic device different from the electronic device; and displaying a third visual object to control the second audio signal associated with the virtual object that is different from the visual object which is a first visual object.
Scavezze discloses output a second audio signal different from the audio signal which is a first audio signal that is associated with a virtual object displayed (In Fig 3, the mixed reality interaction program [14] may generate audio feedback that is perceived by the user [304] as originating from a virtual or physical object in the office [308], para 17) based on information transmitted through the communication circuitry from an external electronic device different from the device (In Fig 3, the mixed reality interaction program (Fig 1 [14]) may generate audio comprising a broadcast of an Anytown Ants basketball game that is perceived by the user [304] as originating from the virtual website [352] or the basketball [332], see para 60. The external device being a computing device (Fig 1 [22]), that operatively connected to the HMD (Fig 3 [200]), [Para 69]; and the computing device can be a smart phone, laptop, tablet, etc.[Para 13]);
and control to display a third visual object to control the second audio signal associated with the virtual object that is different from the visual object which is a first visual object(In Fig 3, the user [304] gazes at the basketball [332], the mixed reality interaction program (Fig 1 [14]) may interpret the user’s gaze to correspond with displaying over the basketball a virtual volume control knob. See para 66. In para 65, the user may provide a user input to multiple virtual objects associated with multiple physical objects. In para 55, depending on the user’s input, the interaction program (Fig 1 [14]) may interpret the user’s orientation as corresponding to a virtual action. I would be obvious to a person of ordinary skill in the art that through routine experimentation would the designer have the option to control a third or multiple visual objects with their respective audio signals).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of comprising: outputting a second audio signal different from the audio signal which is a first audio signal that is associated with a virtual object displayed based on information transmitted through communication circuitry from an external electronic device different from the electronic device; and displaying a third visual object to control the second audio signal associated with the virtual object that is different from the visual object which is a first visual object as taught by Scavezze in Ota’s invention. The motivation would have been to determine the identified physical object based on one or more aspects of the mixed reality environment. In para 2, in Scavezze.
Regarding claim 19, Ota teaches A computer-readable storage medium storing one or more programs (In Fig 9, instructions [924] may also reside, completely or at least partially, within the main memory [904], static memory [906], and/or within the processor [902], see para 58), wherein the one or more programs, when executed individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57) by at least one processor (Fig 1 [902], processor, para 58) of an electronic device (Fig 1 [100], Head mounted display (HMD), para 17), cause the at least one processor of the electronic device individually and/or collectively to:
identify, based on an image obtained by a camera, an external object viewable through a display (In Fig 6, an object (e.g. person) is identified based on the gaze direction [602], see para 35);
obtain, by controlling a microphone array (Fig 5 [512], microphone array, para 24) based on a direction of the identified external object, an acoustic signal corresponding to the direction (In Fig 1, the HMD [100] includes multiple directional microphones [106] and based on the direction of the user’s gaze, these microphones are used to discriminate sound sources corresponding from the direction of user, see para 20);
in the display (Fig 1 [102], display surface, para 17) based on a position of the external object identified in the image (In Fig 4, the HMD [302] displays speech recognized text in a dialog box [400], which is positioned proximate to the person speaking; so that the user is given an intuitive user interface showing which person’s speech is being provided, see para 23);
and output, in response to received based on the visual object, an audio signal associated with the acoustic signal through a speaker (In Fig 6, the user may select the operation from a popup dialog box that appears in the augmented reality (AR) content or verbalize their selection with a voice command, see para 35).
Ota does not explicitly teach display, based on the acoustic signal, a visual object to adjust a volume of the acoustic signal
Scavezze discloses display, based on the acoustic signal, a visual object (Fig 3 [302], basketball, para 66) to adjust a volume of the acoustic signal (In Fig 3, the user [304] gazes at the basketball [332], the mixed reality interaction program (Fig 1 [14]) may interpret the user’s gaze to correspond with displaying over the basketball a virtual volume control knob. See para 66.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of display, based on the acoustic signal, a visual object to adjust a volume of the acoustic signalas taught by Scavezze in Ota’s invention. The motivation would have been to determine the identified physical object based on one or more aspects of the mixed reality environment. In para 2, in Scavezze.
Ota does not explicitly teach an input to adjust the volume
Fullam discloses an input to adjust the volume (In para 26, beamforming may additionally or alternatively include adjusting the amplitude of one or more signals output by the microphone array; specifically adjusted by an amount determined based on the relative distance and/or direction of the gaze target from the individual mics of the microphone arrays)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of an input to adjust the volume as taught by Fullam in Ota’s invention. The motivation would have been to emphasize sounds that originate near the gaze target determined by the eye tracker. In para 3, in Fullam.
Claim 2, 11, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ota (US Publication) 20170277257 A1 in view of Fullam (US Publication) 20160080874 A1, in further view of Scavezze (US Publication) 20140320389 A1, in further view of Mouncer (US Publication) 20200265860 A1
Regarding claim 2, Ota teaches wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device (Fig 1 [100], HMD, para 17) to:
Ota does not explicitly teach obtain, based on a beam having an azimuth angle corresponding to the identified position and formed by the microphone array, the acoustic signal
Mouncer discloses obtain, based on a beam having an azimuth angle corresponding to the identified position and formed by the microphone array, the acoustic signal (In Fig 3, the relative positioning identified at [Step 322] may be represented by an angle measured within the horizontal plane in relation to the mobile device within a range of 0-360 degrees and a distance between the mobile device and the targeted object within the horizontal plane [para 42]. It would be obvious to a person of ordinary skill in the art that the range of 0-360 degrees of the horizontal plane is synonymous of the azimuth angle).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of obtain, based on a beam having an azimuth angle corresponding to the identified position and formed by the microphone array, the acoustic signal as taught by Mouncer in Ota’s invention. The motivation would have been to increase a signal-to-noise ratio of an audio source within the beamforming region for the set of processed audio signals. In para 3, in Mouncer.
Regarding claim 11, Ota teaches comprising: obtaining, and formed by the microphone array (Fig 5 [512], microphone array, para 24), the acoustic signal.
Ota does not explicitly teach based on a beam having an azimuth angle corresponding to the identified position
Mouncer discloses based on a beam having an azimuth angle corresponding to the identified position (In Fig 3, the relative positioning identified at [Step 322] may be represented by an angle measured within the horizontal plane in relation to the mobile device within a range of 0-360 degrees and a distance between the mobile device and the targeted object within the horizontal plane [para 42]. It would be obvious to a person of ordinary skill in the art that the range of 0-360 degrees of the horizontal plane is synonymous of the azimuth angle)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of based on a beam having an azimuth angle corresponding to the identified position as taught by Mouncer in Ota’s invention. The motivation would have been to increase a signal-to-noise ratio of an audio source within the beamforming region for the set of processed audio signals. In para 3, in Mouncer.
Regarding claim 20, Ota teaches wherein the one or more programs, when executed by the at least one processor (Fig 1 [902], processor, para 58) of the electronic device (Fig 1 [100], HMD, para 17), cause the at least one processor of the electronic device to: and formed by the microphone array (Fig 5 [512], microphone array, para 24)
Ota does not explicitly teach obtain, based on a beam having an azimuth angle corresponding to the identified position, the acoustic signal
Mouncer discloses obtain, based on a beam having an azimuth angle corresponding to the identified position, the acoustic signal (In Fig 3, the relative positioning identified at [Step 322] may be represented by an angle measured within the horizontal plane in relation to the mobile device within a range of 0-360 degrees and a distance between the mobile device and the targeted object within the horizontal plane [para 42]. It would be obvious to a person of ordinary skill in the art that the range of 0-360 degrees of the horizontal plane is synonymous of the azimuth angle).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of obtain, based on a beam having an azimuth angle corresponding to the identified position, the acoustic signal as taught by Mouncer in Ota’s invention. The motivation would have been to increase a signal-to-noise ratio of an audio source within the beamforming region for the set of processed audio signals. In para 3, in Mouncer.
Claim 3-4, 6, 12-13, 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ota (US Publication) 20170277257 A1 in view of Fullam (US Publication) 20160080874 A1, in further view of Scavezze (US Publication) 20140320389 A1, in further view of Osterhout (US Publication) 20140063055 A1.
Regarding claim 3, Ota teaches further comprising communication circuitry, wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device (Fig 1 [100], HMD, para 17) to:
Ota does not explicitly teach in a state connected with a second external object through communication circuitry that is different from the external object which is a first external object, display, in association with the second external object viewable through the display,
Osterhout discloses in a state connected with a second external object through communication circuitry that is different from the external object which is a first external object, display, in association with the second external object viewable through the display (In para 75, a communication facility may be included that connects an external device (ED) to the eyepiece, and where the eyepiece may store specific user interfaces based on the (ED) type, wherein when the ED is connected to the eyepiece, the user interface is specifically presented to the user. In para 77, the external device may be an audio system, the user interface may be an audio system controller, the control scheme may be a head nod, and the like. It would be obvious to a person of ordinary skill in the art to recognize a visual object is specifically tied to and distinct for the type of device connected; and a different connected device gets a different UI element),
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of in a state connected with a second external object through communication circuitry that is different from the external object which is a first external object, display, in association with the second external object viewable through the display, a second visual object to control the second external object that is different from the visual object which is a first visual object as taught by Osterhout in Ota’s invention. The motivation would have been to handle content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment. In para 76, in Osterhout.
Ota in view of Osterhout fail to teach a second visual object to control the second external object that is different from the visual object which is a first visual object. However, Scavezze discloses in Fig 3, the user [304] using a displayed basketball as a virtual volume control knob, as interpreted via the mixed reality interaction program (Fig 1 [14]), para 66. Also, the computing device (Fig 1 [22]), being a smart phone, laptop, tablet, etc. [Para 13], is operatively connected to the HMD (Fig 3 [200]), [Para 69]. Based on this, it would be obvious to a person of ordinary skill in the art that through routine experimentation would the designer achieve two simultaneously displayed, independently tracked visual controls).
Regarding claim 4, Ota teaches wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device (Fig 1 [100], HMD, para 17) to:
Ota does not explicitly teach control to transmit, in response to an input to adjust a volume of the second external object received based on the second visual object, a control signal to adjust a volume of another audio signal outputted by the second external object, to the second external object through the communication circuitry.
Osterhout discloses control to transmit, in response to an input to adjust a volume of the second external object received based on the second visual object, a control signal to adjust a volume of another audio signal outputted by the second external object, to the second external object through the communication circuitry (In para 89, a communication facility may connect an external device to the earpiece, wherein when a condition is detected, a command & control interface for the command & control (C&C) of the external device may be presented in the eyepiece. The (C&C) interface may accept input from at least one of the user action capture device and the sensor, and the (C&C) interface may present feedback from the externa device in the eyepiece. See para 89).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of control to transmit, in response to an input to adjust a volume of the second external object received based on the second visual object, a control signal to adjust a volume of another audio signal outputted by the second external object, to the second external object through the communication circuitry as taught by Osterhout in Ota’s invention. The motivation would have been to handle content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment. In para 76, in Osterhout.
Regarding claim 6, Ota teaches wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device (Fig 1 [100], HMD, para 17) to:
Ota does not explicitly teach control to display the second visual object superimposed on at least a portion of an area where the second external object is identified.
Scavezze discloses control to display the second visual object (Fig 3 [322], basketball, para 60) superimposed on at least a portion of an area where the second external object is identified (In Fig 3, the MR interaction program (Fig 1 [14]), may generate audio comprising a broadcast of an Anytown Ants Basketball game that is perceived by the user [304] as originating from the virtual website [352] or a the basketball [332], see para 60. When the user gazes at the basketball, the program (Fig 1 [14]) may interpret the user’s gaze as using the displaying basketball as a virtual volume control knob, see para 66).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of obtain, based on a beam having an azimuth angle corresponding to the identified position and formed by the microphone array, the acoustic signal as taught by Scavezze in Ota’s invention. The motivation would have been to determine the identified physical object based on one or more aspects of the mixed reality environment. In para 2, in Scavezze.
Regarding claim 12, Ota does not explicitly teach comprising: in a state connected with a second external object through communication circuitry that is different from the external object which is a first external object, displaying, in association with the second external object viewable through the display,
Osterhout discloses comprising: in a state connected with a second external object through communication circuitry that is different from the external object which is a first external object, displaying, in association with the second external object viewable through the display (In para 75, a communication facility may be included that connects an external device (ED) to the eyepiece, and where the eyepiece may store specific user interfaces based on the (ED) type, wherein when the ED is connected to the eyepiece, the user interface is specifically presented to the user. In para 77, the external device may be an audio system, the user interface may be an audio system controller, the control scheme may be a head nod, and the like. It would be obvious to a person of ordinary skill in the art to recognize a visual object is specifically tied to and distinct for the type of device connected; and a different connected device gets a different UI element),
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of in a state connected with a second external object through communication circuitry that is different from the external object which is a first external object, display, in association with the second external object viewable through the display, a second visual object to control the second external object that is different from the visual object which is a first visual object as taught by Osterhout in Ota’s invention. The motivation would have been to handle content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment. In para 76, in Osterhout.
Ota in view of Osterhout fail to teach a second visual object to control the second external object that is different from the visual object which is a first visual object. However, Scavezze discloses in Fig 3, the user [304] using a displayed basketball as a virtual volume control knob, as interpreted via the mixed reality interaction program (Fig 1 [14]), para 66. Also, the computing device (Fig 1 [22]), being a smart phone, laptop, tablet, etc. [Para 13], is operatively connected to the HMD (Fig 3 [200]), [Para 69]. Based on this, it would be obvious to a person of ordinary skill in the art that through routine experimentation would the designer achieve two simultaneously displayed, independently tracked visual controls).
Regarding claim 13, Ota does not explicitly teach comprising: transmitting, in response to an input to adjust a volume of the second external object received based on the second visual object, a signal to adjust a volume of another audio signal outputted by the second external object, to the second external object through the communication circuitry.
Osterhout discloses comprising: transmitting, in response to an input to adjust a volume of the second external object received based on the second visual object, a signal to adjust a volume of another audio signal outputted by the second external object, to the second external object through the communication circuitry (In para 89, a communication facility may connect an external device to the earpiece, wherein when a condition is detected, a command & control interface for the command & control (C&C) of the external device may be presented in the eyepiece. The (C&C) interface may accept input from at least one of the user action capture device and the sensor, and the (C&C) interface may present feedback from the externa device in the eyepiece. See para 89).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of comprising: transmitting, in response to an input to adjust a volume of the second external object received based on the second visual object, a signal to adjust a volume of another audio signal outputted by the second external object, to the second external object through the communication circuitry as taught by Osterhout in Ota’s invention. The motivation would have been to handle content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment. In para 76, in Osterhout.
Regarding claim 15, Ota does not explicitly teach further comprising: displaying the second visual object superimposed on at least a portion of an area where the second external object is identified.
Scavezze discloses further comprising: displaying the second visual object (Fig 3 [322], basketball, para 60) superimposed on at least a portion of an area where the second external object is identified (In Fig 3, the MR interaction program (Fig 1 [14]), may generate audio comprising a broadcast of an Anytown Ants Basketball game that is perceived by the user [304] as originating from the virtual website [352] or a the basketball [332], see para 60. When the user gazes at the basketball, the program (Fig 1 [14]) may interpret the user’s gaze as using the displaying basketball as a virtual volume control knob, see para 66).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of further comprising: displaying the second visual object superimposed on at least a portion of an area where the second external object is identified as taught by Scavezze in Ota’s invention. The motivation would have been to determine the identified physical object based on one or more aspects of the mixed reality environment. In para 2, in Scavezze.
Claim 5 are rejected under 35 U.S.C. 103 as being unpatentable over Ota (US Publication) 20170277257 A1 in view of Fullam (US Publication) 20160080874 A1 , in further view of Scavezze (US Publication) 20140320389 A1, in further view of Osterhout (US Publication) 20140063055 A1, in further view of Poore (US Publication) 20210055367 A1.
Regarding claim 5, Ota teaches wherein the instructions, when executed by the at least one processor individually and/or collectively (In Fig 9, the processor [902] is communication with other components through the link [908], see para 55, 57), cause the electronic device (Fig 1 [100], HMD, para 17) to:
Ota does not explicitly teach control to display the visual object superimposed on at least a portion of an area where the external object is identified.
Osterhout discloses control to display the visual (In para 75, a communication facility may be included that connects an external device (ED) to the eyepiece, and where the eyepiece may store specific user interfaces based on the (ED) type, wherein when the ED is connected to the eyepiece, the user interface is specifically presented to the user. In para 77, the external device may be an audio system, the user interface may be an audio system controller, the control scheme may be a head nod, and the like).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of control to display the visual as taught by Osterhout in Ota’s invention. The motivation would have been to handle content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment. In para 76, in Osterhout.
Ota does not explicitly teach object superimposed on at least a portion of an area where the external object is identified
Poore discloses object superimposed on at least a portion of an area where the external object is identified (In Fig 3, a sound [20] of a sound [30] can be located within or outside field of view [90], para 54. In Fig 4, an indicator [300] can be visually output by the display [190] to indicate the location of the source [20], para 56. In Fig 6, even though the source is outside field of view, the indicator will display the location of the source, para 60. It would be obvious to a person of ordinary skill in the art to recognize that visual object is fixed to the target).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of object superimposed on at least a portion of an area where the external object is identified as taught by Poore in Ota’s invention. The motivation would have been to detect audio inputs can be rendered with audio outputs that emphasize (e.g., amplify) certain sounds over others to help the user distinguish between different sounds. In para 16, in Poore.
Claim 14 are rejected under 35 U.S.C. 103 as being unpatentable over Ota (US Publication) 20170277257 A1 in view of Fullam (US Publication) 20160080874 A1, in further view of Scavezze (US Publication) 20140320389 A1, in further view of Poore (US Publication) 20210055367 A1.
Regarding claim 14, Ota does not explicitly teach comprising: displaying the visual object superimposed on at least a portion of an area where the external object is identified.
Poore discloses comprising: displaying the visual object superimposed on at least a portion of an area where the external object is identified (In Fig 3, a sound [20] of a sound [30] can be located within or outside field of view [90], para 54. In Fig 4, an indicator [300] can be visually output by the display [190] to indicate the location of the source [20], para 56. In Fig 6, even though the source is outside field of view, the indicator will display the location of the source, para 60. It would be obvious to a person of ordinary skill in the art to recognize that visual object is fixed to the target).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the teachings of comprising: displaying the visual object superimposed on at least a portion of an area where the external object is identified as taught by Poore in Ota’s invention. The motivation would have been to detect audio inputs can be rendered with audio outputs that emphasize (e.g., amplify) certain sounds over others to help the user distinguish between different sounds. In para 16, in Poore.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
Wang (US patent) 11361735 B1 – tracking virtual objects
Di Censo (US publication) 20150025662 A1 – volume control for different sound sources
McCulloch (US publication) 20140337023 A1 – device layout
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARCUS A BARBOZA whose telephone number is (571)272-9626. The examiner can normally be reached Monday-Friday 7:30 am to 5 pm, Alternate Fridays: off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carolyn R. Edwards can be reached at 571-270-7136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARCUS A BARBOZA/Examiner, Art Unit 2692
/CAROLYN R EDWARDS/ Supervisory Patent Examiner, Art Unit 2692
Prosecution Insights