DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the response to this office action, the Examiner respectfully requests that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line numbers in the specification and/or drawing figure(s). This will assist the Examiner in prosecuting this application.
Priority
2. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
3. The information disclosure statement filed on October 12, 2024 has been considered and placed in the application file.
Claim Rejections - 35 USC § 103
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
7. Claims 1-2, and 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Sheaffer et al. U.S. Patent Application Publication 20220053281 (hereinafter, “Sheaffer”) in view of Skoglund et al. U.S. Patent Application Publication 20190132685 (hereinafter, “Skoglund”, cited by Applicant).
Regarding claim 1, Sheaffer teaches a sound source position determination method (In one aspect, wearable devices that include a microphone array may perform beamforming operations (using beamforming algorithms) to spatially select sound sources within an environment in which the wearable device is located. Such spatial selectivity provides for an improved audio signal that includes sound from a particular sound source (par [0032], see Sheaffer)), which is applied to a head-mounted device (FIG. 1 shows a wearable audio device 100 (hereafter referred to as “wearable device 100”) that is illustrated as a pair of over-the-ear headphones, Fig. 1, par [0026], see Sheaffer), wherein the head-mounted device comprises at least four microphones (Coupled to (or integrated into) the headband and/or housings are several microphones 110a-110d (i.e., four microphone) and several loudspeakers 105a-105d, see Fig. 1, par [0026], see Sheaffer), and the sound source position determination method comprises:
determining microphone coordinates (e.g., a given physical arrangement; a location in space) of the microphones in a space coordinate system (FIGS. 2A-2B show measurements of far-field transfer functions and near-field transfer functions of microphones for a given physical arrangement of a microphone array. In one aspect, at least one of the measurements is performed in a controlled environment (e.g., a laboratory) for a given physical arrangement of a microphone array (par [0036], see Sheaffer). To achieve accurate spatial selectivity, far-field transfer functions (far-field responses) that represent an acoustic transmission path between a location in space and locations of elements that compose a beamforming array are required in order to produce (e.g., properly steer) an expected directional beam pattern of the array towards the location, par [0033], see Sheaffer; The memory includes instructions which when executed by the processor causes the wearable device to obtain a microphone signal from each of the microphones in the microphone array, apply a far-field transfer function to the microphone signal that represents a response between the microphone and a position in space (i.e., coordinates of the microphones in a space coordinate system), determine a current physical arrangement of the microphone array based on at least one of the image data, the sensor data, and the microphone signals, and select at least one different far-field transfer function to be applied to a corresponding microphone signal according to the current physical arrangement, par [0097], see Sheaffer) according to a device coordinate of a preset sound source device (from loudspeaker 215, Fig. 2A; This figure illustrates the measurement of far-field transfer function (far-field response) in the discrete domain, H.sub.F(z), for each of the microphones with respect to loudspeaker 215, Fig. 2A, par [0038], see Sheaffer), wherein the space coordinate system is established in a space where the head-mounted device is located (In one aspect, similar to the microphone array, the loudspeaker array of the wearable device 100 may rely on far-field transfer functions to produce an expected directional sound beam pattern towards a particular location in space, par [0038], see Sheaffer).
Sheaffer further teaches in another aspect, as described herein, a “transfer function” represents a travel path (or response) between an audio element, such as a microphone, and a sound source, such as a loudspeaker, in an acoustic space (par [0025], see Sheaffer).
However, Sheaffer does not explicitly disclose determining time differences between receiving time points at which the microphones receive audio signals sent by a sound source to be positioned; and determining a sound source coordinate of the sound source to be positioned in the space coordinate system according to the microphone coordinates and the time differences.
Skoglund teaches hearing system configured to localize a target sound source (see Title) in which determining time differences between receiving time points at which the microphones receive audio signals sent by a sound source to be positioned (where S.sup.e represent the position of said sound source in an inertial frame of reference, R.sub.t and T.sub.t.sup.e are matrices describing a rotation and a translation, respectively, of the sensor array with respect to the inertial frame at time t, and y.sub.t.sup.ij=τ.sub.ij+e.sub.t represent said sensor array configuration specific data, where τ.sub.ij represent said differences between a time of arrival of sound from said localized sound source S at said respective input transducers i, j, and e.sub.t represents measurement noise, where (i,j)=1, . . . , M, j>i, wherein h.sub.ij is a model of the time differences τ.sub.ij between each microphone pair p.sub.i and p.sub.j, see par [0023], see Skoglund); and determining a sound source coordinate of the sound source to be positioned in the space coordinate system (By localizing the sound sources around the user (e.g. using SLAM), an impression of the original positions of the sound sources can be ‘recreated’ by applying standardized head related transfer functions (HRTFs). Since we know where in space the sources are (e.g. via SLAM), we can project the different sources to their ‘original’ positions when we present the sound to the left and right ear, see par [0041], see Skoglund) according to the microphone coordinates (FIG. 1B shows a sound source S located in a three dimensional coordinate system (x, y, z) relative to a microphone array comprising two microphones (mic.sub.1, mic.sub.2) located a distance d=2a apart on the x-axis symmetrically around origo (0, 0, 0) of the coordinate system (i.e. centred in (a, 0, 0) and (−a, 0, 0), respectively, Fig. 1B, par [0117], see Skoglund) and the time differences (When the sources are not perpendicular to the array, the distance between the sensors and the source will be different resulting in a time difference in the received signals. With known speed of the medium (here e.g. air), the time difference can be converted to a distance and with known separation between the sensors, the angle to the source can be calculated, par [0119], see Skoglund).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the hearing system configured to localize a target sound source taught by Skoglund with the method of Sheaffer such that to obtain determining time differences between receiving time points at which the microphones receive audio signals sent by a sound source to be positioned; and determining a sound source coordinate of the sound source to be positioned in the space coordinate system according to the microphone coordinates and the time differences in order to use this localization to improve the processing of the hearing device, as suggested by Skoglund in paragraph [0195].
Regarding claim 2, Sheaffer in view of Skoglund teaches the method of claim 1. Sheaffer in view of Skoglund, as modified, teaches wherein the preset sound source device comprises at least four loudspeakers (105a-105d, Fig. 1, see Sheaffer) disposed on the head-mounted device (Fig. 1, par [0026], see Sheaffer), and the determining the microphone coordinates of the microphones in the space coordinate system (FIGS. 2A-2B show measurements of far-field transfer functions and near-field transfer functions of microphones for a given physical arrangement of a microphone array. In one aspect, at least one of the measurements is performed in a controlled environment (e.g., a laboratory) for a given physical arrangement of a microphone array (par [0036], see Sheaffer). To achieve accurate spatial selectivity, far-field transfer functions (far-field responses) that represent an acoustic transmission path between a location in space and locations of elements that compose a beamforming array are required in order to produce (e.g., properly steer) an expected directional beam pattern of the array towards the location, par [0033], see Sheaffer; The memory includes instructions which when executed by the processor causes the wearable device to obtain a microphone signal from each of the microphones in the microphone array, apply a far-field transfer function to the microphone signal that represents a response between the microphone and a position in space (i.e., coordinates of the microphones in a space coordinate system), determine a current physical arrangement of the microphone array based on at least one of the image data, the sensor data, and the microphone signals, and select at least one different far-field transfer function to be applied to a corresponding microphone signal according to the current physical arrangement, par [0097], see Sheaffer) according to the device coordinate of the preset sound source device (from loudspeaker 215, Fig. 2A; This figure illustrates the measurement of far-field transfer function (far-field response) in the discrete domain, H.sub.F(z), for each of the microphones with respect to loudspeaker 215, Fig. 2A, par [0038], see Sheaffer) comprises:
determining device coordinates of the loudspeakers respectively (In response to the wearable device 100 changing its physical shape, the physical arrangement of the microphone array and/or the loudspeaker array may also change, since the microphones and loudspeakers are integrated into the wearable device 100, Fig. 1, par [0030], see Sheaffer);
controlling the loudspeakers (via microphone array calibrator 430, Fig. 4) to emit first calibration audio signals respectively (The microphone array calibrator 430 sends the retrieved audio signal to the loudspeaker 415, which then outputs the audio signal, Fig. 4, par [0061], see Sheaffer);
determining first calibration time differences between first calibration receiving time points (The second processor may be configured to estimate data indicative of a location of said localized sound source S relative to the user based on the following expression for stacked residual vectors r(S.sup.e) originating from said time instances t=1, . . . , N, (i.e., t=1 first calibration time), see par [0022], see Skoglund) at which the microphones receive the first calibration audio signals respectively (where S.sup.e represent the position of said sound source in an inertial frame of reference, R.sub.t and T.sub.t.sup.e are matrices describing a rotation and a translation, respectively, of the sensor array with respect to the inertial frame at time t, and y.sub.t.sup.ij=τ.sub.ij+e.sub.t represent said sensor array configuration specific data, where τ.sub.ij represent said differences between a time of arrival of sound from said localized sound source S at said respective input transducers i, j, and e.sub.t represents measurement noise, where (i,j)=1, . . . , M, j>i, wherein h.sub.ij is a model of the time differences τ.sub.ij between each microphone pair p.sub.i and p.sub.j, see par [0023], see Skoglund); and
determining the microphone coordinates according to the first calibration time differences (FIG. 1B shows a sound source S located in a three dimensional coordinate system (x, y, z) relative to a microphone array comprising two microphones (mic.sub.1, mic.sub.2) located a distance d=2a apart on the x-axis symmetrically around origo (0, 0, 0) of the coordinate system (i.e. centred in (a, 0, 0) and (−a, 0, 0), respectively, (Fig. 1B, par [0117], see Skoglund); with known speed of the medium (here e.g. air), the time difference can be converted to a distance and with known separation between the sensors, the angle to the source can be calculated, (par [0119], see Skoglund)) and the device coordinates (since the microphones integrated with the device; In an embodiment, the hearing device comprises a forward or signal path between an input unit (e.g. an input transducer, such as a microphone or a microphone system and/or direct electric input (e.g. a wireless receiver)) and an output unit, e.g. an output transducer, par [0057], see Skoglund ). The motivation is in order to use this localization to improve the processing of the hearing device, as suggested by Skoglund in paragraph [0195].
Regarding claim 9, Sheaffer in view of Skoglund teaches a head-mounted device (In one aspect, the wearable device may be a head-mounted display (HMD), par [0028], see Sheaffer), comprising: a memory, a processor, and a sound source position determination program stored in the memory and executable on the processor, wherein the sound source position determination program (The memory includes instructions which when executed by the processor causes the wearable device to obtain a microphone signal from each of the microphones in the microphone array, apply a far-field transfer function to the microphone signal that represents a response between the microphone and a position in space, determine a current physical arrangement of the microphone array based on at least one of the image data, the sensor data, and the microphone signal, par [0097], see Sheaffer), when executed by the processor, implements steps of the sound source position determination method of claim 1, and is therefore rejected under Sheaffer in view of Skoglund for the same reasons.
Regarding claim 10, Sheaffer in view of Skoglund teaches a non-transitory storage medium, on which a sound source position determination program is stored, wherein when the sound source position determination program is executed by a processor (As previously explained, an aspect of the disclosure may be a non-transitory machine-readable medium (such as microelectronic memory) having stored thereon instructions, which program one or more data processing components (generically referred to here as a “processor”) to perform the audio signal processing operations and sound pickup operations, par [0098], see Sheaffer), steps of the sound source position determination method of claim 1 are implemented, and is therefore rejected under Sheaffer in view of Skoglund for the same reasons.
8. Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Sheaffer et al. U.S. Patent Application Publication 20220053281 (hereinafter, “Sheaffer”) in view of Skoglund et al. U.S. Patent Application Publication 20190132685 (hereinafter, “Skoglund”, cited by Applicant), and further in view of Raykar et al. “Position calibration of microphones and loudspeakers in distributed computing platforms” IEEE transactions on January 2005 - ieeexplore.ieee.org, pages 70-83 (hereinafter, “Raykar”).
Regarding claim 3, Sheaffer in view of Skoglund teaches the method of claim 2. Sheaffer in view of Skoglund, as modified, teaches wherein the determining the device coordinates of the loudspeakers respectively (In response to the wearable device 100 changing its physical shape, the physical arrangement of the microphone array and/or the loudspeaker array may also change, since the microphones and loudspeakers are integrated into the wearable device 100, Fig. 1, par [0030], see Sheaffer) comprises:
establishing the space coordinate system based on the microphones, and obtaining initial microphone coordinates of the microphones in the space coordinate system respectively (FIGS. 2A-2B show measurements of far-field transfer functions and near-field transfer functions of microphones for a given physical arrangement of a microphone array. In one aspect, at least one of the measurements is performed in a controlled environment (e.g., a laboratory) for a given physical arrangement of a microphone array (par [0036], see Sheaffer). To achieve accurate spatial selectivity, far-field transfer functions (far-field responses) that represent an acoustic transmission path between a location in space and locations of elements that compose a beamforming array are required in order to produce (e.g., properly steer) an expected directional beam pattern of the array towards the location, par [0033], see Sheaffer; The memory includes instructions which when executed by the processor causes the wearable device to obtain a microphone signal from each of the microphones in the microphone array, apply a far-field transfer function to the microphone signal that represents a response between the microphone and a position in space (i.e., coordinates of the microphones in a space coordinate system), determine a current physical arrangement of the microphone array based on at least one of the image data, the sensor data, and the microphone signals, and select at least one different far-field transfer function to be applied to a corresponding microphone signal according to the current physical arrangement, par [0097], see Sheaffer);
controlling the loudspeakers (via microphone array calibrator 430, Fig. 4) to emit second calibration audio signals respectively (The microphone array calibrator 430 sends the retrieved audio signal to the loudspeaker 415, which then outputs the audio signal, Fig. 4, par [0061], see Sheaffer), and determining second calibration time differences between second calibration receiving time points (The second processor may be configured to estimate data indicative of a location of said localized sound source S relative to the user based on the following expression for stacked residual vectors r(S.sup.e) originating from said time instances t=1, . . . , N, (i.e., t=2 second calibration time), see par [0022], see Skoglund) at which the microphones receive the second calibration audio signals respectively (where S.sup.e represent the position of said sound source in an inertial frame of reference, R.sub.t and T.sub.t.sup.e are matrices describing a rotation and a translation, respectively, of the sensor array with respect to the inertial frame at time t, and y.sub.t.sup.ij=τ.sub.ij+e.sub.t represent said sensor array configuration specific data, where τ.sub.ij represent said differences between a time of arrival of sound from said localized sound source S at said respective input transducers i, j, and e.sub.t represents measurement noise, where (i,j)=1, . . . , M, j>i, wherein h.sub.ij is a model of the time differences τ.sub.ij between each microphone pair p.sub.i and p.sub.j, see par [0023], see Skoglund).
However, Sheaffer in view of Skoglund does not explicitly disclose determining the device coordinates of the loudspeakers respectively according to the second calibration time differences and the initial microphone coordinates.
Raykar teaches position calibration of microphones and loudspeakers in distributed computing platforms (see Title) in which The total number of observations should be greater than or equal to the total number of parameters to be estimated. This
defines a minimum number of microphones and speakers required for the position estimation method to work. Assuming we have M microphones and Speakers, Table I summarizes the number of independent observations (N) and the number of parameters to be estimated (P) in each of the estimation procedures (e.g., second calibration time differences). In case of the TDOF-based method only (M-1)S out of M(M-1)S/2 pair of TDOF measurements are linearly independent. In Table I, TOF/TDOF Position refers to the case where we are estimating only the positions of the microphones and speakers, i.e., the TOF/TDOF is not corrupted by the capture and the emission start times. TOF/TDOF Joint refers to the case where we are jointly estimating the emission and capture start times along with the microphone and speaker coordinates. Assuming M =S =1, the Table II lists the minimum required for the estimation procedure. Assuming each GPC has one microphone and one speaker this gives the minimum number of GPCs required (see page 74, left column, last paragraph to right column, first paragraph, see Raykar).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the position calibration of microphones and loudspeakers in distributed computing platform taught by Raykar with the method of Sheaffer in view of Skoglund such that to obtain determining the device coordinates of the loudspeakers respectively according to the second calibration time differences and the initial microphone coordinates for purpose of providing multiple GPCs along with their sensors and actuators can be converted to a distributed sensor network in an ad-hoc fashion by just adding appropriate software layers, as suggested by Raykar in page 70, right column first paragraph.
9. Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Sheaffer et al. U.S. Patent Application Publication 20220053281 (hereinafter, “Sheaffer”) in view of Skoglund et al. U.S. Patent Application Publication 20190132685 (hereinafter, “Skoglund”, cited by Applicant), and further in view of Mao et al. U.S. Patent Application Publication 20080247566 (hereinafter, “Mao”)
Regarding claim 5, Sheaffer in view of Skoglund teaches the method of claim 1. Sheaffer in view of Skoglund, as modified, teaches determining a sound source coordinate of the sound source to be positioned in the space coordinate system (By localizing the sound sources around the user (e.g. using SLAM), an impression of the original positions of the sound sources can be ‘recreated’ by applying standardized head related transfer functions (HRTFs). Since we know where in space the sources are (e.g. via SLAM), we can project the different sources to their ‘original’ positions when we present the sound to the left and right ear, see par [0041], see Skoglund) according to the microphone coordinates (FIG. 1B shows a sound source S located in a three dimensional coordinate system (x, y, z) relative to a microphone array comprising two microphones (mic.sub.1, mic.sub.2) located a distance d=2a apart on the x-axis symmetrically around origo (0, 0, 0) of the coordinate system (i.e. centred in (a, 0, 0) and (−a, 0, 0), respectively, Fig. 1B, par [0117], see Skoglund) and the time differences (When the sources are not perpendicular to the array, the distance between the sensors and the source will be different resulting in a time difference in the received signals. With known speed of the medium (here e.g. air), the time difference can be converted to a distance and with known separation between the sensors, the angle to the source can be calculated, par [0119], see Skoglund).
However, Sheaffer in view of Skoglund does not explicitly disclose setting a minimum flying time in which the audio signals reach one of the microphones; constructing equations according to a sound transmission speed, the minimum flying time, the microphone coordinates and the time differences; and determining the sound source coordinate of the sound source to be positioned in the space coordinate system according to solution results of the equations.
Mao teaches sound source localization system and sound source localization method (see Title) in which setting a minimum flying time in which the audio signals reach one of the microphones; constructing equations according to a sound transmission speed, the minimum flying time, the microphone coordinates and the time differences (The arithmetic unit 120 performs a cross spectrum process according to the frequency domain signals T(1) to T(n) to determine time differences of arrival (t.sub.2-t.sub.1) to (t.sub.n-t.sub.1) between the time instants when the wave front of the sound source S enters the sound capturing devices 110(2) to 110(n) and the time instant when the wave front of the sound source S enters the sound capturing device 110(1), and locates the sound source S according to the time differences of arrival (t.sub.2-t.sub.1) to (t.sub.n-t.sub.1), locations of the sound capturing devices 110(1) to 110(n) and the sound velocity c, see Fig. 3, par [0022], see Mao); and determining the sound source coordinate of the sound source to be positioned in the space coordinate system according to solution results of the equations (For example, the coordinates of the sound source S, the sound capturing device 110(1), the sound capturing device 110(2) and the sound capturing device 110(3) are respectively (x.sub.s,y.sub.s), (x.sub.1,y.sub.1), (x.sub.2,y.sub.2) and (x.sub.3,y.sub.3). Substituting the coordinates into the distance formula can get the distances from the sound source S to the sound capturing devices 110(1) to 110(3), see Fig. 3, par [0027], see Mao).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the sound source localization system and sound source localization method taught by Mao with the method of Sheaffer in view of Skoglund such that to obtain setting a minimum flying time in which the audio signals reach one of the microphones; constructing equations according to a sound transmission speed, the minimum flying time, the microphone coordinates and the time differences; and determining the sound source coordinate of the sound source to be positioned in the space coordinate system according to solution results of the equations in order to provide a different localization system to improve the drawback caused in the conventional image localization system, as suggested by Mao in paragraph [0006].
10. Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Sheaffer et al. U.S. Patent Application Publication 20220053281 (hereinafter, “Sheaffer”) in view of Skoglund et al. U.S. Patent Application Publication 20190132685 (hereinafter, “Skoglund”, cited by Applicant), and further in view of Poore et al. U.S. Patent Application Publication 20210055367 (hereinafter, “Poore”).
Regarding claim 6, Sheaffer in view of Skoglund teaches the method of claim 1. Sheaffer in view of Skoglund, as modified, teaches further comprising: after (The source direction then has two degrees of freedom (DOF), par [0124], see Skoglund) the determining a sound source coordinate of the sound source to be positioned in the space coordinate system (By localizing the sound sources around the user (e.g. using SLAM), an impression of the original positions of the sound sources can be ‘recreated’ by applying standardized head related transfer functions (HRTFs). Since we know where in space the sources are (e.g. via SLAM), we can project the different sources to their ‘original’ positions when we present the sound to the left and right ear, see par [0041], see Skoglund) according to the microphone coordinates (FIG. 1B shows a sound source S located in a three dimensional coordinate system (x, y, z) relative to a microphone array comprising two microphones (mic.sub.1, mic.sub.2) located a distance d=2a apart on the x-axis symmetrically around origo (0, 0, 0) of the coordinate system (i.e. centred in (a, 0, 0) and (−a, 0, 0), respectively, Fig. 1B, par [0117], see Skoglund) and the time differences (When the sources are not perpendicular to the array, the distance between the sensors and the source will be different resulting in a time difference in the received signals. With known speed of the medium (here e.g. air), the time difference can be converted to a distance and with known separation between the sensors, the angle to the source can be calculated, par [0119], see Skoglund).
However, Sheaffer in view of Skoglund does not explicitly disclose determining a relative position coordinate of the sound source to be positioned with respect to the head-mounted device according to the sound source coordinate and the microphone coordinates; determining play parameters corresponding to the audio signals according to the relative position coordinate, and playing the audio signals based on the play parameters; and/or indicating a position where the sound source to be positioned is located in a reality environment image displayed by the head-mounted device according to the relative position coordinate.
Poore teaches audio-based feedback for head-mountable device (see Title) in which determining a relative position coordinate of the sound source to be positioned with respect to the head-mounted device according to the sound source coordinate and the microphone coordinates (In particular, a head-mountable device can be provided with multiple microphones for capturing audio information (e.g., sounds) from multiple sources that are located in different directions with respect to the head-mountable device. Multiple microphones distributed across the head-mountable device can provide directional audio detection. The head-mountable device can use the data collected by the microphones to provide visual and/or audio outputs to the user, par [0016], see Poore); determining play parameters corresponding to the audio signals according to the relative position coordinate, and playing the audio signals based on the play parameters; and/or indicating a position where the sound source to be positioned is located in a reality environment image displayed by the head-mounted device according to the relative position coordinate (Additionally or alternatively, the indicator 300 can include visual features such as color, highlighting, glowing, outlines, shadows, or other contrasting features that allow portions thereof to be more distinctly visible when displayed along with the view to the external environment and/or objects therein. The indicator 300 can move across the display 190 as the user moves the head-mountable device to change the field-of-view being captured and/or displayed. For example, the indicator 300 can maintain its position with respect to the source 20 as the source 20 moves within the display 190 due to the user's movement, see Fig. 4, par [0057], see Poore).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the sound source localization system and sound source localization method taught by Poore with the method of Sheaffer in view of Skoglund such that to obtain determining a relative position coordinate of the sound source to be positioned with respect to the head-mounted device according to the sound source coordinate and the microphone coordinates; determining play parameters corresponding to the audio signals according to the relative position coordinate, and playing the audio signals based on the play parameters; and/or indicating a position where the sound source to be positioned is located in a reality environment image displayed by the head-mounted device according to the relative position coordinate in order to provide a user may observe outputs provided by the head-mountable device, such as visual information provided on a display, as suggested by Poore in paragraph [0003].
11. Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Sheaffer et al. U.S. Patent Application Publication 20220053281 (hereinafter, “Sheaffer”) in view of Skoglund et al. U.S. Patent Application Publication 20190132685 (hereinafter, “Skoglund”, cited by Applicant), and further in view of Walsh et al. U.S. Patent Application Publication 20150016642 (hereinafter, “Walsh”).
Regarding claim 8, Sheaffer in view of Skoglund teaches the method of claim 1. Sheaffer in view of Skoglund, as modified, teaches wherein the preset sound source device comprises at least four loudspeakers (105a-105d, Fig. 1, see Sheaffer) with device coordinates which are known previously (FIG. 1B shows a sound source S located in a three dimensional coordinate system (x, y, z) relative to a microphone array comprising two microphones (mic.sub.1, mic.sub.2) located a distance d=2a apart on the x-axis symmetrically around origo (0, 0, 0) of the coordinate system (i.e. centred in (a, 0, 0) and (−a, 0, 0)), respectively, (Fig. 1B, par [0117], see Skoglund)), and the determining microphone coordinates (e.g., a given physical arrangement; a location in space) of the microphones in a space coordinate system (FIGS. 2A-2B show measurements of far-field transfer functions and near-field transfer functions of microphones for a given physical arrangement of a microphone array. In one aspect, at least one of the measurements is performed in a controlled environment (e.g., a laboratory) for a given physical arrangement of a microphone array (par [0036], see Sheaffer). To achieve accurate spatial selectivity, far-field transfer functions (far-field responses) that represent an acoustic transmission path between a location in space and locations of elements that compose a beamforming array are required in order to produce (e.g., properly steer) an expected directional beam pattern of the array towards the location, par [0033], see Sheaffer; The memory includes instructions which when executed by the processor causes the wearable device to obtain a microphone signal from each of the microphones in the microphone array, apply a far-field transfer function to the microphone signal that represents a response between the microphone and a position in space (i.e., coordinates of the microphones in a space coordinate system), determine a current physical arrangement of the microphone array based on at least one of the image data, the sensor data, and the microphone signals, and select at least one different far-field transfer function to be applied to a corresponding microphone signal according to the current physical arrangement, par [0097], see Sheaffer) according to a device coordinate of a preset sound source device (from loudspeaker 215, Fig. 2A; This figure illustrates the measurement of far-field transfer function (far-field response) in the discrete domain, H.sub.F(z), for each of the microphones with respect to loudspeaker 215, Fig. 2A, par [0038], see Sheaffer).
However, Sheaffer in view of Skoglund does not explicitly disclose further comprises: obtaining device coordinates of the loudspeakers respectively; controlling the loudspeakers to emit calibration audio signals respectively; determining calibration time differences between calibration receiving time points at which the microphones receive the calibration audio signals respectively; and determining the microphone coordinates according to the calibration time differences and the device coordinates.
Walsh teaches spatial calibration of surround sound systems including listener position estimation (see Title) in which obtaining device coordinates of the loudspeakers respectively (Based on the estimated distance d and angle .theta., the position estimator 430 can compute the coordinates of the loudspeaker using trigonometry, Fig. 4, par [0051], see Walsh); controlling the loudspeakers to emit calibration audio signals respectively (In one embodiment, the distance between a loudspeaker and a microphone is estimated by playing a test signal and measuring the time of flight (TOF) between the emitting loudspeaker and the receiving microphone, par [0048], see Walsh); determining calibration time differences between calibration receiving time points at which the microphones receive the calibration audio signals respectively (In some embodiments, the position of the loudspeaker and the position of the listener each includes a distance and an angle relative to the microphone array, wherein the position of the loudspeaker is estimated based on a direct component of the received test signal, and wherein the angle of the loudspeaker is estimated using two or more microphones in the microphone array and based on a time difference of arrival (TDOA) of the test signal at the two or more microphones in the microphone array, par [0009], see Walsh); and determining the microphone coordinates according to the calibration time differences and the device coordinates (One solution for such a problem, generally known as spatial calibration, typically requires a user to place a microphone array at the default listening position (or sweet spot). By approximating the location of each loudspeaker, the system can spatially reformat a multichannel soundtrack to the actual speaker layout. However, this calibration process can be intimidating or inconvenient for a typical consumer. Another approach for spatial calibration is to install a microphone at each loudspeaker, which can be very expensive. Besides, when a listener is moving away from the sweet spot, existing methods have no way to detect this change and the listener has to go through the entire calibration process manually by putting the microphone at the new listening position. In contrast, using the integrated microphone array 114 in the soundbar 110, the calibration engine 116 can perform spatial calibration for loudspeakers as well as estimate listener's position with minimal user intervention, par [0034], see Wash); (In one embodiment, the position estimator 430 adopts the TDOA-based sound source localization for estimating the listener position. FIG. 6A illustrates an example three-element linear microphone array used to capture a listener's voice input. The three microphone elements are marked with their respective coordinates of M.sub.1(0, 0), M.sub.2(-L.sub.1, 0), and M.sub.3(L.sub.2, 0). Upon receiving the voice input or other sound cues from the listener 120, a closed-form solution for the distance R and angle .theta. of the listener 120 relative to the microphone array can be computed, Fig. 6A, par [0057], see Walsh).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the spatial calibration of surround sound systems including listener position estimation by Walsh with the method of Sheaffer in view of Skoglund such that obtaining device coordinates of the loudspeakers respectively; controlling the loudspeakers to emit calibration audio signals respectively; determining calibration time differences between calibration receiving time points at which the microphones receive the calibration audio signals respectively; and determining the microphone coordinates according to the calibration time differences and the device coordinates in order to improve the estimation accuracy of the loudspeakers and listener positions, as suggested by Walsh in paragraph [0043].
Allowable Subject Matter
12. Claims 4 and 7 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
13. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Inventor
Publication
Number
Disclosure
Sommerfeldt et al.
US Patent Application Publication
20030219132
The control source strength optimization procedure outlined above has been performed on a number of configurations. For example, FIG. 3 illustrates a four secondary source arrangement, configuration (a), and an eight secondary source arrangement, configuration (b),(paragraph [0042])
Shin et al.
US Patent Application Publication
20230258798
Then, by collecting all possible speaker-microphone pair distance measurements, a neural network can be trained to map a list of distances into geometric details (e.g., temple-hinge angle values, frame coordinates, and/or the like) that can be used to determine (or estimate) the frame configuration of the smart glasses, (paragraph [0019]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CON P TRAN whose telephone number is (571) 272-7532. The examiner can normally be reached M-F (08:30 AM- 05:00 PM) ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VIVIAN C. CHIN can be reached at 571-272-7848. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.P.T/Examiner, Art Unit 2695
/VIVIAN C CHIN/Supervisory Patent Examiner, Art Unit 2695
Prosecution Insights