DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is in response to Applicant’s amendment filed 7/23/26 which has
been entered and made of record. Claims 46, 53-55, and 60 have been amended. No claims cancelled or newly added.. Claims 46-48, 50-55, 57-62, and 64-68 are pending in the application.
Response to Arguments
Applicant’s arguments with respect to claim(s) 46, 53 and 60 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument (due to applicant’s arguments directed to newly amend limitation(s) which is addressed by new prior art presented in this Office Action).
The arguments regarding dependent claims for the virtue of their dependency are moot
because the independent claims are not allowable.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 46-48, 50-55, 57-62, and 64-68 is/are rejected under 35 U.S.C. 103 as being unpatentable over Boyle et al. (U.S. Patent Application Publication No. 2017/0345216), hereinafter referenced as Boyle, in view of Thagadur Shivappa (U.S. Patent Application Publication No. 2018/0046431), hereinafter referenced as Thagadur, SHIRAKIHARA (U.S. Patent Application Publication No. US 2016/0125871), hereinafter referenced as SHIRAKIHARA, Ridihalgh et al. (U.S. Patent No. 9,937,422), hereinafter referenced as Ridihalgh and Berthet (U.S. Patent Application Publication No. 2019/0019520), hereinafter referenced as Berthet.
Regarding claim 46, Boyle teaches at least one non-transitory machine readable medium comprising machine readable instructions to cause at least one processor circuit to at least: build a volumetric model of a scene based on ray tracing, (abstract teaches a non-transitory computer readable medium with code segment executable by a processor to build a model of an environment, paragraph 78 teaches model prepared by the data preparation module which uses ray tracing, and paragraph 81 teaches "Data preparation module 266 aggregates data collected from the environment...which can provide a 3D representation of an acoustic field"; the code segments correspond to instructions and 3D here shows the model is volumetric; geometry of the scene, a material property of an object in the scene, and an observer position (paragraph 75 teaches model can include data such as but not limited to geometry, material properties, and position); although the position pertains to sound emitters, it can also pertain to observer position because as mentioned in paragraph 72, the model would be based off feature vectors regarding tracked position and orientation of an apparatus in motion, which essentially indicates an observer position since the apparatus observes the scene;
However, Boyle fails to teach and process input audio with the audio filters to generate left output audio and right output audio for a head-mounted display device.
However, Thagadur teaches and process input audio with the audio filters to generate left output audio and right output audio for a head-mounted display device (Thagadur, paragraph 132 teaches HMD (head-mounted display) including decoder matrices for left and right ear and that audio spatialization circuitry can include a filter generation component); this means the right and left audio is achieved after input audio is processed by being decoded and filtered. Thagadur is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of processing audio and using it for an HMD. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Boyle's invention with the audio filter processing techniques of Thagadur to result in an improved and more immersive user experience (Thagadur, paragraph 156).
However, the combination of Boyle and Thagadur fails to teach
determine, based on the volumetric model, a first multiplier value applicable to at least a first audio propagation path, the first audio propagation path determined based on the ray tracing; determine, based on the volumetric model, a different second multiplier value applicable to at least a second audio propagation path, the second audio propagation path different from the first audio propagation path, the second audio propagation path determined based on the ray tracing; generate audio filters based on the first multiplier value and the second multiplier value.
However, SHIRAKIHARA teaches determine, based on the volumetric model, a first multiplier value applicable to at least a first audio propagation path, (SHIRAKIHARA, paragraph 54 teaches “sound calculator 102 applies simulation results of propagation characteristics corresponding to respective propagation paths of early reflected sounds to an input signal…allocates resulting signals to plural output lines to which the respective multipliers 107 are connected”, and paragraph 60 teaches “multipliers 220 multiplies output audio signals of the delay line 210 by particular coefficients…More specifically, the multiplier 220 multiplies an output audio signal by a relatively small number if the length of a propagation path correlated with it is long and by a relatively large number if the length of a propagation path correlated with it is short”); first and second/different multiplier values are shown by respective multipliers belonging to each propagation path, the respective propagation paths shows first audio and second/different audio propagation path, and this is based on the volumetric model because when viewed in combination, the volumetric model from Boyle would have audio (since Boyle, paragraph 82 mentions “data preparation module 266 is coupled to an audio tactile rendering module”) that has these techniques from SHIRAKIHARA applied to it;
the first audio propagation path determined based on the ray tracing, (SHIRAKIHARA, paragraph 57 teaches “sound calculator 102…referred to as an “acoustics calculator 100.” The acoustics calculator 100 performs sound field calculation processing that includes processing of calculating reflected sound propagation characteristics using a calculation technique such as a mirror image method or a sound ray tracing method”); since the calculator uses ray tracing, the first audio propagation path is determined based on such; determine, based on the volumetric model, a different second multiplier value applicable to at least a second audio propagation path, the second audio propagation path different from the first audio propagation path, (SHIRAKIHARA, paragraph 54 teaches “sound calculator 102 applies simulation results of propagation characteristics corresponding to respective propagation paths of early reflected sounds to an input signal…allocates resulting signals to plural output lines to which the respective multipliers 107 are connected”, and paragraph 60 teaches “multipliers 220 multiplies output audio signals of the delay line 210 by particular coefficients…More specifically, the multiplier 220 multiplies an output audio signal by a relatively small number if the length of a propagation path correlated with it is long and by a relatively large number if the length of a propagation path correlated with it is short”); first and second/different multiplier values are shown by respective multipliers belonging to each propagation path, the respective propagation paths shows first audio and second/different audio propagation path, and this is based on the volumetric model because when viewed in combination, the volumetric model from Boyle would have audio (since Boyle, paragraph 82 mentions “data preparation module 266 is coupled to an audio tactile rendering module”) that has these techniques from SHIRAKIHARA applied to it; the second audio propagation path determined based on the ray tracing (SHIRAKIHARA, paragraph 57 teaches “sound calculator 102…referred to as an “acoustics calculator 100.” The acoustics calculator 100 performs sound field calculation processing that includes processing of calculating reflected sound propagation characteristics using a calculation technique such as a mirror image method or a sound ray tracing method”); since the calculator uses ray tracing, the second audio propagation path is determined based on such; generate audio filters based on the first multiplier value and the second multiplier value (SHIRAKIHARA, claim 1 teaches “addition noise generator configured to generate an addition noise by adding the noise to the impulse noise; an impulse response generator configured to generate a modified impulse response …an impulse response convolver configured to convolve an input audio signal with the modified impulse response” and claim 2 teaches “wherein the addition noise generator comprises: a first multiplier configured to multiply the impulse noise by a first coefficient; a second multiplier configured to multiply the noise by a second coefficient”); this shows audio filters such as impulse response and impulse response convolver generated and comprising (thus based on) the first and second multiplier values. SHIRAKIHARA is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of respective multiplier values applicable to respective propagation paths. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Boyle and Thagadur with the multiplier values and propagation paths techniques of SHIRAKIHARA to ensure reverberant sound be controlled to a proper value according to the kind and the details of an input signal, the acoustic characteristics of a space, and other factors (SHIRAKIHARA, paragraph 7). This would create a more realistic sound and also enhance user experience.
However, the combination of Boyle, Thagadur and SHIRAKIHARA fails to teach
the volumetric model including a first voxel associated with at least a portion of the object and a second voxel associated with at least a portion of an audio propagation medium, the first voxel associated with a first coefficient corresponding to audio reflection off the object, the first coefficient based on the material property, the second voxel associated with a second coefficient;
However, Ridihalgh teaches the volumetric model including a first voxel associated with at least a portion of the object (Ridihalgh, col. 1, lines 31-33 teach “a plurality of blocked voxels. Blocked voxels are voxels that intersect visual elements of the 3D interactive media environment”); blocked voxel shows first voxel and is associated with portion of object since intersects a visual element/object of the 3D environment; and a second voxel associated with at least a portion of an audio propagation medium, (Ridihalgh, col. 1, lines 38-41 teach “clear voxels can indicate, for example, a shortest path of sound, in the 3D interactive media environment, between a voxel and the listener. The shortest path can account for propagation of sound around obstacles”); this clear voxel shows a second voxel and is for shortest path which accounts for and is associated with propagation of sound/audio and the medium thereof (around obstacles); the first voxel associated with a first coefficient corresponding to audio reflection off the object, (Ridihalgh, col. 3, lines 19-24 teach “locations of blocked voxels inform the way in which sound is reflected around the listener. For example, if a wall in the 3D media environment (represented as blocked voxels) is six inches behind the listener, sound will reverberate differently than if the nearest blocked voxel is 20 feet behind the listener”); this shows blocked/first voxel associated with a first distance/coefficient that corresponds to audio/sound reflection off the object (in this example the wall); the first coefficient based on the material property, (Ridihalgh, col. 3, lines 13-18 teach “determine reverberation information from the listener's perspective. This can be done, for example, by determining a distance, from the listener, to the nearest blocked voxel in many directions, and in some examples, the type of material and corresponding acoustic properties of the blocked voxels”); taking into account type of material when determining distance shows the first coefficient would also be based on material property; the second voxel associated with a second coefficient (Ridihalgh, col. 4, lines 52-55 teach “As in FIG. 2, In FIG. 3, blocked voxels are denoted with an “X,” the listener is denoted with an “L” (in voxel 302), and clear voxels indicate a shortest clear-voxel distance to the listener” and col. 5, lines 14-20 teach “shortest distance corresponds to the length of a line connecting the voxel and the location of the listener. The ratio indicates an audio adjustment that accounts for the length of the path of clear voxels, and can be used to determine an obstruction value. For example, if the clear voxel path is 20, and the shortest (direct, ignoring blocked voxels) path is 10, the ratio is 2:1.”); ratio that indicates length and distance (to clear/second voxel) shown by the numbers in each square of fig. 3 shows a second coefficient;
Ridihalgh is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of voxels alongside sound propagation.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Boyle, Thagadur and SHIRAKIHARA with the voxel techniques of Ridihalgh to ensure computational complexity of determining audio path information (e.g., distance between sound emitters and the listener) and reverberation is reduced through the voxel-based approach, which reduces the amount of processing power, memory, and energy needed to accurately simulate acoustics in the media environment and allows the simulated acoustics to be updated in real time as the location of the listener changes (Ridihalgh, col. 3, lines 25-33). This would ensure a more realistic and efficient invention overall.
However, the combination of Boyle, Thagadur, SHIRAKIHARA and Ridihalgh fails to explicitly teach the second coefficient to represent an audio attenuation property of the audio propagation medium, the second coefficient based on a type of the audio propagation medium;
However, Berthet explicitly teaches the second coefficient to represent an audio attenuation property of the audio propagation medium, the second coefficient based on a type of the audio propagation medium (Berthet, paragraph 95 teaches “the acoustic coefficient is defined as an attenuation coefficient, i.e. a ratio of the sound intensity absorbed by the obstacles to the propagation of sound waves and the path through the air to the intensity of the sound source.”); this shows second coefficient as attenuation coefficient (thus represents audio attenuation property of the following audio propagation medium) and this coefficient is based on type of audio propagation medium due to the mention of ratio of sound intensity absorbed by obstacles to the propagation of sound waves and path through the air. Berthet is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of immersion in 3D audio scene. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Boyle, Thagadur, SHIRAKIHARA and Ridihalgh with the attenuation coefficient and type of propagation medium techniques of Berthet to improve the sensation of immersion in a 3D audio scene (Berthet, abstract). This is done due to more realistic sound from the attenuation coefficients and propagation path types being used for calculations.
Regarding claim 47, the combination of Boyle, Thagadur, SHIRAKIHARA, Ridihalgh and Berthet teaches wherein the instructions are to cause one or more of the at least one processor circuit to build the volumetric model based on an orientation of an observer (Boyle, abstract teaches code executable by a processor and paragraph 72 teaches the model would be based off feature vectors regarding tracked position and orientation of an apparatus in motion); the orientation of an apparatus corresponds to orientation of an observer since the apparatus observes the scene.
Regarding claim 48, the combination of Boyle, Thagadur, SHIRAKIHARA, Ridihalgh and Berthet teaches wherein the volumetric model is to represent reverberation of the input audio (Boyle, paragraph 73 teaches physical properties in environment such as reverberation characteristics being processed).
Regarding claim 50, the combination of Boyle, Thagadur, SHIRAKIHARA, Ridihalgh and Berthet teaches wherein the audio filters include a finite impulse response filter (Boyle, paragraph 82 teaches Finite Impulse Response (FIR) filter generation module is coupled to an output of the data exporter module).
Regarding claim 51, the combination of Boyle, Thagadur, SHIRAKIHARA, Ridihalgh and Berthet teaches wherein the scene is a virtual reality scene (Boyle, paragraph 41 teaches capturing of scene can be done by a virtual reality (VR) system).
Regarding claim 52, the combination of Boyle, Thagadur, SHIRAKIHARA, Ridihalgh and Berthet teaches wherein the instructions are to cause one or more of the at least one processor circuit to generate the audio filters on the fly (Boyle, abstract teaches processor executing instructions/code, paragraph 65 teaches creating real-time mixed reality sounds and paragraph 66 teaches real-time audio enhancements); audio enhancement covers the audio filter generation because the audio filters are generated to enhance audio and doing this in real-time ensures that the audio filters are generated on the fly.
Regarding claim 53, the apparatus claim 53 recites similar limitations as product/non-transitory machine readable medium claim 46, and thus is rejected under similar rationale. In addition, Boyle, fig. 3 teaches an apparatus and fig. 13 teaches an interface subsystem for fig. 3; the interface would indicate that interface circuitry is used.
Regarding claim 54, the apparatus claim 54 recites similar limitations as product/non-transitory machine readable medium claim 47, and thus is rejected under similar rationale.
Regarding claim 55, the apparatus claim 55 recites similar limitations as product/non-transitory machine readable medium claim 48, and thus is rejected under similar rationale.
Regarding claim 57, the apparatus claim 57 recites similar limitations as product/non-transitory machine readable medium claim 50, and thus is rejected under similar rationale.
Regarding claim 58, the apparatus claim 58 recites similar limitations as product/non-transitory machine readable medium claim 51, and thus is rejected under similar rationale.
Regarding claim 59, the apparatus claim 59 recites similar limitations as product/non-transitory machine readable medium claim 52, and thus is rejected under similar rationale.
Regarding claim 60, the method claim 60 recites similar limitations as product/non-transitory machine readable medium claim 46, and thus is rejected under similar rationale.
Regarding claim 61, the method claim 61 recites similar limitations as product/non-transitory machine readable medium claim 47, and thus is rejected under similar rationale.
Regarding claim 62, the method claim 62 recites similar limitations as product/non-transitory machine readable medium claim 48, and thus is rejected under similar rationale.
Regarding claim 64, the method claim 64 recites similar limitations as product/non-transitory machine readable medium claim 50, and thus is rejected under similar rationale.
Regarding claim 65, the method claim 65 recites similar limitations as product/non-transitory machine readable medium claim 51, and thus is rejected under similar rationale.
Regarding claim 66, the combination of Boyle, Thagadur, SHIRAKIHARA, Ridihalgh and Berthet teaches wherein the audio propagation medium is air (Ridihalgh, col. 8, lines 29-32 teaches “In such cases, sound reaching the edge of the island environment can be propagated to the local boat environment using a simple straight-line propagation path”); sound from edge of island to local boat using simple straight-line propagation path shows the audio propagation medium would be air since air is between the boat and edge of island. The same motivations used in claim 46 apply here in claim 66.
Regarding claim 67, the apparatus claim 67 recites similar limitations as product/non-transitory machine readable medium claim 66, and thus is rejected under similar rationale.
Regarding claim 68, the method claim 68 recites similar limitations as product/non-transitory machine readable medium claim 66, and thus is rejected under similar rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kitamura (U.S. Patent No. 5,689,571) col. 14, lines 24-26 teach “two kinds of subsequent reverberation sounds G, J. Moreover, attenuation speed coefficients SPn are changed to change the delay times T1, T2 of reverberation sounds”; this shows second attenuation coefficient based on kind/type of reverberation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAUMAN U AHMAD whose telephone number is (703)756-5306. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.U.A./Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights