DETAILED ACTION
This Communication is a First Action on the Merits (FAOM). Claims 1-12, as originally filed, are pending and have been considered as follows.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-9, 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Das et al (11,598,962 B1) in view of Robinson et al (10,897,570 B1).
As per Claims 1 and 7, Das teaches a method and head-mounted display for compensating audio data (Figures 1A and 1B – References 100 and 105; Column 3, Line 66 – Column 4, Line 20; Column 7, Lines 38-57), comprising: a storage medium, storing a plurality of room impulse responses (Figure 5 – Reference 505; Column 16, Lines 15-41). (Note: Figures 1A and 1B are illustrations of head-mounted displays enabling the compensation of audio data. In Column 16, Lines 15-41; Das describes a database module storing a set of room impulse responses for a respective set of locations)
Das also teaches an image capture device, capturing an image of a field (Figure 1A – Reference 120; Column 4, Lines 5-14; Column 5, Lines 32-40; Column 7, Lines 21-37); a speaker (Figure 1B – Reference 160; Column 7, Lines 51-57); and a processor, coupled to the storage medium, the image capture device, and the speaker (Figures 1A and 1B – Reference 150; Column 6, Lines 33-41; Column 29, Lines 52-63).
(Note: In Column 4, Lines 5-14; Das describes a depth camera assembly [DCA]. In Column 5, Lines 32-40; Das indicates the DCA includes an eye tracking unit that determines eye tracking information and further indicates that the eye tracking unit includes one or more cameras. In Column 7, Lines 21-37; Das describes a passive camera assembly [PCA] that generates color image data using one or more RGB cameras that capture images of the local area)
Das does not teach wherein the processor is configured to execute: performing image recognition on the image according to a machine learning model to obtain field type information; selecting a first room impulse response from the plurality of room impulse responses according to the field type information; processing audio according to the first room impulse response to generate processed audio; and outputting the processed audio through the speaker.
However, Robinson teaches wherein the processor is configured to execute: performing image recognition on the image according to a machine learning model to obtain field type information (Figure 2A – References 210, 212 and 218; Column 4, Line 42 – Column 5, Line 2); selecting a first room impulse response from the plurality of room impulse responses according to the field type information (Figure 2A – References 210, 212 and 218; Column 5, Lines 14-37); processing audio according to the first room impulse response to generate processed audio (Column 5, Lines 38-63); and outputting the processed audio through the speaker (Column 5, Lines 64-67).
(Note: In Column 4, Lines 42-55; Robinson describes a room modeling module [RMM] responsible for generating and updating a model based on image data. The RMM takes received depth data and determines the dimensions [i.e. surfaces of walls, floor, ceiling, etc.] and geometry of the room. The RMM uses retrieved color image data to associate materials with the room surface; and further identify objects in the room with their respective surfaces)
(Note: In Column 26, Lines 40-43; Das indicates that the system may determine acoustical model parameters and the respective identifier using a set of available room impulse responses. In Column 5, Lines 14-37; Robinson describes a room impulse response [RIR] database storing room impulse responses where each room impulse response is associated with room parameters that include one or more dimensions [i.e. length, width, height] of the room; room type; material type [i.e. wood, concrete, paster, carpet, etc.], object type [desk, chair, table, sofa, etc.])
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify the apparatus and method taught by Das with the apparatus and method taught by Robinson to create hyper-realistic depth, effortless sound localization and reduced listening fatigue to allow an individual’s brain to naturally process virtual sounds exactly as it does real-world acoustics thereby reducing the listeners cognitive load.
As per Claims 2 and 8, the combination of Das and Robinson teaches wherein the processor is configured to further execute: measuring a size of the field through the image capture device; and selecting the first room impulse response from the plurality of room impulse responses according to the size and the field type information as described in Claim 1.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify the apparatus and method taught by Das with the apparatus and method taught by Robinson to create hyper-realistic depth, effortless sound localization and reduced listening fatigue to allow an individual’s brain to naturally process virtual sounds exactly as it does real-world acoustics thereby reducing the listeners cognitive load.
As per Claims 3 and 9, the combination of Das and Robinson teaches wherein the processor is configured to further execute: measuring the field through the image capture device to obtain depth information; executing a simultaneous localization and mapping algorithm according to the depth information to obtain grid information; and calculating the size of the field according to the grid information.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify the apparatus and method taught by Das with the apparatus and method taught by Robinson to create hyper-realistic depth, effortless sound localization and reduced listening fatigue to allow an individual’s brain to naturally process virtual sounds exactly as it does real-world acoustics thereby reducing the listeners cognitive load.
As per Claims 5 and 11, the combination of Das and Robinson teaches wherein the field type information comprises a material of a sound reflector as described in Claim 1. (Note: See Robinson = Figure 2B – Reference 256 [Material]) It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify the apparatus and method taught by Das with the apparatus and method taught by Robinson to create hyper-realistic depth, effortless sound localization and reduced listening fatigue to allow an individual’s brain to naturally process virtual sounds exactly as it does real-world acoustics thereby reducing the listeners cognitive load.
As per Claims 6 and 12, the combination of Das and Robinson teaches receiving a plurality of historical images, wherein each of the plurality of historical images is tagged with historical field type information; and training the machine learning model according to the plurality of historical images as described in Claims 1 and 5.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify the apparatus and method taught by Das with the apparatus and method taught by Robinson to create hyper-realistic depth, effortless sound localization and reduced listening fatigue to allow an individual’s brain to naturally process virtual sounds exactly as it does real-world acoustics thereby reducing the listeners cognitive load.
Claim(s) 4 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Das et al (11,598,962 B1) in view of Robinson et al (10,897,570 B1) as applied to Claims 1 and 7 above, and further in view of Woodruff et al (2019/0206417 A1).
As per Claims 4 and 10, the combination of Das and Robinson teaches the apparatus and method of Claims 1 and 7; but does not teach performing convolution on the audio and the first room impulse response to generate the processed audio. However, Woodruff teaches performing convolution on the audio and the first room impulse response to generate the processed audio (Page 3, Paragraph [0030]; Page 6, Paragraphs [0059] and [0061]).
(Note: In paragraph [0030], Woodruff describes audio signals being convoluted with room impulse responses to create a third signal that represents an overlapping area of the two mathematically combined signals. In paragraphs [0059] and [0061], Woodruff describes audio signals used to reproduce a sound environment in virtual/augmented reality applications)
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify the apparatus and method taught by Das and Robinson with the apparatus and method taught by Woodruff to simulate environments ranging from famous concert halls to small studios with perfect mathematical accuracy due to the fact that convolution reverb preserves the specific absorption and diffusion qualities of a room which are difficult to recreate synthetically.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Robinson et al (10,721,521 B1), Visser et al (2018/0020312 A1), Brimijoin, II et al (11,290,837 B1), AUDFRAY et al (2024/0420718 A1), JOT et al (2019/0387352 A1), GUSTAFSSON et al (2024/0273835 A1). Each of these describes systems and methods of implementing audio and video in virtual, augmented and/or mixed reality environments.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KHARYE POPE whose telephone number is (571)270-5587. The examiner can normally be reached Monday - Friday 8AM - 4PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached at 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KHARYE POPE
Primary Examiner
Art Unit 2693
/KHARYE POPE/Primary Examiner, Art Unit 2693
Prosecution Insights