Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/18/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-11 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over US Publication No. 2021/0329405 (“Eubank et al.”) in view of US Publication No. 2018/0232471 (“Schissler et al.”).
Regarding Claim 1, Eubank et al. discloses a method comprising:
at a processor of an electronic device (fig. 7, [0017]):
obtaining data representing one or more objects in a physical environment corresponding to an extended reality (XR) environment ([0017] Cameras, for example, those that are integral to such devices, can generate images of the physical setting or environment. Object detection, room geometry estimation, room layout extraction, and object alignment to come up with a room configuration estimation including the locations of walls and objects in the room can be detected in the image and then modeled);
generating an original three-dimensional (3D) model representing the physical environment based on the sensor data or modeled object data ([0033] A camera 16 generates one or more scene images 20 of a physical environment. An estimated model generator 22 generates, based on the one or more scene images, an estimated model of the physical environment. The estimated model can include a three dimensional space representation of the physical environment, and one or more environmental parameters of the physical environment);
obtaining scattering coefficients for the one or more objects based on the object recognition corresponding to the one or more objects ([0067] The estimated model can include a) a three dimensional representation of the physical environment, and/or b) one or more acoustic surface material parameters, and/or c) one or more scattering parameters. The acoustic surface material parameters and scattering parameters can be estimated for a physical room and detected objects in the physical environment); and
providing simulated acoustics within the XR environment based on the original 3D model and the scattering coefficients for the one or more objects ([0067] In block 224, the process can receive audio signals captured by a microphone array, the audio signals capturing a sound in the physical environment. In block 228, the process can generate one or more measured acoustic parameters of the physical environment based on the received audio signals. [0018] Using a room geometry optimized for acoustic simulation (e.g., an acoustically correct or simplified version of the physical room configuration) along with the acoustic parameter estimation, one can characterize the acoustics of the physical environment. The characterization can be sent to other applications or users for processing. In this case, other users that are brought into the virtual setting can experience a virtualized audio that matched that of the enhanced reality setting, which in turn, matches that of the physical environment, for example, of the user.).
Eubank et al. does not specify the object recognition specifically comprising determining semantic information corresponding to the one or more objects ([0065] the geometry estimation block 202 is performed by classifying regions of the image as surfaces and shapes in three dimensional space. In other words, the image can be processed without trying to classify objects in the image as they might relate to known objects in a database).
In the same field of endeavor, Schissler et al. also discloses acoustic classification and optimization for multi-modal rendering of real-world scenes. Schissler applies semantic/material classification and automatic assignment of acoustic properties to the reconstructed scene. Given an image of a scene, material segmentation approaches use visual characteristics to determine a material type for each pixel in the image, combines both low-level (e.g., color) and mid-level (e.g., SIFT) image features trained in a Bayesian framework and achieve moderate material recognition accuracy. By using object and material recognition together, it has been shown that recognition results can be improved ([0035]).
It would have been obvious to incorporate the visual material-classification techniques of Schissler et al. into the enhanced-reality audio processing system of Eubank et al. to automatically initialize the acoustic properties, including scattering parameters, associated with recognized objects in the reconstructed 3D environment. Doing so would reduce manual configuration and provide a more accurate starting point for subsequent acoustic optimization based on measured responses.
Claim 18 recites a system comprising: a processor; a computer readable medium storing instructions that when executed by the processor cause the processor to perform operations comprising the method of claim 1. Thus claim 18 is rejected in view of Eubank et al. in view of Schissler et al. for the same reasons discussed with respect to claim 1. Eubank et al. additionally discloses a processor; a computer readable medium storing instructions that when executed by the processor ([0021]).
Claim 20 recites a non-transitory computer-readable medium comprising instructions that when executed by a processor cause the processor to perform operations comprising the method of claim 1. Thus claim 20 is rejected in view of Eubank et al. in view of Schissler et al. for the same reasons discussed with respect to claim 1. Eubank et al. additionally discloses a processor; a computer readable medium storing instructions that when executed by the processor ([0021]).
Regarding claims 2 and 19, Eubank et al. in view of Schissler et al. discloses the method of claim 1, wherein obtaining the scattering coefficients comprises identifying mean scattering coefficients for objects corresponding to each of the one or more objects (Schissler, [0058].
Regarding claim 3, Eubank et al. in view of Schissler et al. discloses the method of claim 1, wherein obtaining the scattering coefficients comprises identifying the scattering coefficients stored in a database based on the semantic information (Schissler, fig. 4).
Regarding claim 4, Eubank et al. in view of Schissler et al. discloses the method of claim 3, wherein the scattering coefficients stored in the database are determined based on simulations of annotated geometrical acoustical (GT) data ([Eubank, [0054] a block 110 performs room geometry estimation. This block estimates the size, shape, and/or volume of the physical environment, for example, if the physical environment is a room, the size, shape and/or volume of the room can be estimated. The room geometry estimation can include classifying regions of the image, and based on the classified regions, a room layout extraction block 112 can generate a geometry of the physical environment).
Regarding claim 5, Eubank et al. in view of Schissler et al. discloses the method of claim 3, wherein the scattering coefficients stored in the database are determined using a simplified mesh with coefficients determined based on displacement information (Schissler, 3.1.3 Mesh simplification, acoustic simulation may involve simplifying a dense triangle mesh, Fig. 8).
Regarding claim 6, Eubank et al. in view of Schissler et al. discloses the method of claim 1, wherein said obtaining the scattering coefficients comprises interpolating values with respect to dimensions of each of the one or more objects (Eubank, [0065] the image processing can include edge detection, semantic segmentation, instance segmentation, and other computer vision techniques that may utilize one or more neural networks to classify regions of the image as surfaces and shapes).
Regarding claim 7, Eubank et al. in view of Schissler et al. discloses the method of claim 1, wherein the semantic information comprises object types of the one or more objects (Eubank, [0053] object definition based on characteristic).
Regarding claim 8, Eubank et al. in view of Schissler et al. discloses The method of claim 1, wherein the semantic information comprises 3D shape dimensions of the one or more objects (Eubank, [0034] based on size structure or shape).
Regarding claim 9, Eubank et al. in view of Schissler et al. discloses the method of claim 1, wherein the semantic information comprises materials of the one or more objects ((Eubank, [0020] acoustic surface material).
Regarding claim 10, Eubank et al. in view of Schissler et al. discloses the method of claim 1, wherein the original 3D model is a 3D mesh (Eubank, [0034] mesh data structure).
Regarding claim 11, Eubank et al. in view of Schissler et al. discloses the method of claim 1, wherein said providing simulated acoustics comprises determining acoustic wave reflection or scattering from the one or more objects based on the scattering coefficients (Eubank, [0042] optimize or tune acoustic parameters of the model (e.g., scattering characteristics/parameters, acoustic absorption coefficients, and/or sound reduction parameters of an object in the model) based on inputs from audio signals in the physical environment).
Regarding claim 16, Eubank et al. in view of Schissler et al. discloses the method of claim 1 further comprising determining to redetermine the original 3D model or scattering coefficients based on detecting a change in the XR environment exceeding a threshold (Eubank, [0058] update model based on threshold detection).
Regarding claim 17, Eubank et al. in view of Schissler et al. discloses the method of claim 1, wherein the data comprises sensor data or modeled object data (Eubank, [0014] various sensors for capturing data).
Allowable Subject Matter
Claims 12-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIRAPON TULOP whose telephone number is (571)270-7491. The examiner can normally be reached Monday to Friday, 10:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached at 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JIRAPON TULOP/Examiner, Art Unit 2693
Prosecution Insights