DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-11, 13, 15-17, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jouet et al. (PGPUB Document No. US 2021/0125414) in view of Berkebile (PGPUB Document No. US 2023/0298280).
Regarding claim 1, Jouet teaches a method ("a method for creating a map" (Jouet: 0008, FIG.4, FIG.5)) for providing assisted scene capture for an artificial reality environment, the method comprising:
Obtaining an artificial reality space model corresponding to a real-world space ("In step 510, virtual elements of the AR scene are obtained" (Jouet: 0067)),
Wherein the real-world space comprises at least one of a physical wall, a physical ceiling, a physical floor, or any combination thereof (refer to the walls and floors of FIG.6 that represent the real environment (Jouet: 0069))
And wherein the artificial reality space model comprises at least one of an artificial reality wall corresponding to the physical wall, an artificial reality ceiling corresponding to the physical ceiling, an artificial reality floor corresponding to the physical floor, or any combination thereof ("The wall planes are selected among the vertical planes that surround the scene." (Jouet: 0062));
Scanning, via an artificial reality system, the real-world space, the real-world space surrounding a user of the artificial reality system ("Figure 4 illustrates an example flowchart of process to generate an AR map according to at least one embodiment. In step 410, the AR terminal first obtains data representing the real environment, the data being captured by the AR terminal." (Jouet: 0061));
Displaying a mesh, corresponding to the scanned real-world space, overlaid onto a view of the real-world space ("the scene and the 3D textured mesh of the ground are rendered
separately but using the exact same camera setup and thus generate two pictures: one for the scene and one for the ground. The result of the rendering is illustrated in figures 10A to 10D: figure 10A shows a screenshot of the rendering of the 3D textured mesh of the scene" (Jouet: 0082))
However, Jouet does not expressly teach but Berkebile teaches
Processing the mesh, the processing including at least one of:
Collapsing one or more variations in the mesh corresponding to the at least one of the physical wall, the physical ceiling, the physical floor, or any combination thereof, onto the artificial reality space model,
Clipping the mesh to the artificial reality space model where the mesh extends beyond the artificial reality space model,
Simplifying the mesh corresponding to the at least one of the artificial reality wall, the artificial reality ceiling, the artificial reality floor, or any combination thereof ("For each of the selected first mesh blocks, a multi-stage simplification may be performed." (Berkebile: 0182, 0180) Also, see “simplifying the projected meshes on each of the corresponding planes based on, for example, a target triangle count” (Berkebile: 0187)),
Determining that the mesh is complete by identifying that a threshold percentage of the artificial reality space model is covered by the mesh,
Or any combination thereof.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings of Jouet such as to apply the simplification teaching of Berkebile, because this enables an effective method of to simplify/remove unnecessarily detailed meshes for further operations.
Regarding claim 2, the combined teachings above teach the method of claim 1, further comprising: providing at least one of the artificial reality space model, the processed mesh, or both to an artificial reality application executing on the artificial reality system (“the user is seeing the rendered scene as a mix of real and virtual elements…ground picture of the AR map is rendered as a semi-transparent area to keep other elements such as user's avatars, walls areas, objects as fully or more opaque areas” (Jouet: 0109, FIG.16)).
Regarding claim 3, the combined teachings above teach the method of claim 2, wherein a virtual object in the artificial reality application, executing on the artificial reality system, interacts with the at least one of the artificial reality space model, the processed mesh, or both (interacting with virtual objects, wherein “a digital representation of the AR scene 120 is handled by an AR controller 110 that also manages the coordination of the interactions between users in the augmented environment” (Jouet: 0035)).
Regarding claim 4, the combined teachings above teach the method of wherein the mesh includes one or more artificial reality objects corresponding to one or more physical objects in the scanned real-world space (refer to the mesh of reconstructed environment comprising physical objects such as a chair, etc. (Jouet: FIG.7A, 0070)).
Regarding claim 5, the combined teachings above teach the method of claim 4, further comprising: displaying a virtual object interacting with at least one of the one or more artificial reality objects, the virtual object being provided by the artificial reality application (interacting with virtual objects (Jouet: 0035)).
Regarding claim 7, the combined teachings above teach the method of claim 1, wherein the artificial reality space model is adjustable by the user via the artificial reality system (“Then the reconstructed 3D model is transformed into a user-defined coordinate system, thanks to a reference object generally identified as the origin (point with null coordinates) in the virtual environment” (Jouet: 0071)).
Regarding claim 8, the combined teachings above teach the method of claim 1, wherein the mesh is adjustable by the user via the artificial reality system (as part of the AR map generation process, the cleaned mesh is obtained (Jouet: 0070) according to steps disclosed in para 0071 of Jouet, which involves “the gravity direction can be indicated interactively by the user, in the case of the scene model containing a specific reference object that could be used to re-align that model regarding gravity direction…Then the reconstructed 3D model is transformed into a user-defined coordinate system…” (Jouet: 0071)).
Regarding claim 9, the combined teachings above teach the method of claim 1, wherein the artificial reality system scans the real-world space using at least one of one or more cameras, one or more depth sensors, or any combination thereof (“other type of data may be acquired for example comprising depth information captured using a time-of-flight sensor” (Jouet: 0061, 0041)).
Regarding claim 10, the combined teachings above teach the method of claim 1, wherein the artificial reality space model was captured, at least in part, via detected positions of one or more controllers of the artificial reality system (“In step 440, the meshes are rendered from a top view to generate 2D images. For that purpose, an orthographic camera is positioned over the scene mesh” (Jouet: 0063)).
Regarding claim 11, the combined teachings above do not expressly teach but Berkebile teaches the method of claim 1, wherein the artificial reality space model was captured, at least in part, by automatically identifying one or more flat surfaces in the real-world space (“The planarization operation may include detecting planar areas in the second mesh block, for example, using a region growing algorithm, projecting meshes of the detected planar areas to corresponding planes, adjusting plane normals of the detected planar areas to be substantially perpendicular to the corresponding planes, and simplifying the projected meshes on each of the corresponding planes based on, for example, a target triangle count.” (Berkebile: 0187)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the planarization teaching of Berkebile, because this enables an effective method of detecting flat surfaces.
Claim(s) 13, 15 and 16 is/are corresponding computer-readable storage medium claim(s) of claim(s) 1-3. The limitations of claim(s) 13, 15 and 16 are substantially similar to the limitations of claim(s) 1-3. Therefore, it has been analyzed and rejected substantially similar to claim(s) 13, 15 and 16.
Claim(s) 17, 19 and 20 is/are corresponding computer system claim(s) of claim(s) 1, 4 and 6. The limitations of claim(s) 17, 19 and 20 are substantially similar to the limitations of claim(s) 1, 4 and 6. Therefore, it has been analyzed and rejected substantially similar to claim(s) 17, 19 and 20.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jouet in view of Berkebile as applied to the claim(s) above, and further in view of Bleyer et al. (PGPUB Document No. US 2019/0325644).
Regarding claim 6, the combined teachings do not expressly teach but Bleyer teaches the method of claim 4, wherein the processing further includes: determining that at least one of the one or more physical objects is dynamic; and filtering the at least one of the one or more physical objects out of the mesh (dynamic objects is prevented from being added to the resulting 3D mesh…” (Bleyer: 0031, 0037)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings above such to apply the dynamic object filtering teaching of Bleyer, because educe and/or eliminate the amount of post-processing that occurs by preemptively addressing dynamic objects using pre-process filtering.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jouet in view of Berkebile as applied to the claim(s) above, and further in view of Latif et al. (US Patent No. 11829959).
Regarding claim 12, the combined teachings above do not expressly teach but Latif teaches the method of The method of wherein the mesh is generated by estimating depth data for the scanned real- world space, and wherein the depth data is estimated by applying a machine learning model to one or more images of the scanned real-world space (“The image processing algorithm and/or machine learning algorithm may scan and perform the depth analysis of the road frame 2.sup.nd using various image processing techniques and/or machine learning processes” (Latif: col. 13, line 12-23)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the machine learning teaching of Jouet, because this enables an effective and efficient manner of determining depth information.
Claim(s) 14 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jouet in view of Berkebile as applied to the claim(s) above, and further in view of Busto et al. (PGPUB Document No. US 2019/0392643).
Regarding claim 14, the combined teachings do not expressly teach but Busto teaches the computer-readable storage medium of claim 13, wherein the processing further comprises: determining that the mesh is complete by identifying that a threshold percentage of the artificial reality space model is covered by the mesh (Busto teaches the concept of “the completion criterion can include that all waypoint locations have been completed or attempted, that a 3D region is meshed to a threshold percentage of completion (Busto: 0150)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the teaching of Busto, because this is one of the well-known methods of determining a completion. Further, the application of the Busto teaching to the combined teachings above would have been effective in completing the mesh creating process.
Claim 18 is similar in scope to claim 14.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David H Chu whose telephone number is (571)272-8079. The examiner can normally be reached M-F: 9:30 - 1:30pm, 3:30-8:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H CHU/Primary Examiner, Art Unit 2616