Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6, 12-17, 19, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Atlas et al. (U.S. Patent No. 10,984,242), referred herein as Atlas.
Regarding claim 1, Atlas teaches a display method comprising: determining data of an environment map of a physical space, and an association between a coordinate system of the environment map and a coordinate system of a virtual scene (column 6, lines 52-54; column 7, lines 3-11 and 58-64; column 22, lines 32-35 and 63-66; an environment map of a physical space is obtained, and an association is determined between the map of the physical space and a virtual space);
obtaining a first environment image of the physical space, and determining a first pose of an extended reality XR device under the coordinate system of the environment map according to the first environment image and the data of the environment map (column 7, lines 15-30; column 22, lines 35-42 and 63-66; an image of the physical space is obtained, and a first [physcial] pose of an XR device in environment map coordinates is determined using the image and the map);
calculating a second pose of the XR device under the coordinate system of the virtual scene according to the first pose and the association; and displaying the virtual scene according to the second pose and the virtual scene (column 7, line 58 through column 8, line 3; column 22, line 63 through column 23, line 9; column 23, lines 9-23; a second [virtual] pose of the XR device in virtual scene coordinates is determined based on the association between the physical and virtual coordinate systems, and the virtual scene is displayed according to the second [virtual] pose).
Regarding claim 2, Atlas teaches the method according to claim 1, wherein before the determining data of an environment map of a physical space, and an association between a coordinate system of the environment map and a coordinate system of a virtual scene, the method further comprises: acquiring a plurality of second environment images of the physical space, and generating the environment map of the physical space according to the plurality of second environment images; and determining the association between the coordinate system of the environment map and the coordinate system of the virtual scene according to an input of a user, or determining the association between the coordinate system of the environment map and the coordinate system of the virtual scene according to a preset mark identified in at least one second environment image (column 7, lines 15-30 and 58-67; column 22, lines 25-35 and 63-66; column 23, lines 1-6; the process of obtaining images of the physical space, generating the environment map, and determining physical-to-virtual associations is iterative and continuous as a user moves and/or interacts with the environment, and determining the associations may be instigated by input from a user).
Regarding claim 3, Atlas teaches the method according to claim 2, wherein the determining the association between the coordinate system of the environment map and the coordinate system of the virtual scene according to an input of a user, comprises: determining the association between the coordinate system of the environment map and the coordinate system of the virtual scene according to a first point in the physical space determined by the user (column 7, lines 15-22 and 58-67; column 17, lines 46-54; column 23, lines 1-6).
Regarding claim 4, Atlas teaches the method according to claim 3, wherein the determining the association between the coordinate system of the environment map and the coordinate system of the virtual scene according to a first point in the physical space determined by the user, comprises: determining the association between the coordinate system of the environment map and the coordinate system of the virtual scene according to the first point and a direction in the physical space determined by the user; or, determining the association between the coordinate system of the environment map and the coordinate system of the virtual scene according to the first point and a second point in the physical space determined by the user (column 7, lines 15-22 and 58-67; column 17, lines 33-54; column 22, line 66 through column 23, line 9).
Regarding claim 5, Atlas teaches the method according to claim 3, wherein a physical marker is provided at the first point in the physical space, or the first point in the physical space is a corner point (column 7, lines 58-67; column 18, lines 42-51; column 23, lines 1-9).
Regarding claim 6, Atlas teaches the method according to claim 1, wherein the virtual scene is matched with the physical space (column 7, lines 3-15 and 58-67; column 22, line 63 through column 23, line 12).
Regarding claim 12, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the processor, memory, and program, which are taught by Atlas, column 23, lines 37-50; column 23, line 65 through column 24, line 12); thus they are rejected on similar grounds.
Regarding claims 13-17, the limitations of these claims substantially correspond to the limitations of claims 2-6, respectively; thus they are rejected on similar grounds as their corresponding claims.
Regarding claim 19, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the medium, program, and computer, which are taught by Atlas, column 23, lines 37-50; column 23, line 65 through column 24, line 12); thus they are rejected on similar grounds.
Regarding claim 20, the limitations of this claim substantially correspond to the limitations of claim 2; thus they are rejected on similar grounds.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 7, 8, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Atlas, in view of Xu (U.S. Patent Application Publication No. 2022/0292796), referred herein as Xu.
Regarding claim 7, Atlas teaches the method according to claim 1, wherein after the calculating a second pose of the XR device under the coordinate system of the virtual scene according to the first pose and the association, the method further comprises: sending information so that another device in the physical space displays a virtual object corresponding to the XR device according to the second pose (column 7, lines 3-23; column 8, lines 45-66; column 9, lines 7-19 and 33-58).
Atlas does not explicitly teach sending the second pose.
However, in a similar field of endeavor, Xu teaches a method comprising determining data of an environment map of a physical space and an association between first and second coordinate systems, obtaining an environment image of the physical space, determining a pose of an XR device under the first coordinate system, calculating another pose under the second coordinate system, and displaying the scene according to the second pose and the map (figs 1 and 4; paragraphs 22 and 23; paragraph 24, lines 1-10; paragraphs 57 and 58; paragraph 59, lines 1-8; paragraph 67), and further comprising sending the pose so that another device in the physical space displays a virtual object corresponding to the XR device (paragraph 20; paragraph 28, the last 12 lines; paragraph 71, the last 9 lines).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the pose transmission of Xu with the information transmission of Atlas because this helps to ensure that the virtual space is displayed consistently across devices, thereby improving both the quality of the images and the user interaction in the virtual space (see, for example, Xu, paragraph 4; paragraph 34, the last 7 lines; paragraph 37, the last 7 lines; paragraph 68).
Regarding claim 8, Atlas teaches the method according to claim 2, wherein after the determining the association between the coordinate system of the environment map and the coordinate system of the virtual scene according to an input of a user, the method further comprises: sending information to another device in the physical space, so that the other device determines its pose under the coordinate system of the environment map according to the association and the environment image of the physical space (column 7, lines 3-23; column 8, lines 45-66; column 9, lines 7-19 and 33-58).
Atlas does not explicitly teach sending the association.
However, in a similar field of endeavor, Xu teaches a method comprising determining data of an environment map of a physical space and an association between first and second coordinate systems, obtaining an environment image of the physical space, determining a pose of an XR device under the first coordinate system, calculating another pose under the second coordinate system, and displaying the scene according to the second pose and the map (figs 1 and 4; paragraphs 22 and 23; paragraph 24, lines 1-10; paragraphs 57 and 58; paragraph 59, lines 1-8; paragraph 67), and further comprising sending the association so that another device in the physical space determines its pose under the first coordinate system (paragraph 28, the last 12 lines; paragraph 61, lines 1-9; paragraph 63, lines 1-10).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the association transmission of Xu with the information transmission of Atlas because this helps to ensure that the virtual space is displayed consistently across devices, thereby improving both the quality of the images and the user interaction in the virtual space (see, for example, Xu, paragraph 4; paragraph 34, the last 7 lines; paragraph 37, the last 7 lines; paragraph 68).
Regarding claim 18, the limitations of this claim substantially correspond to the limitations of claim 7; thus they are rejected on similar grounds.
Claims 9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Atlas, in view of Cao et al. (U.S. Patent Application Publication No. 2021/0279949), referred herein as Cao.
Regarding claim 9, Atlas teaches the method according to claim 1, wherein the data of the environment map comprises three-dimensional coordinates in the environment map, and pixel coordinates; and the determining a first pose of the XR device under the coordinate system of the environment map according to the first environment image and the data of the environment map, comprises: extracting coordinates in the first environment image, and performing pose solution according to three-dimensional coordinates, pixel coordinates in the first environment image, and pixel coordinates in the environment map, to obtain the first pose of the XR device under the coordinate system of the environment map (column 7, lines 15-30 and 58-67; column 8, lines 1-3; column 22, lines 35-42 and 63-67; column 23, lines 1-23).
Atlas does not explicitly teach that the environment comprises feature points, pixel coordinates of the feature points, and descriptors of the feature points; extracting feature points in the first environment image, and calculating descriptors of the feature points in the first environment image; matching the descriptors of the feature points in the first environment image with the descriptors of the feature points in the environment map; and performing pose solution according to pixel coordinates in the environment map of matched feature points.
However, in a similar field of endeavor, Cao teaches and method comprising determining data of an environment map of a physical space and an association between coordinate systems of the map and a virtual scene, obtaining an environmental image of the physical space, and determining a pose of an XR device according to the image ad the map (paragraph 124; paragraph 126, lines 1-12; paragraph 127), wherein the environment map comprises feature points, pixel coordinates of the feature points, and descriptors of the feature points, and the method further comprises extracting feature points in the first environment image, calculating descriptors of the feature points in the first environment image, matching the descriptors of the feature points in the first environment image with the descriptors of the feature points in the environment map, and performing pose solution according to pixel coordinates in the environment map of matched feature points (paragraph 126, lines 1-19; paragraph 297, lines 1-7; paragraphs 298 and 299; paragraph 305).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the feature point and descriptors of Cao with the pose processing of Atlas because this provides enough information to accurately process the positioning and interaction of an XR device in a virtual environment, while minimizing the amount of information necessary to do so, thus increasing the efficiency of the processing and sharing of the environment information while maintaining quality and accuracy (see, for example, Cao, paragraphs 131 and 134; paragraph 291).
Regarding claim 11, Atlas teaches the method according to claim 2, but does not explicitly teach the method, wherein the generating the environment map of the physical space according to the second environmental image, comprises: extracting feature points in the second environment image, determining pixel coordinates of the feature points, and calculating descriptors of the feature points; calculating three-dimensional coordinates of the feature points according to the pixel coordinates of the feature points; and establishing the environment map according to the pixel coordinates, the descriptors and the three-dimensional coordinates of the feature points.
However, in a similar field of endeavor, Cao teaches and method comprising determining data of an environment map of a physical space and an association between coordinate systems of the map and a virtual scene, obtaining an environmental image of the physical space, and determining a pose of an XR device according to the image ad the map (paragraph 124; paragraph 126, lines 1-12; paragraph 127), wherein generating the environment map of the physical space according to the environmental image, comprises: extracting feature points in the second environment image, determining pixel coordinates of the feature points, and calculating descriptors of the feature points; calculating three-dimensional coordinates of the feature points according to the pixel coordinates of the feature points; and establishing the environment map according to the pixel coordinates, the descriptors and the three-dimensional coordinates of the feature points (paragraph 126, lines 1-19; paragraph 297, lines 1-7; paragraphs 298 and 299; paragraph 305).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the feature point and descriptors of Cao with the pose processing of Atlas because this provides enough information to accurately process the positioning and interaction of an XR device in a virtual environment, while minimizing the amount of information necessary to do so, thus increasing the efficiency of the processing and sharing of the environment information while maintaining quality and accuracy (see, for example, Cao, paragraphs 131 and 134; paragraph 291).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Atlas, in view of Cao, and further in view of Xu.
Regarding claim 10, Atlas in view of Cao teaches the method according to claim 9, but does not explicitly teach the method, wherein the XR device performs the pose solution by using a perspective-n-point PnP algorithm.
However, in a similar field of endeavor, Xu teaches a method comprising determining data of an environment map of a physical space and an association between first and second coordinate systems, obtaining an environment image of the physical space, determining a pose of an XR device under the first coordinate system by performing pose solution according to three-dimensional coordinates, calculating another pose under the second coordinate system, and displaying the scene according to the second pose and the map (figs 1 and 4; paragraph 21, lines 1-13; paragraphs 22 and 23; paragraph 24, lines 1-10; paragraphs 57 and 58; paragraph 59, lines 1-8; paragraph 67), wherein the XR device performs the pose solution by using a perspective-n-point PnP algorithm (paragraph 21, the last 17 lines; paragraph 54).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the PnP algorithm of Xu with the pose solution of Atlas in view of Cao because this helps generate poses of higher accuracy, thus ensuring that the virtual space is displayed consistently across devices, and improving both the quality of the images and the user interaction in the virtual space (see, for example, Xu, paragraph 4; paragraph 34, the last 7 lines; paragraph 37, the last 7 lines; paragraph 68).
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Kellogg (U.S. Patent Application Publication No. 2019/0026948); Markerless augmented reality (AR) system.
Ohashi (U.S. Patent Application Publication No. 2022/0138982); Information processing apparatus, information processing method, and program.
Atlas (U.S. Patent No. 10,984,242); Virtual proximity compass for navigating artificial reality environments.
Stokking (U.S. Patent Application Publication No. 2021/0195157); Augmented virtuality self view.
Chan (U.S. Patent No. 11,460,913); Directed emitter/sensor for electromagnetic tracking in augmented reality systems.
Bouhnik (U.S. Patent Application Publication No. 2021/0279953); Cross reality system with WiFi/GPS based map merge.
Shahrokni (U.S. Patent Application Publication No. 2021/0264674); Cross reality system with buffering for localization accuracy.
Cao (U.S. Patent Application Publication No. 2021/0279949); Cross reality system for large scale environment reconstruction.
Agarwal (U.S. Patent Application Publication No. 2021/0209859); Electronic system used to provide x reality or cross reality scenes, comprises electronic device that includes processor, computer-readable medium having first and second coordinate frames, data channel and display system.
Ahmed (U.S. Patent Application Publication No. 2025/0086830); Configuration method for the display of a building information model.
Huang (U.S. Patent Application Publication No. 2022/0245859); Data processing method and electronic device.
Xu (U.S. Patent Application Publication No. 2022/0292796); Method and system for associating device coordinate systems in a multi-person ar system.
Luh (U.S. Patent Application Publication No. 2025/0329105); Coordination between independent rendering frameworks.
Takehara (U.S. Patent Application Publication No. 2024/0221240); Display device, method of controlling display device, and computer-readable storage medium.
Melax (U.S. Patent Application Publication No. 2025/0104580); Optical artifact mitigation and content display enhancement.
Lai (U.S. Patent Application Publication No. 2025/0044599); Computing system with head wearable display.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached on Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights