DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-10 and 12-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-3, 5-11, 16, and 18 of U.S. Patent No. 12209880 in view of Gorur et al. (US 20200271450 A1), hereinafter Gorur.
Instant Application
18/982,220
Patented Application
US 12209880 B2
Claim 1
A method comprising:
-----------___________________________________
determining that a user who is wearing a mixed-reality (MR) system is located on a moveable surface;
___________________________________
acquiring scanning data of the moveable surface;
___________________________________
determining a type for the moveable surface;
___________________________________
based on the determined type, generating a three-dimensional (3D) boundary that approximates a shape for the type;
___________________________________
imposing the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data;
___________________________________
and using the scanning data to build or supplement a library of one or more local maps,
wherein the library of one or more local maps is representative of the moveable surface.
Claim 1
A method for enabling a library of one or more local maps to remain de-coupled from a global map, said method comprising:
___________________________________
determining that a mixed-reality (MR) system is located on a platform that is currently moving or that has an ability to readily move;
Claim 1 of 18/982,220 recites a user who is wearing a mixed-reality (MR) system. Claim 1 of US 12209880 B2 lacks a user who is wearing a mixed-reality (MR) system. Gorur discloses a user who is wearing a mixed-reality (MR) system (Paragraph 0144 – “FIG. 4A illustrates a side view 400 of a mobile platform 402 configured with sensors 132 and 134 for calculating relative pose information and providing an XR experience to a user 404 on the mobile platform 402. As can be seen in the side view 400, a user 404 wearing an HMD 408 is located within the mobile platform 402”; Note: the HMD is the MR system in this case). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified claim 1 of US 12209880 B2 to have a user wear the MR system because most MR systems are designed to be worn by a person for the benefit of providing an immersive experience that allows for mobility.
Patent claim 1 of US 12209880 B2 recites a platform that is currently moving or that has an ability to readily move. Therefore, patent claim 1 of US 12209880 B2 is in essence a “species” of the generic invention of application claim 1, which recites a moveable surface. It has been held that a generic invention is “anticipated” by a “species” within the scope of the generic invention. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993). The same reasoning applies to the dependent claims in later rows.
___________________________________
acquiring at least a threshold amount of scanning data that is representative of at least a portion of the platform;
Claim 1 of 18/982,220 recites scanning data of the moveable surface. Claim 1 of US 12209880 B2 lacks scanning data of the moveable surface. Gorur discloses scanning data of the moveable surface (Paragraph 0062, 0064, 0175 – “the one or more light emitters 136 can include a structured light sensor or device for scanning and/or determining the dimensions and movement of an object or scene…The pose estimation engine 104 in the virtual content processing system 102 can receive sensor data from the sensors 130, and use the sensor data to estimate a pose of one or more objects, track the one or more objects, and generate one or more maps of one or more real-world environments… The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”; Note: the object is the moveable surface). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified claim 1 of US 12209880 B2 to have scanning data of the moveable surface because it provides direct information on the location and structure of the moveable surface, rather than merely having information based on a representation. The direct information would allow for generating a more accurate map of the moveable surface.
___________________________________
based on the acquired scanning data, determining a type for the platform;
___________________________________
based on the determined type for the platform, generating a three-dimensional (3D) boundary that approximates a shape for the platform's type;
___________________________________
imposing the 3D boundary onto the platform; acquiring additional scanning data that is representative of additional portions of the platform, wherein a bounds of all scanning data that represents the platform is at least initially limited to that of the 3D boundary;
Patent claim 1 of US 12209880 B2 recites imposing the 3D boundary onto the platform. Therefore, patent claim 1 of US 12209880 B2 is in essence a “species” of the generic invention of application claim 1, which recites imposing the 3D boundary around the moveable surface. It has been held that a generic invention is “anticipated” by a “species” within the scope of the generic invention. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993).
___________________________________
using the scanning data that represents the platform to build or supplement a library of one or more local maps, wherein the library of one or more local maps is representative of the platform;
Claim 2
wherein acquiring the scanning data of the moveable surface includes acquiring at least a threshold amount of scanning data that is representative of at least a portion of the moveable surface.
Claim 1
acquiring at least a threshold amount of scanning data that is representative of at least a portion of the platform;
Claim 3
wherein the method includes acquiring additional scanning data that is representative of additional portions of the moveable surface.
Claim 1
acquiring additional scanning data that is representative of additional portions of the platform
Claim 4
wherein a bounds of all scanning data that represents the moveable surface is at least initially limited to that of the 3D boundary.
Claim 1
wherein a bounds of all scanning data that represents the platform is at least initially limited to that of the 3D boundary
Claim 5
wherein the scanning data is used to build the library.
Claim 1
using the scanning data that represents the platform to build or supplement a library of one or more local maps
Claim 6
wherein the scanning data is used to supplement the library.
Claim 1
using the scanning data that represents the platform to build or supplement a library of one or more local maps
Claim 7
wherein the method further includes preventing the library of one or more local maps from being coupled to a global map.
Claim 1
preventing the library of one or more local maps from being coupled to a global map.
Claim 8
wherein the method further includes: determining that the MR system has left the moveable surface in response to a determination that the MR system has left a confines of the 3D boundary.
Claim 2
wherein the method further includes: determining that the MR system has left the platform in response to a determination that the MR system has left a confines of the 3D boundary.
Claim 9
wherein the method further includes: subsequent to determining that the MR system has left the moveable surface, re-evaluating whether the MR system is still on the moveable surface, wherein said re-evaluating is performed in response to a determination that the MR system is still moving with a movement of the moveable surface.
Claim 3
wherein the method further includes: subsequent to determining that the MR system has left the platform, re-evaluating whether the MR system is still on the platform, wherein said re-evaluating is performed in response to a determination that the MR system is still moving with a movement of the platform;
Claim 10
wherein the method further includes expanding a size of the 3D boundary based on a determination that the MR system has not left the moveable surface.
Claim 3
expanding a size of the 3D boundary based on a determination that the MR system has not left the platform.
Claim 12
wherein determining that the user who is wearing the MR system is located on the moveable surface is based on one or more of motion data or image data.
Claim 5
wherein determining that the MR system is located on the platform is based on one or more of motion data or image data.
Claim 12 of 18/982,220 recites a user who is wearing the MR system. Claim 5 of US 12209880 B2 lacks a user who is wearing the MR system. Gorur discloses a user who is wearing the MR system (Paragraph 0144 – “FIG. 4A illustrates a side view 400 of a mobile platform 402 configured with sensors 132 and 134 for calculating relative pose information and providing an XR experience to a user 404 on the mobile platform 402. As can be seen in the side view 400, a user 404 wearing an HMD 408 is located within the mobile platform 402”; Note: the HMD is the MR system in this case). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified claim 5 of US 12209880 B2 to have a user wear the MR system because most MR systems are designed to be worn by a person for the benefit of providing an immersive experience that allows for mobility.
Claim 13
wherein determining the type for the moveable surface is based on one or more of motion data or image data.
Claim 6
wherein determining the type for the platform is based on one or more of motion data or image data.
Claim 14
wherein a shape of the 3D boundary is at least one of: a rectangular prism, a triangular prism, a cylinder, a cone, or a sphere.
Claim 7
wherein a shape of the 3D boundary is (i) a simple shape comprising at least one of: a rectangular prism, a triangular prism, a cylinder, a cone, or a sphere or, alternatively (ii) a custom platform specific shape.
Claim 15
wherein the scanning data that is representative of the moveable surface is generated by the MR system.
Claim 8
wherein the scanning data that is representative of the platform is generated by the MR system.
Claim 16
wherein the library of one or more local maps is merged with a second library of one or more maps, where the second library of one or more maps is also representative of the moveable surface.
Claim 9
wherein the library of one or more local maps is merged with a second library of one or more maps, where the second library of one or more maps is also representative of the platform.
Claim 17
wherein the method further includes: acquiring additional scanning data, wherein at least a portion of the additional scanning data represents content that is not a part of the moveable surface;
___________________________________
and appending a tag onto said portion of the additional scanning data, wherein the tag indicates that said portion of additional scanning data is not representative of the moveable surface and was generated by the MR system while the MR system was within a confines of the 3D boundary.
Claim 10
wherein the method further includes: acquiring additional scanning data, wherein at least a portion of the additional scanning data represents content that is not a part of the platform;
___________________________________
and appending a tag onto said portion of the additional scanning data, wherein the tag indicates that said portion of additional scanning data is not representative of the platform and was generated by the MR system while the MR system was within a confines of the 3D boundary.
Claim 18
wherein the method further includes uploading the tagged scanning data to a map aggregator.
Claim 11
wherein the method further includes uploading the tagged scanning data to a map aggregator.
Claim 19
A computer system comprising:
___________________________________
one or more processors;
___________________________________
and one or more hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to:
___________________________________
determine that a user who is wearing a mixed-reality (MR) system is located on a moveable surface;
___________________________________
acquire scanning data of the moveable surface;
___________________________________
determine a type for the moveable surface;
___________________________________
based on the determined type, generate a three-dimensional (3D) boundary that approximates a shape for the type;
___________________________________
impose the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data;
___________________________________
and use the scanning data to build or supplement a library of one or more local maps, wherein the library of one or more local maps is representative of the moveable surface.
Claim 16
A computer system that enables a library of one or more local maps to remain de-coupled from a global map, said computer system comprising:
___________________________________
at least one processor;
___________________________________
and at least one hardware storage device that stores instructions that are executable by the at least one processor to cause the computer system to:
___________________________________
determine that a mixed-reality (MR) system is located on a platform that is currently moving or that has an ability to readily move;
Claim 19 of 18/982,220 recites a user who is wearing a mixed-reality (MR) system. Claim 16 of US 12209880 B2 lacks a user who is wearing a mixed-reality (MR) system. Gorur discloses a user who is wearing a mixed-reality (MR) system (Paragraph 0144 – “FIG. 4A illustrates a side view 400 of a mobile platform 402 configured with sensors 132 and 134 for calculating relative pose information and providing an XR experience to a user 404 on the mobile platform 402. As can be seen in the side view 400, a user 404 wearing an HMD 408 is located within the mobile platform 402”; Note: the HMD is the MR system in this case). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified claim 16 of US 12209880 B2 to have a user wear the MR system because most MR systems are designed to be worn by a person for the benefit of providing an immersive experience that allows for mobility.
Patent claim 16 of US 12209880 B2 recites a platform that is currently moving or that has an ability to readily move. Therefore, patent claim 16 of US 12209880 B2 is in essence a “species” of the generic invention of application claim 19, which recites a moveable surface. It has been held that a generic invention is “anticipated” by a “species” within the scope of the generic invention. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993).
___________________________________
acquire scanning data that is representative of one or more portions of the platform,
Claim 19 of 18/982,220 recites scanning data of the moveable surface. Claim 16 of US 12209880 B2 lacks scanning data of the moveable surface. Gorur discloses scanning data of the moveable surface (Paragraph 0062, 0064, 0175 – “the one or more light emitters 136 can include a structured light sensor or device for scanning and/or determining the dimensions and movement of an object or scene…The pose estimation engine 104 in the virtual content processing system 102 can receive sensor data from the sensors 130, and use the sensor data to estimate a pose of one or more objects, track the one or more objects, and generate one or more maps of one or more real-world environments… The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”; Note: the object is the moveable surface). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified claim 16 of US 12209880 B2 to have scanning data of the moveable surface because it provides direct information on the location and structure of the moveable surface, rather than merely having information based on a representation. The direct information would allow for generating a more accurate map of the moveable surface.
___________________________________
determine a type for the platform;
___________________________________
based on the determined type for the platform, generate a three-dimensional (3D) boundary that that approximates a shape for the platform's type;
___________________________________
impose the 3D boundary onto the platform;… wherein a bounds of all scanning data is at least initially limited to that of the 3D boundary;
Patent claim 16 of US 12209880 B2 recites imposing the 3D boundary onto the platform. Therefore, patent claim 16 of US 12209880 B2 is in essence a “species” of the generic invention of application claim 19, which recites imposing the 3D boundary around the moveable surface. It has been held that a generic invention is “anticipated” by a “species” within the scope of the generic invention. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993).
___________________________________
use the scanning data to build or supplement a library of one or more local maps, wherein the library of one or more local maps is representative of the platform;
Claim 20
A mixed-reality (MR) system comprising:
___________________________________
one or more processors;
___________________________________
and one or more hardware storage devices that store instructions that are executable by the one or more processors to cause the MR system to:
___________________________________
determine that a user who is wearing the MR system is located on a moveable surface;
___________________________________
acquire scanning data of the moveable surface;
___________________________________
determine a type for the moveable surface;
___________________________________
based on the determined type, generate a three-dimensional (3D) boundary that approximates a shape for the type;
___________________________________
impose the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data;
___________________________________
and use the scanning data to build or supplement a library of one or more local maps, wherein the library of one or more local maps is representative of the moveable surface.
Claim 18
A computer system that enables a library of one or more local maps to remain de-coupled from a global map, said computer system comprising:
…wherein the computer system is the MR system
___________________________________
at least one processor;
___________________________________
and at least one hardware storage device that stores instructions that are executable by the at least one processor to cause the computer system to:
___________________________________
determine that a mixed-reality (MR) system is located on a platform that is currently moving or that has an ability to readily move;
Claim 20 of 18/982,220 recites a user who is wearing a mixed-reality (MR) system. Claim 18 of US 12209880 B2 lacks a user who is wearing a mixed-reality (MR) system. Gorur discloses a user who is wearing a mixed-reality (MR) system (Paragraph 0144 – “FIG. 4A illustrates a side view 400 of a mobile platform 402 configured with sensors 132 and 134 for calculating relative pose information and providing an XR experience to a user 404 on the mobile platform 402. As can be seen in the side view 400, a user 404 wearing an HMD 408 is located within the mobile platform 402”; Note: the HMD is the MR system in this case). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified claim 18 of US 12209880 B2 to have a user wear the MR system because most MR systems are designed to be worn by a person for the benefit of providing an immersive experience that allows for mobility.
Patent claim 18 of US 12209880 B2 recites a platform that is currently moving or that has an ability to readily move. Therefore, patent claim 18 of US 12209880 B2 is in essence a “species” of the generic invention of application claim 20, which recites a moveable surface. It has been held that a generic invention is “anticipated” by a “species” within the scope of the generic invention. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993).
___________________________________
acquire scanning data that is representative of one or more portions of the platform,
Claim 20 of 18/982,220 recites scanning data of the moveable surface. Claim 18 of US 12209880 B2 lacks scanning data of the moveable surface. Gorur discloses scanning data of the moveable surface (Paragraph 0062, 0064, 0175 – “the one or more light emitters 136 can include a structured light sensor or device for scanning and/or determining the dimensions and movement of an object or scene…The pose estimation engine 104 in the virtual content processing system 102 can receive sensor data from the sensors 130, and use the sensor data to estimate a pose of one or more objects, track the one or more objects, and generate one or more maps of one or more real-world environments… The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”; Note: the object is the moveable surface). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified claim 18 of US 12209880 B2 to have scanning data of the moveable surface because it provides direct information on the location and structure of the moveable surface, rather than merely having information based on a representation. The direct information would allow for generating a more accurate map of the moveable surface.
___________________________________
determine a type for the platform;
___________________________________
based on the determined type for the platform, generate a three-dimensional (3D) boundary that approximates a shape for the platform's type;
___________________________________
impose the 3D boundary onto the platform;… wherein a bounds of all scanning data that represents the platform is at least initially limited to that of the 3D boundary;
Patent claim 18 of US 12209880 B2 recites imposing the 3D boundary onto the platform. Therefore, patent claim 18 of US 12209880 B2 is in essence a “species” of the generic invention of application claim 20, which recites imposing the 3D boundary around the moveable surface. It has been held that a generic invention is “anticipated” by a “species” within the scope of the generic invention. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993).
___________________________________
use the scanning data that represents the platform to build or supplement a library of one or more local maps, wherein the library of one or more local maps is representative of the platform;
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-8, 12-15, 17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gorur et al. (US 20200271450 A1) in view of Hill (US 10885704 B1), hereinafter Gorur and Hill respectively.
Regarding claim 1, Gorur teaches a method (Paragraph 0010 – “a method is provided for immersive extended reality experiences on mobile platforms”) comprising:
determining that a user who is wearing a mixed-reality (MR) system is located on a moveable surface (Fig. 4A, Paragraph 0084, 0086, 0144 – “in use cases where a user is within a mobile platform (e.g., a vehicle, an elevator, a train, a conveyor belt, a vessel, an aircraft, a boat, a skateboard, a bicycle, a scooter, a conveyance, etc.) that moves relative to an external environment or scene, the process 200 can provide virtual content that matches the perceived motion (e.g., due to inertial forces) of the user in the mobile platform and accounts for the view or pose of the user… the process 200 can track the pose of a user within the mobile platform…FIG. 4A illustrates a side view 400 of a mobile platform 402 configured with sensors 132 and 134 for calculating relative pose information and providing an XR experience to a user 404 on the mobile platform 402. As can be seen in the side view 400, a user 404 wearing an HMD 408 is located within the mobile platform 402. In this example, the mobile platform 402 is a car”; Note: The HMD is equivalent to the MR system, and the mobile platform is equivalent to the moveable surface. It is implied that it is determined that the user wearing an HMD is located on the mobile platform because the pose of the user is tracked within the mobile platform);
PNG
media_image1.png
317
517
media_image1.png
Greyscale
Screenshot of Fig. 4A (taken from Gorur)
acquiring scanning data of the moveable surface (Paragraph 0062, 0064, 0175 – “the one or more light emitters 136 can include a structured light sensor or device for scanning and/or determining the dimensions and movement of an object or scene… The pose estimation engine 104 in the virtual content processing system 102 can receive sensor data from the sensors 130, and use the sensor data to estimate a pose of one or more objects, track the one or more objects, and generate one or more maps of one or more real-world environments… The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”; Note: scanning data is acquired for an object. The object in this case is a moveable surface, such as a car);
determining a type for the moveable surface (Paragraph 0175 – “The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”);
generating a three-dimensional (3D) boundary (Paragraph 0175 – “The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene) and the localization can include a bounding box indicating the location of the object or scene”; Note: the bounding box is a 3D boundary for the object);
limiting the scanning data (Paragraph 0169 – “The features 814 and 816 can be tracked based on sensors 130 on the car 812 and/or a device associated with a user on the elevator, such as an HMD or mobile computing device…instead of using a global map (either in addition to or in lieu of the mobile map) to track the features 814 and 816, the features 814 and 816 may be tracked from only map points inside of the mobile map in order to avoid potential errors in trajectory that may result when tracking features in map points of a global map associated with the car 812”; Note: the feature sensor/scanning data is limited to the area inside the mobile map);
and using the scanning data to build or supplement a library of one or more local maps, wherein the library of one or more local maps is representative of the moveable surface (Paragraph 0070, 0072– “The mapper 108 can use data from the sensors 130 to generate one or more maps or representations of one or more environments, such as the mobile platform and the outside environment…the mapper 108 can generate a mobile map that charts, plots, models, or identifies objects, space, and/or characteristics (e.g., shape, volume, size, position, etc.) of the mobile platform, and a global map that charts, plots, models or identifies objects, space, and/or characteristics of the outside environment… the mapper 108 or pose estimation engine 104 can store any maps generated in a maps store 124 for use in providing XR experiences to users. The maps store 124 can be a storage or repository of maps available for one or more environments, such as the mobile platform and/or the outside environment”; Note: the sensor data is used to build or add to a mobile map, which is equivalent to the local map representing the moveable surface. The stored mobile maps make-up a library).
Gorur does not teach based on the determined type, generating a three-dimensional (3D) boundary that approximates a shape for the type; imposing the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data. However, Hill teaches based on the determined type, generating a three-dimensional (3D) boundary that approximates a shape for the type (Col. 15 lines 6-17 – “FIG. 11 shows how the disclosed principles can also be performed in connection with any type of vehicle 1100 (e.g., a car, airplane, train, etc.)…Because vehicles often move, it is often desirable to map out only the inside area 1110 and to refrain from mapping out the external environment. In this regard, the environment that is desired to be scanned may be an inside area of a frame of a vehicle, and the boundary may include the frame of the vehicle, including any windows of the vehicle”; Note: a boundary is generated for a vehicle, and it is implied to approximate a shape for the vehicle because the boundary includes frames of the vehicle. The frames of the vehicle logically depend on the vehicle type, as each type of vehicle may have different types of frames); and imposing the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data (Col. 15 lines 26-39 – “Once this scanning data is acquired, then the embodiments can identify depths corresponding to the inside area 1110…if the vehicle 1100 is moving, then the depths for the outside area may be skewed, inaccurate, or otherwise ghosted. By imputing the depths for the frame 1105 to the area corresponding to the window 1115, the embodiments can configure the window 1115 to operate as the boundary. Subsequently, the embodiments can then selectively filter out the outside area depths and rely only on the depth data for the inside area 1110 to generate a 3D representation for the inside area 1110”; Note: the window is established to be a boundary for the vehicle, which is a moveable surface. The window limits the scanning data to be inside. It is implied that the boundary is 3D because the window is a 3D object). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Hill to generate 3D boundary that approximates a shape for the type based on the determined type because knowing the type would help provide a more accurate estimation of the geometry of the actual surface, and as a result, the boundary would better resemble the surface. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Hill to impose the 3D boundary around the moveable surface and have the boundary limit the scanning data for the benefit of knowing what data corresponds to the surface, which would help determine the surface position and the user’s position in relation to the surface, separate from the external environment, in order to prevent inconsistencies. This concern is expressed by Gorur: “when anchoring on object relative to a road sign visible through a window of a moving car, the motion of the user within the car (e.g., which can result from head movements, posture changes, etc.) and the relative movement of other features based on the trajectory of the car (e.g., the global motion of the car) can create inconsistent results (and errors) in the XR experience. This can be especially frustrating in scenes where the mobile platform is moving at a high rate of speed, which can result in increasingly inconsistent or misaligned XR experiences. Accordingly, to accurately match the virtual content with the perceived motion of the user and limit or eliminate any errors and inconsistencies in the XR experience, the process 200 can track the pose of a user within the mobile platform, which can be represented by a mobile map or local motion map, and the pose of the mobile platform relative to the external scene, which can be represented by a global or world map” (Paragraph 0085-0086).
Regarding claim 2, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches wherein acquiring the scanning data of the moveable surface includes acquiring at least a threshold amount of scanning data that is representative of at least a portion of the moveable surface (Paragraph 0010, 0020 – “The method can include obtaining sensor measurements from one or more sensors on a mobile platform and/or a device associated with a user in the mobile platform. The sensor measurements can include motion parameters associated with the mobile platform and the user in the mobile platform. The method can further include identifying features of the mobile platform and features of an environment outside of the mobile platform; tracking, using the sensor measurements, a first pose of the mobile platform relative to the environment outside of the mobile platform; tracking, using the sensor measurements, a second pose of the user relative to at least one of the features of the mobile platform… at least one additional sensor can be configured to detect one or more features associated with the mobile platform and the environment outside the mobile platform”; Note: it is implied that a threshold amount of sensor/scanning data is acquired because there needs to be “at least” one feature of the mobile platform (moveable surface) detected by a sensor in order to perform the tracking process. The “at least” one feature is a threshold).
Regarding claim 3, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches acquiring additional scanning data that is representative of additional portions of the moveable surface (Paragraph 0020 – “at least one additional sensor can be configured to detect one or more features associated with the mobile platform and the environment outside the mobile platform. The one or more features can include motion parameters and/or scene properties”; Note: the features of the mobile platform are equivalent to the additional scanning data since it comes from an additional sensor).
Regarding claim 4, Gorur in view of Hill teaches the method of claim 3. Gorur does not teach wherein a bounds of all scanning data that represents the moveable surface is at least initially limited to that of the 3D boundary. However, Hill teaches wherein a bounds of all scanning data that represents the moveable surface is at least initially limited to that of the 3D boundary (Col. 15 lines 26-39 – “Once this scanning data is acquired, then the embodiments can identify depths corresponding to the inside area 1110…if the vehicle 1100 is moving, then the depths for the outside area may be skewed, inaccurate, or otherwise ghosted. By imputing the depths for the frame 1105 to the area corresponding to the window 1115, the embodiments can configure the window 1115 to operate as the boundary. Subsequently, the embodiments can then selectively filter out the outside area depths and rely only on the depth data for the inside area 1110 to generate a 3D representation for the inside area 1110”; Note: the window is established to be a boundary for the vehicle, which is a moveable surface. The window limits the scanning data to be inside the vehicle. It is implied that the boundary is 3D because the window is a 3D object). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Hill to limit a bounds of all scanning data of the moveable surface to that of the 3D boundary for the benefit of being able to separate the surface data from the environment data, which can help prevent inconsistences. This concern is expressed by Gorur: “when anchoring on object relative to a road sign visible through a window of a moving car, the motion of the user within the car (e.g., which can result from head movements, posture changes, etc.) and the relative movement of other features based on the trajectory of the car (e.g., the global motion of the car) can create inconsistent results (and errors) in the XR experience. This can be especially frustrating in scenes where the mobile platform is moving at a high rate of speed, which can result in increasingly inconsistent or misaligned XR experiences. Accordingly, to accurately match the virtual content with the perceived motion of the user and limit or eliminate any errors and inconsistencies in the XR experience, the process 200 can track the pose of a user within the mobile platform, which can be represented by a mobile map or local motion map, and the pose of the mobile platform relative to the external scene, which can be represented by a global or world map” (Paragraph 0085-0086).
Regarding claim 5, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches wherein the scanning data is used to build the library (Paragraph 0070, 0072– “The mapper 108 can use data from the sensors 130 to generate one or more maps or representations of one or more environments, such as the mobile platform and the outside environment…the mapper 108 can generate a mobile map that charts, plots, models, or identifies objects, space, and/or characteristics (e.g., shape, volume, size, position, etc.) of the mobile platform, and a global map that charts, plots, models or identifies objects, space, and/or characteristics of the outside environment… the mapper 108 or pose estimation engine 104 can store any maps generated in a maps store 124 for use in providing XR experiences to users. The maps store 124 can be a storage or repository of maps available for one or more environments, such as the mobile platform and/or the outside environment”; Note: the sensor data is used to build maps that make-up a library/store).
Regarding claim 6, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches wherein the scanning data is used to supplement the library (Paragraph 0072, 0139 – “the mapper 108 or pose estimation engine 104 can store any maps generated in a maps store 124 for use in providing XR experiences to users…if the mapper 108 determines that the user has entered a new mobile platform or environment and a corresponding or matching map is not available in the maps store 124, the mapper 108 can create a new map for the new mobile platform or environment. The mapper 108 can generate a new map using the sensor data 208”; Note: the sensor data is used to generate a new map to supplement the already stored maps).
Regarding claim 7, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches preventing the library of one or more local maps from being coupled to a global map (Paragraph 0169 – “The features 814 and 816 can be tracked based on sensors 130 on the car 812 and/or a device associated with a user on the elevator, such as an HMD or mobile computing device…instead of using a global map (either in addition to or in lieu of the mobile map) to track the features 814 and 816, the features 814 and 816 may be tracked from only map points inside of the mobile map in order to avoid potential errors in trajectory that may result when tracking features in map points of a global map associated with the car 812”; Note: only features from a mobile/local map are used, meaning that it is not coupled to features from a global map. The mobile map corresponds to a library, as explained in the rejection of claim 1).
Regarding claim 8, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches determining that the MR system has left the moveable surface in response to a determination that the MR system has left a confines of the 3D boundary (Paragraph 0076-0077, 0170 – “the environment classifier 114 can detect whether a user has left a first mobile map or platform (e.g., an elevator) and has entered a different mobile map or platform (e.g., a car)…The environment classifier 114 can detect a change in the applicable mobile map or platform based on one or more images received from the sensors 130. The one or more images can capture a scene or view of the mobile map or platform applicable to the user. The environment classifier 114 can process the one or more images to classify the user's current environment… FIG. 9 illustrates a diagram of a scheme 900 for detecting when a user leaves a mobile platform (and/or associated mobile map) and enters a new mobile platform (and/or associated mobile map). In this example, a user located in the elevator 802 leaves and enters the car 812. To detect that the user has entered the new mobile platform (e.g., the car 812), an image of the elevator 802 and an image of the car 812 (e.g., captured by one or more image sensors 134) can be processed”; Note: it is detected that the user enters a new mobile platform, meaning the user left the confines of a mobile map of a previous mobile platform (moveable surface). The mobile map is a boundary. In response to the detection, it is determined that the user left the mobile platform (moveable surface). The 3D boundary was previously taught by Hill in the rejection of claim 1).
Regarding claim 12, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches wherein determining that the user who is wearing the MR system is located on the moveable surface is based on one or more of motion data or image data (Paragraph 0076-0077 – “The virtual content processing system 102 can also include an environment classifier 114 to identify a user's environment (e.g., a specific mobile platform) and detect whether a user has entered, or is entering, a different mobile map or mobile platform…The environment classifier 114 can detect a change in the applicable mobile map or platform based on one or more images received from the sensors 130. The one or more images can capture a scene or view of the mobile map or platform applicable to the user. The environment classifier 114 can process the one or more images to classify the user's current environment”; Note: image data is used to determine if the user of the virtual content processing system (equivalent to MR system) has entered a mobile platform (equivalent to moveable surface)).
Regarding claim 13, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches wherein determining the type for the moveable surface is based on one or more of motion data or image data (Paragraph 0175 – “the output layer 1006 can provide a classification and/or localization of one or more objects in an input image. The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”; Note: the classification of the object type is based on an input image).
Regarding claim 14, Gorur in view of Hill teaches the method of claim 1. Gorur does not teach wherein a shape of the 3D boundary is at least one of: a rectangular prism, a triangular prism, a cylinder, a cone, or a sphere. However, Hill teaches wherein a shape of the 3D boundary is at least one of: a rectangular prism, a triangular prism, a cylinder, a cone, or a sphere (Fig. 7 – The figure shows a 3D boundary that is a rectangular prism; see screenshot of Fig. 7 below). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Hill to have the shape of the 3D boundary to be a rectangular prism because the rectangular prism is a simple shape that can easily encompass many different types of moveable surfaces, such as cars or elevators. In Gorur, examples of moveable surfaces were cars and elevators (see Fig. 8A and 8B).
PNG
media_image2.png
705
1104
media_image2.png
Greyscale
Screenshot of Fig. 7 (taken from Hill)
Regarding claim 15, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches wherein the scanning data that is representative of the moveable surface is generated by the MR system (Paragraph 0061, 0066, 0069-0070 – “one or more of the sensors 130 can be mounted on (or implemented by) the virtual content processing system 102. To illustrate, the virtual content processing system 102 can include an IMU (132), an image sensor (134), and/or a GPS device (138)… The tracker 106 can use sensor data from sensors (130)… the tracker 106 can estimate and track the pose of the mobile platform…The mapper 108 can use data from the sensors 130 to generate one or more maps or representations of one or more environments, such as the mobile platform”; Note: the sensor data is generated by the sensors mounted on the virtual content processing system (MR system). The sensor data represents data of the mobile platform (moveable surface)).
Regarding claim 17, Gorur in view of Hill teaches the method of claim 1. Gorur further teaches wherein at least a portion of the scanning data represents content that is not a part of the moveable surface (Paragraph 0164, 0166 – “The features 804 can include visual features outside of a mobile map associated with the elevator 802. For example, the features 804 can include features of an outside of the elevator 802 (e.g., features in a view of the outside environment). Moreover, the features 804 can represent features on a global map associated with the elevator 802, which maps the outside view or environment visible from the elevator 802…The features 804 and 806 can be tracked based on sensors 130 on the elevator 802 and/or a device associated with a user on the elevator, such as an HMD or mobile computing device”; Note: a portion of the features detected by the sensors are not part of the moveable surface, which in this case is the elevator); and appending a tag onto said portion of the scanning data (Fig. 8A, Paragraph 0164, 0166 – “The features 804 can include visual features outside of a mobile map associated with the elevator 802. For example, the features 804 can include features of an outside of the elevator 802 (e.g., features in a view of the outside environment). Moreover, the features 804 can represent features on a global map associated with the elevator 802, which maps the outside view or environment visible from the elevator 802…The features 804 and 806 can be tracked based on sensors 130 on the elevator 802 and/or a device associated with a user on the elevator, such as an HMD or mobile computing device”; Note: Fig. 8A shows labels of features outside the mobile map, which are not part of the elevator moveable surface; see modified screenshot of Fig. 8A below. These labels are equivalent to the tag), wherein the tag indicates that said portion of scanning data is not representative of the moveable surface and was generated by the MR system while the MR system was within a confines of the 3D boundary (Fig. 8A, Paragraph 0164, 0166 – “The features 804 can include visual features outside of a mobile map associated with the elevator 802. For example, the features 804 can include features of an outside of the elevator 802 (e.g., features in a view of the outside environment). Moreover, the features 804 can represent features on a global map associated with the elevator 802, which maps the outside view or environment visible from the elevator 802…The features 804 and 806 can be tracked based on sensors 130 on the elevator 802 and/or a device associated with a user on the elevator, such as an HMD or mobile computing device”; Note: Fig. 8A shows labels/tags of features outside the mobile map, which are not part of the elevator moveable surface; see modified screenshot of Fig. 8A below. The figure also shows that the image view was captured from within the mobile map of an elevator, the mobile map being the boundary). Gorur does not directly teach the “additional scanning data” in the limitations: “wherein at least a portion of the additional scanning data represents content that is not a part of the moveable surface; and appending a tag onto said portion of the additional scanning data, wherein the tag indicates that said portion of additional scanning data is not representative of the moveable surface and was generated by the MR system while the MR system was within a confines of the 3D boundary”. However, Gorur separately teaches acquiring additional scanning data (Paragraph 0130 – “In addition, the tracker 106 can optionally (as denoted by the dashed outline of block 272) receive auxiliary sensor data 272”). A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that the scanning data could have been substituted for the additional scanning data because both the scanning data and additional scanning data serve the purpose of representing an object or scene. Furthermore, a person of ordinary skill in the art would have been able to carry out the substitution. Finally, the substitution achieves the predictable result of appending a tag to the data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the scanning data for the additional scanning data according to known methods to yield the predictable result of appending a tag to the data.
PNG
media_image3.png
450
620
media_image3.png
Greyscale
Modified Screenshot of Fig. 8A (taken from Gorur)
Regarding claim 19, Gorur teaches a computer system (Paragraph 0059 – “The virtual content processing system 102 can be part of, or implemented by, one or more computing devices”) comprising:
one or more processors (Paragraph 0058 – “The components of the virtual content processing system 102 can include and/or can be implemented using electronic circuits or other electronic hardware, which can include, for example, one or more programmable electronic circuits (e.g., microprocessors, graphics processing units (GPUs), digital signal processors (DSPs), central processing units (CPUs), image signal processors (ISPs)”);
and one or more hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to (Paragraph 0219, 0222 – “the computing device architecture 1300 can implement the virtual content processing system 102 shown in FIG. 1…Storage device 1330 is a non-volatile memory and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs) 1325, read only memory (ROM) 1320, and hybrids thereof. The storage device 1330 can include services 1332, 1334, 1336 for controlling the processor 1310”):
determine that a user who is wearing a mixed-reality (MR) system is located on a moveable surface (Fig. 4A, Paragraph 0084, 0086, 0144 – “in use cases where a user is within a mobile platform (e.g., a vehicle, an elevator, a train, a conveyor belt, a vessel, an aircraft, a boat, a skateboard, a bicycle, a scooter, a conveyance, etc.) that moves relative to an external environment or scene, the process 200 can provide virtual content that matches the perceived motion (e.g., due to inertial forces) of the user in the mobile platform and accounts for the view or pose of the user… the process 200 can track the pose of a user within the mobile platform…FIG. 4A illustrates a side view 400 of a mobile platform 402 configured with sensors 132 and 134 for calculating relative pose information and providing an XR experience to a user 404 on the mobile platform 402. As can be seen in the side view 400, a user 404 wearing an HMD 408 is located within the mobile platform 402. In this example, the mobile platform 402 is a car”; Note: The HMD is equivalent to the MR system, and the mobile platform is equivalent to the moveable surface. It is implied that it is determined that the user wearing an HMD is located on the mobile platform because the pose of the user is tracked within the mobile platform);
acquire scanning data of the moveable surface (Paragraph 0062, 0064, 0175 – “the one or more light emitters 136 can include a structured light sensor or device for scanning and/or determining the dimensions and movement of an object or scene… The pose estimation engine 104 in the virtual content processing system 102 can receive sensor data from the sensors 130, and use the sensor data to estimate a pose of one or more objects, track the one or more objects, and generate one or more maps of one or more real-world environments… The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”; Note: scanning data is acquired for an object. The object in this case is a moveable surface, such as a car);
determine a type for the moveable surface (Paragraph 0175 – “The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”);
generate a three-dimensional (3D) boundary (Paragraph 0175 – “The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene) and the localization can include a bounding box indicating the location of the object or scene”; Note: the bounding box is a 3D boundary for the object);
limit the scanning data (Paragraph 0169 – “The features 814 and 816 can be tracked based on sensors 130 on the car 812 and/or a device associated with a user on the elevator, such as an HMD or mobile computing device…instead of using a global map (either in addition to or in lieu of the mobile map) to track the features 814 and 816, the features 814 and 816 may be tracked from only map points inside of the mobile map in order to avoid potential errors in trajectory that may result when tracking features in map points of a global map associated with the car 812”; Note: the feature sensor/scanning data is limited to the area inside the mobile map);
and use the scanning data to build or supplement a library of one or more local maps, wherein the library of one or more local maps is representative of the moveable surface (Paragraph 0070, 0072– “The mapper 108 can use data from the sensors 130 to generate one or more maps or representations of one or more environments, such as the mobile platform and the outside environment…the mapper 108 can generate a mobile map that charts, plots, models, or identifies objects, space, and/or characteristics (e.g., shape, volume, size, position, etc.) of the mobile platform, and a global map that charts, plots, models or identifies objects, space, and/or characteristics of the outside environment… the mapper 108 or pose estimation engine 104 can store any maps generated in a maps store 124 for use in providing XR experiences to users. The maps store 124 can be a storage or repository of maps available for one or more environments, such as the mobile platform and/or the outside environment”; Note: the sensor data is used to build or add to a mobile map, which is equivalent to the local map representing the moveable surface. The stored mobile maps make-up a library).
Gorur does not teach based on the determined type, generating a three-dimensional (3D) boundary that approximates a shape for the type; imposing the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data. However, Hill teaches based on the determined type, generating a three-dimensional (3D) boundary that approximates a shape for the type (Col. 15 lines 6-17 – “FIG. 11 shows how the disclosed principles can also be performed in connection with any type of vehicle 1100 (e.g., a car, airplane, train, etc.)…Because vehicles often move, it is often desirable to map out only the inside area 1110 and to refrain from mapping out the external environment. In this regard, the environment that is desired to be scanned may be an inside area of a frame of a vehicle, and the boundary may include the frame of the vehicle, including any windows of the vehicle”; Note: a boundary is generated for a vehicle, and it is implied to approximate a shape for the vehicle because the boundary includes frames of the vehicle. The frames of the vehicle logically depend on the vehicle type, as each type of vehicle may have different types of frames); and imposing the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data (Col. 15 lines 26-39 – “Once this scanning data is acquired, then the embodiments can identify depths corresponding to the inside area 1110…if the vehicle 1100 is moving, then the depths for the outside area may be skewed, inaccurate, or otherwise ghosted. By imputing the depths for the frame 1105 to the area corresponding to the window 1115, the embodiments can configure the window 1115 to operate as the boundary. Subsequently, the embodiments can then selectively filter out the outside area depths and rely only on the depth data for the inside area 1110 to generate a 3D representation for the inside area 1110”; Note: the window is established to be a boundary for the vehicle, which is a moveable surface. The window limits the scanning data to be inside. It is implied that the boundary is 3D because the window is a 3D object). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Hill to generate 3D boundary that approximates a shape for the type based on the determined type because knowing the type would help provide a more accurate estimation of the geometry of the actual surface, and as a result, the boundary would better resemble the surface. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Hill to impose the 3D boundary around the moveable surface and have the boundary limit the scanning data for the benefit of knowing what data corresponds to the surface, which would help determine the surface position and the user’s position in relation to the surface, separate from the external environment, in order to prevent inconsistencies. This concern is expressed by Gorur: “when anchoring on object relative to a road sign visible through a window of a moving car, the motion of the user within the car (e.g., which can result from head movements, posture changes, etc.) and the relative movement of other features based on the trajectory of the car (e.g., the global motion of the car) can create inconsistent results (and errors) in the XR experience. This can be especially frustrating in scenes where the mobile platform is moving at a high rate of speed, which can result in increasingly inconsistent or misaligned XR experiences. Accordingly, to accurately match the virtual content with the perceived motion of the user and limit or eliminate any errors and inconsistencies in the XR experience, the process 200 can track the pose of a user within the mobile platform, which can be represented by a mobile map or local motion map, and the pose of the mobile platform relative to the external scene, which can be represented by a global or world map” (Paragraph 0085-0086).
Regarding claim 20, Gorur teaches a mixed-reality (MR) system (Paragraph 0054, 0057, 0059 – “extended reality (XR) technologies can combine real or physical environments and virtual environments (and/or virtual content) to provide users with extended reality experiences (e.g., virtual reality, augmented reality, mixed reality, etc.)…The virtual content processing system 102 can be implemented to provide immersive XR experiences…The virtual content processing system 102 can be part of, or implemented by, one or more computing devices”) comprising:
one or more processors (Paragraph 0058 – “The components of the virtual content processing system 102 can include and/or can be implemented using electronic circuits or other electronic hardware, which can include, for example, one or more programmable electronic circuits (e.g., microprocessors, graphics processing units (GPUs), digital signal processors (DSPs), central processing units (CPUs), image signal processors (ISPs)”);
and one or more hardware storage devices that store instructions that are executable by the one or more processors to cause the MR system to (Paragraph 0219, 0222 – “the computing device architecture 1300 can implement the virtual content processing system 102 shown in FIG. 1…Storage device 1330 is a non-volatile memory and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs) 1325, read only memory (ROM) 1320, and hybrids thereof. The storage device 1330 can include services 1332, 1334, 1336 for controlling the processor 1310”):
determine that a user who is wearing a mixed-reality (MR) system is located on a moveable surface (Fig. 4A, Paragraph 0084, 0086, 0144 – “in use cases where a user is within a mobile platform (e.g., a vehicle, an elevator, a train, a conveyor belt, a vessel, an aircraft, a boat, a skateboard, a bicycle, a scooter, a conveyance, etc.) that moves relative to an external environment or scene, the process 200 can provide virtual content that matches the perceived motion (e.g., due to inertial forces) of the user in the mobile platform and accounts for the view or pose of the user… the process 200 can track the pose of a user within the mobile platform…FIG. 4A illustrates a side view 400 of a mobile platform 402 configured with sensors 132 and 134 for calculating relative pose information and providing an XR experience to a user 404 on the mobile platform 402. As can be seen in the side view 400, a user 404 wearing an HMD 408 is located within the mobile platform 402. In this example, the mobile platform 402 is a car”; Note: The HMD is equivalent to the MR system, and the mobile platform is equivalent to the moveable surface. It is implied that it is determined that the user wearing an HMD is located on the mobile platform because the pose of the user is tracked within the mobile platform);
acquire scanning data of the moveable surface (Paragraph 0062, 0064, 0175 – “the one or more light emitters 136 can include a structured light sensor or device for scanning and/or determining the dimensions and movement of an object or scene… The pose estimation engine 104 in the virtual content processing system 102 can receive sensor data from the sensors 130, and use the sensor data to estimate a pose of one or more objects, track the one or more objects, and generate one or more maps of one or more real-world environments… The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”; Note: scanning data is acquired for an object. The object in this case is a moveable surface, such as a car);
determine a type for the moveable surface (Paragraph 0175 – “The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene)”);
generate a three-dimensional (3D) boundary (Paragraph 0175 – “The classification can include a class identifying the type of object or scene (e.g., a car, an elevator, a train, a vessel, an aircraft, or any other object or scene) and the localization can include a bounding box indicating the location of the object or scene”; Note: the bounding box is a 3D boundary for the object);
limit the scanning data (Paragraph 0169 – “The features 814 and 816 can be tracked based on sensors 130 on the car 812 and/or a device associated with a user on the elevator, such as an HMD or mobile computing device…instead of using a global map (either in addition to or in lieu of the mobile map) to track the features 814 and 816, the features 814 and 816 may be tracked from only map points inside of the mobile map in order to avoid potential errors in trajectory that may result when tracking features in map points of a global map associated with the car 812”; Note: the feature sensor/scanning data is limited to the area inside the mobile map);
and use the scanning data to build or supplement a library of one or more local maps, wherein the library of one or more local maps is representative of the moveable surface (Paragraph 0070, 0072– “The mapper 108 can use data from the sensors 130 to generate one or more maps or representations of one or more environments, such as the mobile platform and the outside environment…the mapper 108 can generate a mobile map that charts, plots, models, or identifies objects, space, and/or characteristics (e.g., shape, volume, size, position, etc.) of the mobile platform, and a global map that charts, plots, models or identifies objects, space, and/or characteristics of the outside environment… the mapper 108 or pose estimation engine 104 can store any maps generated in a maps store 124 for use in providing XR experiences to users. The maps store 124 can be a storage or repository of maps available for one or more environments, such as the mobile platform and/or the outside environment”; Note: the sensor data is used to build or add to a mobile map, which is equivalent to the local map representing the moveable surface. The stored mobile maps make-up a library).
Gorur does not teach based on the determined type, generating a three-dimensional (3D) boundary that approximates a shape for the type; imposing the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data. However, Hill teaches based on the determined type, generating a three-dimensional (3D) boundary that approximates a shape for the type (Col. 15 lines 6-17 – “FIG. 11 shows how the disclosed principles can also be performed in connection with any type of vehicle 1100 (e.g., a car, airplane, train, etc.)…Because vehicles often move, it is often desirable to map out only the inside area 1110 and to refrain from mapping out the external environment. In this regard, the environment that is desired to be scanned may be an inside area of a frame of a vehicle, and the boundary may include the frame of the vehicle, including any windows of the vehicle”; Note: a boundary is generated for a vehicle, and it is implied to approximate a shape for the vehicle because the boundary includes frames of the vehicle. The frames of the vehicle logically depend on the vehicle type, as each type of vehicle may have different types of frames); and imposing the 3D boundary around the moveable surface, wherein the 3D boundary limits the scanning data (Col. 15 lines 26-39 – “Once this scanning data is acquired, then the embodiments can identify depths corresponding to the inside area 1110…if the vehicle 1100 is moving, then the depths for the outside area may be skewed, inaccurate, or otherwise ghosted. By imputing the depths for the frame 1105 to the area corresponding to the window 1115, the embodiments can configure the window 1115 to operate as the boundary. Subsequently, the embodiments can then selectively filter out the outside area depths and rely only on the depth data for the inside area 1110 to generate a 3D representation for the inside area 1110”; Note: the window is established to be a boundary for the vehicle, which is a moveable surface. The window limits the scanning data to be inside. It is implied that the boundary is 3D because the window is a 3D object). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Hill to generate 3D boundary that approximates a shape for the type based on the determined type because knowing the type would help provide a more accurate estimation of the geometry of the actual surface, and as a result, the boundary would better resemble the surface. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Hill to impose the 3D boundary around the moveable surface and have the boundary limit the scanning data for the benefit of knowing what data corresponds to the surface, which would help determine the surface position and the user’s position in relation to the surface, separate from the external environment, in order to prevent inconsistencies. This concern is expressed by Gorur: “when anchoring on object relative to a road sign visible through a window of a moving car, the motion of the user within the car (e.g., which can result from head movements, posture changes, etc.) and the relative movement of other features based on the trajectory of the car (e.g., the global motion of the car) can create inconsistent results (and errors) in the XR experience. This can be especially frustrating in scenes where the mobile platform is moving at a high rate of speed, which can result in increasingly inconsistent or misaligned XR experiences. Accordingly, to accurately match the virtual content with the perceived motion of the user and limit or eliminate any errors and inconsistencies in the XR experience, the process 200 can track the pose of a user within the mobile platform, which can be represented by a mobile map or local motion map, and the pose of the mobile platform relative to the external scene, which can be represented by a global or world map” (Paragraph 0085-0086).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Gorur in view of Hill and Dedonato et al. (US 20210150818 A1), hereinafter Dedonato.
Regarding claim 16, Gorur in view of Hill teaches the method of claim 1. Gorur does not teach wherein the library of one or more local maps is merged with a second library of one or more maps, where the second library of one or more maps is also representative of the moveable surface. However, Dedonato teaches wherein the library of one or more local maps is merged with a second library of one or more maps, where the second library of one or more maps is also representative of the moveable surface (Paragraph 0226 – “When more than two maps, for example, three or four maps relating to the same or adjacent regions of the physical world, have been stored, the map merge algorithm 124 merges all the maps into the canonical map 120 to render a new canonical map 120”; Note: each map is equivalent to a library of a local map. Each map that represents the same region is merged with one another. The moveable surface was previously taught by Gorur in the rejection of claim 1, and in this case, corresponds to the same region shared by each map). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Dedonato to merge libraries of maps representing the same moveable surface for the benefit of simplicity; having a single map to represent the moveable surface allows for consistency and preventing redundancy. If there were multiple maps of the same surface, there would be uncertainty and more time consumption in deciding which map to use for locating, displaying, or making calculations of the surface.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Gorur in view of Hill and Velasquez et al. (US 20210256767 A1), hereinafter Velasquez.
Regarding claim 18, Gorur in view of Hill teaches the method of claim 17. Gorur does not teach wherein the method further includes uploading the tagged scanning data to a map aggregator. However, Velasquez teaches uploading the tagged scanning data to a map aggregator (Paragraph 0359, 0389, 0557-0558 – “The map merge portion may merge maps by identifying maps that depict overlapping portions of the physical world. Those overlapping portions may be aligned such that information in both maps may be aggregated into a final map. Canonical maps may merged with other canonical maps and/or tracking maps… The associated metadata may include area identities derived from the area identities of the AR devices that supplied data from which the environment maps were constructed…The map merge process 6900 begins with a map merge portion (e.g., map merge portion 810 of FIG. 26) receiving a tracking map (e.g., a new tracking map) from a portable device and one or more environment maps stored in a database in the cloud. Receiving the tracking map may include receiving data associated with the tracking map (e.g., sets of feature points and/or other information described herein)…the one or more environment maps may be selected from the database as candidates for comparison to the tracking map. This selection may be based on area identifiers, area attributes, or other metadata associated with the new tracking map”; Note: the map merge portion is equivalent to the map aggregator. The area identifiers are equivalent to the tagged scanning data, which gets uploaded to the map merge portion. The tagged scanning data was also previously taught by Gorur in the rejection of claim 17). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gorur to incorporate the teachings of Velasquez to upload the tagged scanning data to a map aggregator for the benefit of being able to accurately generate maps that only include the necessary data, as the tags assist in differentiating the data and determining whether the data should be matched with a map.
Allowable Subject Matter
Claims 9-10 are objected to as being dependent upon a rejected base claim, but would be allowable if the double patenting rejection was overcome and the claims were rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 11 is objected to as being dependent upon a rejected base claim, but would be allowable if the claims were rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 9 would be allowable for disclosing subsequent to determining that the MR system has left the moveable surface, re-evaluating whether the MR system is still on the moveable surface, wherein said re-evaluating is performed in response to a determination that the MR system is still moving with a movement of the moveable surface.
Regarding claim 9, Gorur in view of Hill teaches the method of claim 8. However, none of the prior art teaches the claim limitation of subsequent to determining that the MR system has left the moveable surface, re-evaluating whether the MR system is still on the moveable surface, wherein said re-evaluating is performed in response to a determination that the MR system is still moving with a movement of the moveable surface. Gorur teaches determining that the MR system has left the moveable surface (Paragraph 0076-0077, 0170 – “the environment classifier 114 can detect whether a user has left a first mobile map or platform (e.g., an elevator) and has entered a different mobile map or platform (e.g., a car)…The environment classifier 114 can detect a change in the applicable mobile map or platform based on one or more images received from the sensors 130. The one or more images can capture a scene or view of the mobile map or platform applicable to the user. The environment classifier 114 can process the one or more images to classify the user's current environment… FIG. 9 illustrates a diagram of a scheme 900 for detecting when a user leaves a mobile platform (and/or associated mobile map) and enters a new mobile platform (and/or associated mobile map). In this example, a user located in the elevator 802 leaves and enters the car 812. To detect that the user has entered the new mobile platform (e.g., the car 812), an image of the elevator 802 and an image of the car 812 (e.g., captured by one or more image sensors 134) can be processed”; Note: it is detected that the user enters a new mobile platform, meaning the user left the previous mobile platform (moveable surface)) and re-evaluating whether the MR system is still on the moveable surface (Paragraph 0076-0077, 0170 – “the environment classifier 114 can detect whether a user has left a first mobile map or platform (e.g., an elevator) and has entered a different mobile map or platform (e.g., a car)… The environment classifier 114 can detect a change in the applicable mobile map or platform based on one or more images received from the sensors 130. The one or more images can capture a scene or view of the mobile map or platform applicable to the user. The environment classifier 114 can process the one or more images to classify the user's current environment… FIG. 9 illustrates a diagram of a scheme 900 for detecting when a user leaves a mobile platform (and/or associated mobile map) and enters a new mobile platform (and/or associated mobile map). In this example, a user located in the elevator 802 leaves and enters the car 812. To detect that the user has entered the new mobile platform (e.g., the car 812), an image of the elevator 802 and an image of the car 812 (e.g., captured by one or more image sensors 134) can be processed”; Note: the environment classifier re-evaluates the user’s position and whether they are still on the mobile platform by detecting if they are on a new mobile platform), but it does not teach subsequent to determining that the MR system has left the moveable surface, re-evaluating whether the MR system is still on the moveable surface, wherein said re-evaluating is performed in response to a determination that the MR system is still moving with a movement of the moveable surface. Based on the configuration, it would be improper hindsight to modify Gorur to have these features. Therefore, the combination of features is considered allowable.
Claim 10 would be allowable for disclosing expanding a size of the 3D boundary based on a determination that the MR system has not left the moveable surface.
Regarding claim 10, none of the prior art teaches the method of claim 9, which claim 10 is dependent on, nor the limitation: expanding a size of the 3D boundary based on a determination that the MR system has not left the moveable surface. Therefore, the combination of features is considered allowable.
Claim 11 would be allowable for disclosing after determining that the MR system has left the moveable surface, acquiring new scanning data; and preventing the new scanning data from being included in the library of one or more local maps.
Regarding claim 11, none of the prior art teaches the method of claim 9, which claim 11 is dependent on. Therefore, the combination of features is considered allowable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Rober et al. (US 10482669 B2) teaches a VR system that displays virtual experiences to users in moving vehicles by matching the virtual experiences to the vehicle motion data. Dine et al. (US 12039674 B2) teaches an extended reality system that is useable on a moveable platform by separating motion of the platform from motion of the extended reality device.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE HAU MA whose telephone number is (571)272-2187. The examiner can normally be reached M-Th 7-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE HAU MA/ Examiner, Art Unit 2617
/KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights