DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Objections
Claims 3, 5, 11, 13, 14, and 19 are objected to because of the following informalities:
Claim 3 line 3 recites “XR environment”, which should be “XR virtual environment.”
Claim 5, line 7 recites “a gesture command or a combination thereof;”, which should be “a gesture command, or a combination thereof;”. “,” should be placed.
Claim 5, line 9 recites “a second gesture command or a combination thereof;”, which should be “a second gesture command, or a combination thereof;”. “,” should be placed.
Claim 6 line 1 recites “the system executes”, which should be “the system further executes.”
Claim 11 line 1 recites “the system comprises”, which should be “the system further comprises.”
Claim 11 line 11 recites “issuing a second voice command or a second gesture command by the second user;”, which should be “issuing a second voice command, or a second gesture command by the second user;”. “,” should be placed.
Claim 13 line 13 recites “g”, which should be “f”.
Claim 11 line 1 recites “the system comprises”, which should be “the system further comprises.”
Claim 14 line 1 recites “the headset”, which should be “the XR headset.”
Claim 19 line 1 recites “the system comprises”, which should be “the system further comprises.”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 12 and 15 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. In particular, claims 12 and 15 recite limitation “wherein the system is scalable in any instance for any amount of users located anywhere.” Original disclosure does not disclose or explain how to support the scalability to any instance neither for large number of users nor for large number of locations. Also, it had been well-known that such virtual reality environment was operated with hardware system with limited resources (e.g., server and memory), and scaling to unlimited number of virtual instances has not been known. Thus, given the lack of any further detail in the specification, the ordinary person in the art would not believe the inventor possessed the claimed limitations at the time of effective filing date.
Claim 18 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claim 18 recites limitation “other functions.” Original disclosure does not disclose or explain how to describe and support other functions in real-time monitoring of games sessions and user interactions. Thus, given the lack of any further detail in the specification, the ordinary person in the art would not believe the inventor possessed the claimed limitations at the time of effective filing date.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The following claims recite the limitations having insufficient antecedent basis:
Claim 1, line 15: "the one or more than one users' XR headset integrated display system"
Claim 1 line 12: “the one or more than one shared XR virtual environment"
Claim 1 line 17-18: “the XR virtual environment, assets, content, theme, script/narrative, and interactions in in real time”
Claim 4 line 4: “the ..one or more than one voice command, one or more than one gesture command”
Claim 5, lines 2-3: “the layout of an entire physical location"
Claim 5, line 14: “the area”
Claim 1 line 18 recites "script/narrative," which renders the claim indefinite because it is unclear whether the limitation "/" means "or" or "and."
The term “expensive” in claim 1 line 14 is a relative term which renders the claim indefinite. The term “expensive” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The word “expensive” has a plain meaning of “something that costs a lot of money” as defined by the dictionary. However, the claim is indefinite since one of ordinary skill in the art cannot translate the definition into a meaningful precise claim scope to interpret the metes and bounds of the term “expensive”. For examination purposes, the term “expensive” is being interpreted as something that costs a lot of money.
The term “spatially accurate” in claim 2 line 2 is a relative term which renders the claim indefinite. The term “spatially accurate” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The word “spatially accurate” has a plain meaning of “location corresponds to correctly to position” as defined by the dictionary. However, the claim is indefinite since one of ordinary skill in the art cannot translate the definition into a meaningful precise claim scope to interpret the metes and bounds of the term “spatially accurate”. For examination purposes, the term “spatially accurate” is being interpreted as being located correctly based on position.
The term “correctly, quickly, and accurately” in claim 4 line 1 is a relative term which renders the claim indefinite. The term “correctly, quickly, and accurately” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The word “correctly, quickly, and accurately” has a plain meaning of “action performed without errors, in a fast or highly responsive matter, and with exact precision” as defined by the dictionary. However, the claim is indefinite since one of ordinary skill in the art cannot translate the definition into a meaningful precise claim scope to interpret the metes and bounds of the term “correctly, quickly, and accurately”. For examination purposes, the term “correctly, quickly, and accurately” is being interpreted as being without error, being in a fast response matter, and being precise when executing inductions in a computer system.
Claim 9 recites “the resulting digital twin”. Claim 8 from which it provides the digital twin as an alternative limitation. Thus, “the resulting digital twin” lacks antecedent basis and unclear in its intended meaning. For examination purposes, the term “resulting digital twin is being interpreted as a “digital twin”.
Claim 10 line 2 recites “and/or”, which renders the claim indefinite because it is unclear whether the limitation "/" means "or" or "and."
Claims 12 and 15 recite “wherein the system is scalable in any instance for any amount of users located anywhere.” The ordinary meaning of “any” means unlimited, indefinite scalability. However, as being rejected in the supra 112(a) written description rejection, such indefinite scalability appears to be not supported. Thus, it is unclear what is metes and bounds of the limitation “any instance for any amount of users located anywhere.”
Claim 17 line 4 recites “, or imported from a different XR virtual environment platform, all in the original physical location,” which renders the claim indefinite because it is unclear to understand.
Regarding claim 4, the phrase "can" renders the claim indefinite because it is unclear whether the limitations following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Regarding claim 7, the phrase "can" renders the claim indefinite because it is unclear whether the limitations following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
The term “enhance precision” in claim 14 line 3 is a relative term which renders the claim indefinite. The term “enhance precision” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The word “enhance precision” has a plain meaning of “improve the exactness or details of something” as defined by the dictionary. However, the claim is indefinite since one of ordinary skill in the art cannot translate the definition into a meaningful precise claim scope to interpret the metes and bounds of the term “correctly, quickly, and accurately”. For examination purposes, the term “enhance precision” is being interpreted as to improve the exactness of location.
Regarding claim 17, the phrase "can" renders the claim indefinite because it is unclear whether the limitations following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
The term “easily and quickly” in claim 17 line 5 is a relative term which renders the claim indefinite. The term “easily and quickly” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The word “easily and quickly” has a plain meaning of “performing an action with little or no difficulty or effort and completing an action in a short amount of time” as defined by the dictionary. However, the claim is indefinite since one of ordinary skill in the art cannot translate the definition into a meaningful precise claim scope to interpret the metes and bounds of the term “easily and quickly”. For examination purposes, the term “easily and quickly” is being interpreted as being executed as expected by the computer system without any delay or effort.
Claim 18 recites “other functions.” The ordinary meaning of “other” means being additional from what has been previously mentioned. However, as being rejected in the supra 112(a) written description rejection, such being additional appears to be not supported. Thus, it is unclear what is metes and bounds of the limitation “other functions.”
Claims 2,3,6,8,11,16,19 depend from claim 1 and inherit the indefiniteness from claim 1.
Claims 1-20 will be examined as best understood by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gullicksen (US 20180061127 A1) (herein after Gullicksen) in view of Zavesky et al. (US 20210335050 A1) (hereinafter Zavesky).
Regarding Claim 1, Gullicksen discloses A computer implemented platform agnostic system for spatial synchronization of physical and virtual locations that provide user experiences to be created or where local and remote users can interact, the system comprising: (para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0013] “In examples where applications involve multiple users, the server may track locations of users in the same physical space and maintain a global perspective of all of them, such that the server maintains all user locations even when the user have no line of sight to one another.”; para. [0014] “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; FIG. 2, FIG. 3; para. [0042]-[0072], Examiner’s note: a system with a server and a headset corresponds to platform agnostic system.
a. One or more than one central server, wherein the one or more than one central server comprises one or more than one processor; (para. [0012], "The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server."; para. [0051], "FIG. 2 shows an example of the server apparatus 150 and related components in additional detail. Here, the server apparatus 150 includes an LPS (local positioning service) base station 210, a vault appliance 220, an application server 230 (such as a game hub), and a switch/router 240. In various examples, these components may be provided separately, as a single integrated unit, or as any combination of physical units."; para. [0055], “The application server 230 is a computerized device configured to run software applications, such as an application 232. The application server 230 may be implemented as a general purpose computer or as a specialized system, such as a game hub. The game hub may be similar to conventional game consoles (e.g., X-BOX, PlayStation, etc.) but is adapted to work with MR (mixed reality) games and to participate in the particular procedures and protocols as described herein. In the conventional way, the game hub may download games and other content over the Internet. It may also receive content via conventional software media, DVDs, Blu-ray disks, etc.”)
b. one or more than one XR headset operably connected to the one or more than one central server, wherein the XR headset comprises one or more than one processor; and (para. [0059], “FIG. 3 shows a schematic view of the user 120 wearing the headset 130 as well as various wearable sensors 310. The headset 130 and the wearable sensors 310 track the user's movements and enable the server apparatus 250 to construct a skeletal model of the user 120. The resulting skeletal model has two main uses. First, it enables the server apparatus 150 to track the user's body position and movements, such that the server apparatus 150 can construct a sharable model of the user that moves in synchronization with the user. Second, it enables the server apparatus 150 and/or the headset 130 to map user movements to a motion-based set of control commands. For example, the server apparatus 150 and/or headset 130 may interpret particular movements of the user's head, arms, legs, and/or torso as control commands, which are interpreted to initiate particular activities.”; para. [0007], “At the center of VR technology is the VR headset, which displays images onto opaque screens placed in front of a user's eyes, allowing the user to visualize displayed content in three dimensions. The VR headset typically includes accelerometers and other sensors, which provide input to a 3-D rendering engine, such that the user can change the displayed view of the virtual environment by making natural head movements. The VR headset typically also includes speakers and a microphone, which allow users to hear sounds in the virtual environment and to communicate with others. The user may control a VR application, such as a game, virtual walk-through, or other application using a hand-held controller. A well-known example of a VR headset is the Oculus Rift, available from Oculus VR, LLC.”)
c. instructions executable on the one or more than one central server and the one or more than one XR headset for: (para. [0055], “The application server 230 is a computerized device configured to run software applications, such as an application 232. The application server 230 may be implemented as a general purpose computer or as a specialized system, such as a game hub. The game hub may be similar to conventional game consoles (e.g., X-BOX, PlayStation, etc.) but is adapted to work with MR (mixed reality) games and to participate in the particular procedures and protocols as described herein. In the conventional way, the game hub may download games and other content over the Internet. It may also receive content via conventional software media, DVDs, Blu-ray disks, etc.”; para. [0011], "The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data."; para. [0044], “The headset 130 processes the user location and map data 160 to render the holograms. For example, the headset 130 applies hologram information and map data 160 to generate a virtual model of objects within its line of sight 132, based on the headset's current 3-D location and orientation”; para. [0058], "Although not specifically shown, one should appreciate that the LPS base station 210, the vault appliance 220, and the application server 230 each include their own processing circuitry and memory. Each memory may store instructions which, when run by the respective processing circuitry, cause the processing circuitry to carry out various procedures and activities as described herein. Unless otherwise specified, one should appreciate that any activities ascribed to the server apparatus 150 may be performed by any of the included components, or by any combination of such components.")
1) mapping one or more than one physical location into (para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0018], “In some examples, the server may generate virtual content in the form of scenes, where a “scene” includes a collection of holograms and virtual interconnects. A “hologram” is a synthesized 3-D image, and a “virtual interconnect” is an identified space in the play zone that a user may enter to perform an action. Such actions may include, for example, rotating a virtual play zone (e.g., to accommodate a physical space inconsistent with the virtual play zone), teleporting to a different location in the virtual play zone, or entering a VR play area, e.g., an entirely synthetic VR realm.”; para. [0019] “Certain embodiments are directed to a method of managing virtual content to be displayed to users via three-dimensional imaging headsets. The method includes measuring locations of a user in a physical space as the user moves through the physical space, by a server apparatus receiving inputs from multiple stationary sensors positioned at respective sensor locations within the physical space and processing the inputs to generate the locations of the user. The method further includes storing map data that describes a map of the physical space and specifying a set of holograms that have apparent locations that are defined relative to the map data. The method still further includes providing the measured locations of the user and at least a portion of the map data to a headset worn by the user, to enable the headset to render the set of holograms at the apparent locations relative to the map data and from a user perspective based on the measured locations of the user.”)
2) mapping one or more than one XR virtual environment; (para. [0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0018], “In some examples, the server may generate virtual content in the form of scenes, where a “scene” includes a collection of holograms and virtual interconnects. A “hologram” is a synthesized 3-D image, and a “virtual interconnect” is an identified space in the play zone that a user may enter to perform an action. Such actions may include, for example, rotating a virtual play zone (e.g., to accommodate a physical space inconsistent with the virtual play zone), teleporting to a different location in the virtual play zone, or entering a VR play area, e.g., an entirely synthetic VR realm.”; para. [0019], “Certain embodiments are directed to a method of managing virtual content to be displayed to users via three-dimensional imaging headsets. The method includes measuring locations of a user in a physical space as the user moves through the physical space, by a server apparatus receiving inputs from multiple stationary sensors positioned at respective sensor locations within the physical space and processing the inputs to generate the locations of the user. The method further includes storing map data that describes a map of the physical space and specifying a set of holograms that have apparent locations that are defined relative to the map data. The method still further includes providing the measured locations of the user and at least a portion of the map data to a headset worn by the user, to enable the headset to render the set of holograms at the apparent locations relative to the map data and from a user perspective based on the measured locations of the user.”)
3) interacting with the one or more than one shared XR virtual environment; (para. [0043], “The server apparatus 150 specifies holograms (virtual objects) relative to the map data 160. For example, each hologram has a location, dimensions, and an orientation that is aligned with a coordinate system of the map data 160. Example holograms 114, 116, and 118 are shown. These holograms are not physical features but rather virtual features that the headset 130 may render such that they appear to be present in the indicated locations from the perspective of the user 120 through the headset 130. The server apparatus 150 may further specify holograms as artificial coverings (e.g., skins) on physical objects present in the physical space 110, such as on walls 112a and 112b. In some examples, virtual objects are constructed around physical objects. For example, hologram 114 may be shown as a garden whereas in fact it covers a physical object, such as a sofa”; para. [0014], ”The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars”; para. [0011], "The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data.")
4) tracking one or more than one user accurately in both the physical and the one or more than one XR virtual environment without needing expensive equipment external to the one or more than one users’ XR headset integrated display system, wherein the executable instructions are platform agnostic; and (para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. ”Examiner’s note: AR or MR headset is used alone. Also, Gullicksen discloses wearable sensors (310 in FIG. 3) stationary sensors (140a-140d in FIG. 1). It’s well-known that they are generally moderately priced”; para. [0021], “Further embodiments include methods of generating skeletal models of users, methods of interpreting user movements as control commands, methods of calibrating positions and/or orientations of wearable sensors, methods of reorienting virtually-rendered spaces via virtual interconnects, methods of teleporting users between virtual spaces via virtual interconnects, and methods of entering and exiting VR realms. Still further embodiments are directed to wearable sensors, sensor platters, MR headsets, game consoles, location positioning systems, and security vaults, for implementing any of the above methods, as well as to computer-readable media from which software instructions for carrying out such methods may be installed. Some embodiments involve multiple components, which may be deployed over a computer network.”; para. [0059], “FIG. 3 shows a schematic view of the user 120 wearing the headset 130 as well as various wearable sensors 310. The headset 130 and the wearable sensors 310 track the user's movements and enable the server apparatus 250 to construct a skeletal model of the user 120. The resulting skeletal model has two main uses. First, it enables the server apparatus 150 to track the user's body position and movements, such that the server apparatus 150 can construct a sharable model of the user that moves in synchronization with the user. Second, it enables the server apparatus 150 and/or the headset 130 to map user movements to a motion-based set of control commands. For example, the server apparatus 150 and/or headset 130 may interpret particular movements of the user's head, arms, legs, and/or torso as control commands, which are interpreted to initiate particular activities”; para. [0092], “One should appreciate that other occasions may be appropriate for correcting sensor drift, as well. For example, the system may issue a command to the user to assume the neutral position at any suitable time.”; para. [0093], “Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows: [0094] Quick internal rotation of wrists; [0095] Quick external rotation of wrists;”; para. “[0115] Synthetic content may be supplied by users, by games, and/or by third-party software, for example. Sharing of content may go in any direction. For example, the second user 120-2 may design a virtual object and place that object in a particular location within the play zone, sharing that object back to other users via respective server apparatus.”; para. [0116], “In some examples, a game or other software application may generate the play zone, which need not correspond to features in any physical space. For example, the game may define the play zone and share that play zone with all users. The server apparatus at each location may instantiate the play zone locally. Users can then view features of the play zone and avatars of other users through their respective headsets. Games or other software applications may also produce computer-generated avatars, which move through the play zone and can be viewed at corresponding locations by all users.”; para. [0131], “At 1540, the measured locations of the user 120 and at least a portion of the map data 160 are provided to a headset 130 worn by the user 120, to enable the headset 130 to render the set of holograms (e.g., 114, 116, 118) at the apparent locations relative to the map data 160 and from a user perspective based on the measured locations of the user.”; FIG. 2, FIG. 3; para. [0042]-[0072]; para. [0058], “[0058] Although not specifically shown, one should appreciate that the LPS base station 210, the vault appliance 220, and the application server 230 each include their own processing circuitry and memory. Each memory may store instructions which, when run by the respective processing circuitry, cause the processing circuitry to carry out various procedures and activities as described herein. Unless otherwise specified, one should appreciate that any activities ascribed to the server apparatus 150 may be performed by any of the included components, or by any combination of such components”; Examiner’s note: a system with a server and a headset corresponds to platform agnostic system. Therefore, instruction which is executed in this system is platform agnostic )
5) controlling the XR virtual environment, assets (software application, game application), content, theme (skins, clothing), ([0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0015], “In some examples, the server may filter shared map information sent to other users to protect privacy. The server may also apply artificial content, e.g., coverings, colors, textures, etc., to provide simplified and/or stylized renderings of physical features in the play zone.”; para. [0016], “In some examples, a user can apply wearable sensors to the user's body and execute a series of predetermined movements, from which the headset and/or server may generate a 3-D skeletal model of the user. Skins and/or clothing may be applied to the skeletal model to create a 3-D avatar, which moves in synchronization with the user. Thus, as the user moves, so too does the avatar move in a like manner.”; para. [0049], “In some examples, additional map data 160 may derive from a software application. For example, a game or other application may define particular rooms, realms, spaces, and other artificial content, which the server apparatus 150 receives and may integrate into the map data 160 at suitable locations and orientations relative to the physical space 110.”; para. [0057], “Using the WAN port, the switch/router 240 may connect to one or more public servers 260. These may include on-line stores (e.g., for buying games) and various servers to support vault-based communications. The switch/router 240 also supports communication over the WAN/Internet 250 with similarly-configured networks of other users, e.g., to support multi-player games or other applications across different local networks and locations.”; para. [0114], “In some examples, the shared map data 160a represents physical objects in the first physical location 110 in simplified form, with certain details of the physical space removed to avoid distraction and/or to maintain privacy. For example, a wall with a built-in bookshelf may be represented as a flat wall, or as a wall with a stylized surface texture or synthetic content. A desk covered with papers may be represented as a flat surface.”; para. [0113], “In addition to receiving shared map data 160a, the second server apparatus 150-2 also receives, over the WAN/Internet 250, a real-time representation of the avatar 430 of the first user 120, including its location and orientation relative to the play zone. With this arrangement, the second user 120-2 is able to see the avatar 430 of the first user 120 move within the second physical space 110-2 (movement shown by arrow 1010) in a manner that mirrors the movement of the first user 120 within the first physical space 110. In a like manner, the first user 120, within the first physical space 110, sees an avatar of the second user 120-2 displayed and moving in real time in synchronization with the second user 120-2. This arrangement is extendible to any number of users, with each user's headset rendering the holograms 114, 116, and 118 and the avatars of the other users, with the avatars moving in real time to reflect movements of the users they represent. Also, each user's headset, with the possible exception of the first user's (where it may not be necessary), renders a virtual rendition of the shared portions of the physical space 110. Owing to the global perspective afforded by the stationary sensors and the computerized apparatus in each physical space, the position and orientation of each avatar relative to the play zone matches the position and orientation of the respective physical user in that user's local play zone.”)
Gullicksen does not explicitly disclose a computer implemented platform agnostic system for spatial synchronization of physical and virtual locations that provide user experiences to be created or where local and remote users can interact, the system comprising:
1) mapping one or more than one physical location into a digital twin;
4) tracking one or more than one user accurately in both the physical and the one or more than one XR virtual environment without needing expensive equipment external to the one or more than one users’ XR headset integrated display system, wherein the executable instructions are platform agnostic
5) controlling the XR virtual environment, assets, content, theme, script/narrative, and interactions in real-time.
Zavesky more explicitly teaches a computer implemented platform agnostic system for spatial synchronization of physical and virtual locations that provide user experiences to be created or where local and remote users can interact, the system comprising: (para. [0093], “In particular, a cloud networking architecture is shown that leverages cloud technologies and supports rapid innovation and scalability via a transport layer 350, a virtualized network function cloud 325 and/or one or more cloud computing environments 375. In various embodiments, this cloud networking architecture is an open architecture that leverages application programming interfaces (APIs); reduces complexity from services and operations; supports more nimble business models; and rapidly and seamlessly scales to meet evolving customer requirements including traffic growth, diversity of traffic types, and diversity of performance and reliability expectations.”; para. [0122], “For radio technologies that exploit packetized communication, server(s) 514 in mobile network platform 510 can execute numerous applications that can generate multiple disparate packetized data streams or flows, and manage (e.g., schedule, queue, format . . . ) such flows. Such application(s) can comprise add-on features to standard services (for example, provisioning, billing, customer support . . . ) provided by mobile network platform 510. Data streams (e.g., content(s) that are part of a voice call or data session) can be conveyed to PS gateway node(s) 518 for authorization/authentication and initiation of a data session, and to serving node(s) 516 for communication thereafter. In addition to application server, server(s) 514 can comprise utility server(s), a utility server can comprise a provisioning server, an operations and maintenance server, a security server that can implement at least in part a certificate authority and firewalls as well as other security mechanisms, and the like. In an aspect, security server(s) secure communication served through mobile network platform 510 to ensure network's operation and data integrity in addition to authorization and authentication procedures that CS gateway node(s) 512 and PS gateway node(s) 518 can enact. Moreover, provisioning server(s) can provision services from external network(s) like networks operated by a disparate service provider; for instance, WAN 550 or Global Positioning System (GPS) network(s) (not shown). Provisioning server(s) can also provision coverage through networks associated to mobile network platform 510 (e.g., deployed and operated by the same service provider), such as the distributed antennas networks shown in FIG. 1(s) that enhance wireless service coverage by providing more network coverage.”; para. [0035], “As seen in this FIG. 2B, system 250 can include XR Device 254 (which can be used by User 252 (e.g., a person whose home is being (or is to be) remodeled). The XR Device 254 can be, for example, an augmented reality device or a virtual reality device.”; para. [0049]Next, step 2208 comprises sending by the processing system, to a virtual reality device used by a second user in a second environment, the model, the second environment being physically remote from the first environment, the sending of the model facilitating display to the second user via the virtual reality device a depiction of the first environment as proposed by the remodeling proposal-(a computer implemented platform system that provides users experiences to be created or where local and remote users can interaction; para. [0049], “Referring now to FIG. 2E, various steps of a method 2200 according to an embodiment are shown. As seen in this FIG. 2E, step 2202 comprises storing by a processing system including a processor, in a database, a decorating style preference of a first user. Next, step 2204 comprises receiving by the processing system, from an augmented reality device used by the first user in a first environment, one or more images depicting the first environment in which remodeling is desired. Next, step 2206 comprises generating by the processing system, via a machine learning process, a model comprising a remodeling proposal for the first environment, the generating the model being based upon the decorating style preference of the first user and the generating the model being based upon the one or more images. Next, step 2208 comprises sending by the processing system, to a virtual reality device used by a second user in a second environment, the model, the second environment being physically remote from the first environment, the sending of the model facilitating display to the second user via the virtual reality device a depiction of the first environment as proposed by the remodeling proposal.”; para. [0085], “As described herein, various embodiments provide mechanisms that operate in the context of XR, digital twinning, and/or synthesis of environments with high interaction and personalization by the user.”)
1) mapping one or more than one physical location into a digital twin; (para. [0041], In one example, a model can either come from fixed CAD model or on-site SLAM (Simultaneous Localization and Mapping) for geometry scanning and object detection.”; para. [0084], “As described herein, various embodiments provide mechanisms to facilitate active remodeling based on style and design preferences (e.g., model synthesis). [0085] “As described herein, various embodiments provide mechanisms that operate in the context of XR, digital twinning, and/or synthesis of environments with high interaction and personalization by the user. [0086]As described herein, various embodiments provide mechanisms that operate in the context of XR and/or digital twinning for construction.”)
(para. [0093], “In particular, a cloud networking architecture is shown that leverages cloud technologies and supports rapid innovation and scalability via a transport layer 350, a virtualized network function cloud 325 and/or one or more cloud computing environments 375. In various embodiments, this cloud networking architecture is an open architecture that leverages application programming interfaces (APIs); reduces complexity from services and operations; supports more nimble business models; and rapidly and seamlessly scales to meet evolving customer requirements including traffic growth, diversity of traffic types, and diversity of performance and reliability expectations.”; para. [0122], “For radio technologies that exploit packetized communication, server(s) 514 in mobile network platform 510 can execute numerous applications that can generate multiple disparate packetized data streams or flows, and manage (e.g., schedule, queue, format . . . ) such flows. Such application(s) can comprise add-on features to standard services (for example, provisioning, billing, customer support . . . ) provided by mobile network platform 510. Data streams (e.g., content(s) that are part of a voice call or data session) can be conveyed to PS gateway node(s) 518 for authorization/authentication and initiation of a data session, and to serving node(s) 516 for communication thereafter. In addition to application server, server(s) 514 can comprise utility server(s), a utility server can comprise a provisioning server, an operations and maintenance server, a security server that can implement at least in part a certificate authority and firewalls as well as other security mechanisms, and the like. In an aspect, security server(s) secure communication served through mobile network platform 510 to ensure network's operation and data integrity in addition to authorization and authentication procedures that CS gateway node(s) 512 and PS gateway node(s) 518 can enact. Moreover, provisioning server(s) can provision services from external network(s) like networks operated by a disparate service provider; for instance, WAN 550 or Global Positioning System (GPS) network(s) (not shown). Provisioning server(s) can also provision coverage through networks associated to mobile network platform 510 (e.g., deployed and operated by the same service provider), such as the distributed antennas networks shown in FIG. 1(s) that enhance wireless service coverage by providing more network coverage.”; para. [0099], “Each of these devices can be implemented via computer-executable instructions that can run on one or more computers, and/or in combination with other program modules and/or as a combination of hardware and software. For example, computing environment 400 can facilitate in whole or in part image collection, machine learning, construction (e.g., remodeling) visualization, and vendor product advertising as described herein.”; para. [0103], “Computing devices typically comprise a variety of media, which can comprise computer-readable storage media and/or communications media, which two terms are used herein differently from one another as follows. Computer-readable storage media can be any available storage media that can be accessed by the computer and comprises both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable storage media can be implemented in connection with any method or technology for storage of information such as computer-readable instructions, program modules, structured data or unstructured data.”)
5) controlling the XR virtual environment(para. [0003], “Further, customers are often absent from the design process: both in the proposal of a specific style (the customer often cannot provide examples or specific instances) and in the modification of the plans proposed by a designer (the customer often cannot express simple movement of a wall (or spatial discomfort) with words alone to achieve the desired effect).”; para. [0070], “As described herein, various embodiments can provide a better user experience (e.g., with cost-controlled suggestions that are quickly available and/or with ML models trained from a wider set of styles or examples (instead of a single designer's input)). [0071] As described herein, various embodiments can provide a better user experience (e.g., with fast personalization from historical homes visited, experience with the system (and/or CAD), photo examples from user ideal homes, and/or examples from social circles (e.g., friends' homes, prior positive experiences in other locations).” para. [0081], “As described herein, various embodiments provide mechanisms that can apply all style and demographic understanding by the analysis of other personal media (e.g., social media, user-generated content, personal web logs, stories, etc.) into the generation of home preferences.”)
As both Gullicksen and Zavesky are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gullicksen to include a computer implemented platform agnostic system for spatial synchronization of physical and virtual locations that provide user experiences to be created or where local and remote users can interact, the system comprising: 1) mapping one or more than one physical location into a digital twin;4) tracking one or more than one user accurately in both the physical and the one or more than one XR virtual environment without needing expensive equipment external to the one or more than one users’ XR headset integrated display system, wherein the executable instructions are platform agnostic and 5) controlling the XR virtual environment, assets, content, theme, script/narrative, and interactions in real-time, in the context of spatial synchronization, according to the teaching of Zavesky, in order to provide a high degree of accuracy of digital representation (Abstract and para. [0007] of Zavesky).
Regarding claim 2, Gullicksen discloses The system of claim 1, further comprising one or more than one XR hand controller and a real-time, spatially accurate, multi-user voice communications system operably connected to the one or more than one XR headset. (para. [0013], “[0013] In examples where applications involve multiple users, the server may track locations of users in the same physical space and maintain a global perspective of all of them, such that the server maintains all user locations even when the user have no line of sight to one another.”; para. [0007], “VR (virtual reality) enables users to enter immersive, virtual environments without leaving the comfort and safety of their physical environments. Users of VR technology can move through virtual spaces and interact with rendered content, such as 3-D (three dimensional) representations of objects, scenery, and other users. At the center of VR technology is the VR headset, which displays images onto opaque screens placed in front of a user's eyes, allowing the user to visualize displayed content in three dimensions. The VR headset typically includes accelerometers and other sensors, which provide input to a 3-D rendering engine, such that the user can change the displayed view of the virtual environment by making natural head movements. The VR headset typically also includes speakers and a microphone, which allow users to hear sounds in the virtual environment and to communicate with others. The user may control a VR application, such as a game, virtual walk-through, or other application using a hand-held controller. A well-known example of a VR headset is the Oculus Rift, available from Oculus VR, LLC.”; [0067] User bends left elbow and, with left hand extending straight from forearm, touches right elbow crease with middle finger. User repeats on other side. These actions provide locations of the elbows.” para. [0081] At 520, an initial 3-D location of each of the wearable sensors 310 is established relative to the headset 130. For example, the user 120 may place each wearable sensor 310 adjacent to an object having known dimensions. The user 120 may then point the line of sight 132 of the headset 130 toward the object and direct the headset 130 to compute a precise location in space of the wearable sensor 310 relative to the headset's own location. This location is also relative to the physical space 110, as it is based on the headset's location as measured by the accurate antenna/LPS system. The headset 130 may simultaneously query the wearable sensor 310 to obtain its self-reported location as indicated by the measurement circuitry 314 within the wearable sensor. This self-reported location may be used as a baseline going forward. For example, the headset 130 may compute a 3-D correction vector based on a difference between the initial location of the wearable sensor 310 as reported by the headset 130 and the self-reported location as reported by the wearable sensor 310.”; para. [0082], “At 530, the user 120 may place the wearable sensor 310 on the body, such as on the wrist. As the user moves, the headset 130 may again query the wearable sensor 310 for its self-reported location, correcting the measured values by applying the 3-D vector. Similar activities may be performed for other wearable sensors.”; para. [0106], “At 960 of FIG. 9B, the headset 130 prepares to render the hologram 930, i.e., on the display elements in the headset 130. Owing to the close proximity of the wrist sensor 310a to the user's hand, the headset 130 identifies the arm 910 within the field of view and proceeds to identify pixel locations corresponding to parts of the arm 910 as foreground. The headset 130 also identifies as foreground pixel locations of any object connected to the arm 910, such as the controller 920.” Examiner’s note: Maintaining multiple users’ locations and communication with speakers and a microphone corresponds to a real-time, spatially accurate, multiple user voice communications system.)
Regarding claim 3, Gullicksen discloses The system of claim 2, wherein the one or more than one user is co-located with other users in a physical location and with other non-collocated users virtually join each other in the physical location to have a common experience in the XR environment where the users can interact with each other. (para. [0013], “In examples where applications involve multiple users, the server may track locations of users in the same physical space and maintain a global perspective of all of them, such that the server maintains all user locations even when the user have no line of sight to one another.”; para. [0014] “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0109], “Attention to this point has been focused on activities involving a single user 120. However, many applications, such as games, may involve multiple users. For example, multiple users may occupy the physical space 110 shown in FIG. 1, with each user's location tracked by the sensors 140 and server apparatus 150, i.e., in the same manner as described for the user 120. Each user may create a skeletal model 420, which may be rendered as an avatar 430 (FIG. 4). In some examples, each user in the physical space 110 sees all the other users as avatars, which may be superimposed over the user's physical bodies.”; para. [0115], “Synthetic content may be supplied by users, by games, and/or by third-party software, for example. Sharing of content may go in any direction. For example, the second user 120-2 may design a virtual object and place that object in a particular location within the play zone, sharing that object back to other users via respective server apparatus.”; para. [0126], “FIG. 14 shows an example spot 1410 that users may step into to enter a VR realm. A game or other software application may place such “VR Anywhere” spots 1410 at desired locations within a play zone. Once inside a VR Anywhere spot 1410, the user 120 is able perform VR commands to navigate a synthetic VR space, e.g., using the above-described physical movements mapped to control commands. The VR space is not required to have any relationship to the physical space in which the user is located. The VR Anywhere spot 1410 is preferably a small area, e.g., just large enough to safely accommodate the user 120 executing the predefined movements. In some examples, VR control commands are enabled only when the user 120 remains within the VR Anywhere spot 1410 and are disabled if the user steps out of the VR Anywhere spot 1410. In some examples, the system checks local map data 160 and locations of other users to ensure that no mapped objects are within some predetermined radius of the VR Anywhere spot 1410, to protect the user 120 (and other users) from harm that may result from sudden movements of the user's head, arms, legs, or torso.”
Regarding claim 4, Gullicksen discloses The system of claim 2, wherein the user can correctly, quickly, and accurately position virtual bounding walls and elements of a physical location, creating ([0041], “FIG. 1 shows an example environment 100 in which embodiments of the improved technique hereof can be practiced. Here, a user 120 is seen moving through a physical space 110. The physical space 110 includes rooms 110 a and 110 b and walls 112 a and 112 b. The user 120 wears a three-dimensional imaging headset 130, such as an AR (augmented reality) or MR (mixed reality) headset, which is capable of displaying holograms via display elements positioned in front of the user's eyes. The display elements in the headset 130 are transparent or translucent, such that the user 120 is able to look directly through the display elements to see the immediate surroundings. In the usual way, the headset 130 may project holograms, via the display elements, that appear fixed in relation to physical features in the physical space 110, such as floors, ceilings, walls, and other physical objects.”; para. [0048], “The map data 160 may incorporate information from a variety of sources. For example, the user may direct the server apparatus 150 to enter a learning mode, whereupon the user proceeds to walk around a perimeter of the physical space 110 while wearing the headset 130 (or carrying some other wireless device detectable by the sensors 140). As the user does so, the server apparatus 150 receives inputs from the sensors 140, generates user locations therefrom, and follows the user's movements to define the perimeter of the space. In some examples, input from one or more cameras in the headset 130 augments the location information to create a more precise map of the physical space 110, e.g., the precise locations of walls, floors, ceilings, and the like, as imaged by the camera(s) and aligned with the measured locations. One should appreciate that the physical space 110 may span multiple rooms, stories, and even outdoor spaces, provided such spaces are within range of the sensors 140.”; para. [0082], “At 530, the user 120 may place the wearable sensor 310 on the body, such as on the wrist. As the user moves, the headset 130 may again query the wearable sensor 310 for its self-reported location, correcting the measured values by applying the 3-D vector. Similar activities may be performed for other wearable sensors.”; para. [0093] “Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows: [0094] Quick internal rotation of wrists; [0095] Quick external rotation of wrists; [0096] Raising elbows and bringing forearms to vertical position; [0097] Moving both arms forward (or back); [0098] Pushing one wrist forward while pulling the other back. [0099] Simulating turning a steering wheel (each direction); [0100] Shift weight onto one leg (each leg); [0101] Shift weight forward (or back); [0102] Rotating torso right (or left); [0103] Bending forward (or leaning back).”; para. “[0123] Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”; para. [0124], “FIG. 13 shows an example of a different play zone, which is laid out as a simple maze 1310. Here, the walls of the maze are set up as barriers, such that users must stay within the designated paths of the maze to avoid becoming separated from their avatars. FIG. 13 also shows various uses of virtual interconnects. For example, the system provides virtual interconnects 118a and 118b at maze decision points. A user 120 entering these virtual interconnects 118a and 118b may rotate the perspective, as described in connection with FIGS. 11A and 11B. The user may also specify a rate of movement. For example, the user may specify accelerated movement, such that each meter the user walks in physical space is translated to N meters of movement in virtual space—through the maze 1310.”; para. [0125], “In an example, virtual interconnects 118c and 118d specify a 1:1 scale movement, such that every meter the user walks in physical space is mapped 1:1 to a meter in virtual space. Virtual interconnects 118e and 118f may require the user 120 to enter a VR realm to cross the indicated distance. The VR realm may present particular challenges that the user must overcome to cross the indicated distance.” Examiner’s note: the user locates the virtual wall by precisely, correctly, and quickly sensing the locations along a virtual boundary defining a stored space map related to the physical parameter (lengths and heights) using wearable hand controller and gesture commands.)
Thus, Gullicksen teaches creating accurate lengths and heights of the physical location using the one or more than one XR hand controller, one or more than one voice command, one or more than one gesture command, or a combination thereof.
Gullicksen fails to explicitly disclose the system of claim 2 wherein creating a digital twin with accurate lengths and heights of the physical location.
Zavesky further teaches (para. [0084], “As described herein, various embodiments provide mechanisms to facilitate active remodeling based on style and design preferences (e.g., model synthesis). [0085] “As described herein, various embodiments provide mechanisms that operate in the context of XR, digital twinning, and/or synthesis of environments with high interaction and personalization by the user. [0086]As described herein, various embodiments provide mechanisms that operate in the context of XR and/or digital twinning for construction.” Examiner’s note: It’s well-known that digital twins in Virtual Reality are dimensionally accurate in height, length and volume.)
As both Gullicksen and Zavesky are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gullicksen to include wherein the user can correctly, quickly, and accurately position virtual bounding walls and elements of a physical location, creating a digital twin with accurate lengths and heights of the physical location using the one or more than one XR hand controller, one or more than one voice command, one or more than one gesture command, or a combination thereof, in the context of spatial synchronization, according to the teaching of Zavesky, in order to provide a high degree of accuracy of digital representation (Abstract and para. [0007] of Zavesky).
Regarding claim 5, Gullicksen discloses The system of claim 4, wherein the system comprises instructions for one or more than one user to iteratively plot a plurality of reference points for the layout of an entire physical location to create an aligned (para. [0042], “Positioned at multiple locations in the physical space 110 are respective stationary sensors 140 (e.g., 140a-140d). Each of the sensors 140 is coupled to a server apparatus 150 using wired and/or wireless connections. The sensors 140 are configured to detect the presence of the user 120 and to provide detection information as inputs to the server apparatus 150. The server apparatus 150 is configured to process the inputs from the sensors 140 and to measure therefrom the 3-D locations of the user 120 in the physical space 110. The server apparatus 150 is further configured to store map data 160, which describes physical features of the physical space 110, such as locations, sizes, and orientations of walls, floors, ceilings, furniture, stairways, doors, windows, and the like.”; para. [0043], “The server apparatus 150 specifies holograms (virtual objects) relative to the map data 160. For example, each hologram has a location, dimensions, and an orientation that is aligned with a coordinate system of the map data 160. Example holograms 114, 116, and 118 are shown. These holograms are not physical features but rather virtual features that the headset 130 may render such that they appear to be present in the indicated locations from the perspective of the user 120 through the headset 130. The server apparatus 150 may further specify holograms as artificial coverings (e.g., skins) on physical objects present in the physical space 110, such as on walls 112a and 112b. In some examples, virtual objects are constructed around physical objects. For example, hologram 114 may be shown as a garden whereas in fact it covers a physical object, such as a sofa.”; para. [0048], “The map data 160 may incorporate information from a variety of sources. For example, the user may direct the server apparatus 150 to enter a learning mode, whereupon the user proceeds to walk around a perimeter of the physical space 110 while wearing the headset 130 (or carrying some other wireless device detectable by the sensors 140). As the user does so, the server apparatus 150 receives inputs from the sensors 140, generates user locations therefrom, and follows the user's movements to define the perimeter of the space. In some examples, input from one or more cameras in the headset 130 augments the location information to create a more precise map of the physical space 110, e.g., the precise locations of walls, floors, ceilings, and the like, as imaged by the camera(s) and aligned with the measured locations. One should appreciate that the physical space 110 may span multiple rooms, stories, and even outdoor spaces, provided such spaces are within range of the sensors 140.”)
a. identifying a first point by touching the one or more than one XR hand controller or the user’s hand at a first point and pressing a first XR hand controller button, issuing a voice command, a gesture command or a combination thereof; (para. [0093] Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows: [0094] Quick internal rotation of wrists; [0095] Quick external rotation of wrists; [0096] Raising elbows and bringing forearms to vertical position; [0097] Moving both arms forward (or back); [0098] Pushing one wrist forward while pulling the other back. [0099] Simulating turning a steering wheel (each direction); [0100] Shift weight onto one leg (each leg); [0101] Shift weight forward (or back); [0102] Rotating torso right (or left); [0103] Bending forward (or leaning back).)
b. identifying a second point by moving to a second point and pressing a second XR hand controller button, issuing a second voice command, a second gesture command or combination thereof; (para.[0090] At 730, a 3-D bounding region is generated for each of the second measurements. Each 3-D bounding region represents a volume in space that is predicted to contain the respective wearable sensor 310. This bounding region may be regarded as an error band around the second measurements, which may be made, for example, using 2.4 GHz (low power) Wi-Fi.; para. [0093] Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows: [0094] Quick internal rotation of wrists; [0095] Quick external rotation of wrists; [0096] Raising elbows and bringing forearms to vertical position; [0097] Moving both arms forward (or back); [0098] Pushing one wrist forward while pulling the other back. [0099] Simulating turning a steering wheel (each direction); [0100] Shift weight onto one leg (each leg); [0101] Shift weight forward (or back); [0102] Rotating torso right (or left); [0103] Bending forward (or leaning back).; para. [0085] With the wearable sensors 310 inserted into the sensor platter 600, the user 120 may calibrate the wearable sensors 310 by wearing the headset 130 and pointing the line of sight 132 toward the visual indication marks 640. As the locations of the wearable sensors 310 relative to the sensor platter 600 are known and the visual indication marks 640 have a known size and direction (as indicated by the arrow), the headset 130 can compute the location of each wearable sensor 310 relative to the headset 130, and can thus establish the initial 3-D locations of all of the wearable sensors 310 at once.;
c. calibrating the alignment and rotation of the first point and the second point; (para. [0021] “Further embodiments include methods of generating skeletal models of users, methods of interpreting user movements as control commands, methods of calibrating positions and/or orientations of wearable sensors, methods of reorienting virtually-rendered spaces via virtual interconnects, methods of teleporting users between virtual spaces via virtual interconnects, and methods of entering and exiting VR realms. Still further embodiments are directed to wearable sensors, sensor platters, MR headsets, game consoles, location positioning systems, and security vaults, for implementing any of the above methods, as well as to computer-readable media from which software instructions for carrying out such methods may be installed. Some embodiments involve multiple components, which may be deployed over a computer network.”; para. [0085], “[0085] With the wearable sensors 310 inserted into the sensor platter 600, the user 120 may calibrate the wearable sensors 310 by wearing the headset 130 and pointing the line of sight 132 toward the visual indication marks 640. As the locations of the wearable sensors 310 relative to the sensor platter 600 are known and the visual indication marks 640 have a known size and direction (as indicated by the arrow), the headset 130 can compute the location of each wearable sensor 310 relative to the headset 130, and can thus establish the initial 3-D locations of all of the wearable sensors 310 at once.”; para. [0090], “At 730, a 3-D bounding region is generated for each of the second measurements. Each 3-D bounding region represents a volume in space that is predicted to contain the respective wearable sensor 310. This bounding region may be regarded as an error band around the second measurements, which may be made, for example, using 2.4 GHz (low power) Wi-Fi.”)
d. generating an XR virtual environment defined by the first point and the second point; (para. [0021], “[0021] Further embodiments include methods of generating skeletal models of users, methods of interpreting user movements as control commands, methods of calibrating positions and/or orientations of wearable sensors, methods of reorienting virtually-rendered spaces via virtual interconnects, methods of teleporting users between virtual spaces via virtual interconnects, and methods of entering and exiting VR realms. Still further embodiments are directed to wearable sensors, sensor platters, MR headsets, game consoles, location positioning systems, and security vaults, for implementing any of the above methods, as well as to computer-readable media from which software instructions for carrying out such methods may be installed. Some embodiments involve multiple components, which may be deployed over a computer network.”; para. [0048]. “[0048] The map data 160 may incorporate information from a variety of sources. For example, the user may direct the server apparatus 150 to enter a learning mode, whereupon the user proceeds to walk around a perimeter of the physical space 110 while wearing the headset 130 (or carrying some other wireless device detectable by the sensors 140). As the user does so, the server apparatus 150 receives inputs from the sensors 140, generates user locations therefrom, and follows the user's movements to define the perimeter of the space. In some examples, input from one or more cameras in the headset 130 augments the location information to create a more precise map of the physical space 110, e.g., the precise locations of walls, floors, ceilings, and the like, as imaged by the camera(s) and aligned with the measured locations. One should appreciate that the physical space 110 may span multiple rooms, stories, and even outdoor spaces, provided such spaces are within range of the sensors 140.”; para. [0123] Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts. [0124] FIG. 13 shows an example of a different play zone, which is laid out as a simple maze 1310. Here, the walls of the maze are set up as barriers, such that users must stay within the designated paths of the maze to avoid becoming separated from their avatars. FIG. 13 also shows various uses of virtual interconnects. For example, the system provides virtual interconnects 118a and 118b at maze decision points. A user 120 entering these virtual interconnects 118a and 118b may rotate the perspective, as described in connection with FIGS. 11A and 11B. The user may also specify a rate of movement. For example, the user may specify accelerated movement, such that each meter the user walks in physical space is translated to N meters of movement in virtual space—through the maze 1310.
[0125] In an example, virtual interconnects 118c and 118d specify a 1:1 scale movement, such that every meter the user walks in physical space is mapped 1:1 to a meter in virtual space. Virtual interconnects 118e and 118f may require the user 120 to enter a VR realm to cross the indicated distance. The VR realm may present particular challenges that the user must overcome to cross the indicated distance.
e. repeating steps a-b in the physical location until the area is completely identified; and (para. [0090], “ At 730, a 3-D bounding region is generated for each of the second measurements. Each 3-D bounding region represents a volume in space that is predicted to contain the respective wearable sensor 310. This bounding region may be regarded as an error band around the second measurements, which may be made, for example, using 2.4 GHz (low power) Wi-Fi.”; para. [0086] “It is expected that the measurement circuitry 314 within the wearable sensors 310 may drift over time. Such drift may degrade proper rendering of avatar movements and may cause errors to arise in interpreting control commands. [0087] FIG. 7 shows an example method 700 for detecting the need to retrain wearable sensors 310 and for initiating such retraining. [0088] At 710, first measurements are made of wearable sensor locations by querying the measurement circuitry 314 in the wearable sensors 310 and applying correction vectors, e.g., as described in connection with the method 500 above. [0091] At 740, the system determines whether the first measurement for any wearable sensor 310 falls outside the respective 3-D bounding region for that wearable sensor. If so, the system initiates a wearable-sensor-retraining operation at the first opportunity. For example, the system may wait until the user 120 enters a designated region, such as the region indicated by object 118 in FIG. 1. The user 120 may previously have been instructed to assume a neutral body position when standing within that region, such as standing up straight with arms at the sides. For example, assuming this position may be pre-arranged as port of normal game play. Upon detecting that the user has entered the region indicated by object 118, the system measures the locations of the wearable sensors 310 using the measurement circuitry 314 (as done in the first measurements, above), and compares the sensor locations with expected locations based on the skeletal model 420. The system then generates a new correction vector for each wearable sensor 310 and applies the new correction vectors going forward.”)
f. fully mapping (para. [0064] To generate a skeletal model 420, the system (headset 130 and server apparatus 150) assumes that the user's shoulder, hip, neck, and waist joints are ball-and-socket joints and that knee and elbow joints are hinges. The system guides the user through the training procedure 412. In an example, the headset 130 projects a mirror image of the skeletal model 420 to the user as the user performs the procedure. Auditory controls may guide the user through the procedure, which may include the following steps; para. [0066] User turns head to the left, to the right, touches chin to chest, and tilts head upward. These motions provide an estimate of the neck location (assumed to be a ball joint). [0067] User bends left elbow and, with left hand extending straight from forearm, touches right elbow crease with middle finger. User repeats on other side. These actions provide locations of the elbows. [0068] User extends both arms straight in front at shoulder height. This movement provides another estimate of the shoulder joint location. This time, shoulder height and width are estimated from the wrist sensors. [0069] User rests arms to the side and then bends elbows at 90 degrees. The user rotates palms to face each other, then palms up, and finally palms down. These movements verify the range of motion of the wrist joint and the effect of turning on forearm estimation. [0070] The user locks both legs straight. The user takes one large side step with a single foot directly to the side, then relocks legs straight. User then brings feet back to neutral underneath the hips. These motions provide an estimate of the leg length (hip height) and hip width. [0071] User bends knees slightly and touches the wrist bands to the outside of the knee joint. This gives an estimate of knee height.; para. [0088] At 710, first measurements are made of wearable sensor locations by querying the measurement circuitry 314 in the wearable sensors 310 and applying correction vectors, e.g., as described in connection with the method 500 above. [0089] At 720, second measurements are made of wearable sensor locations. In this case, however, the measurements are made more directly using the antennas (140) and LPS base station 210, i.e., the same way the location of the headset 130 is measured. [0090] At 730, a 3-D bounding region is generated for each of the second measurements. Each 3-D bounding region represents a volume in space that is predicted to contain the respective wearable sensor 310. This bounding region may be regarded as an error band around the second measurements, which may be made, for example, using 2.4 GHz (low power) Wi-Fi.; para. [0093] Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows: [0094] Quick internal rotation of wrists; [0095] Quick external rotation of wrists; [0096] Raising elbows and bringing forearms to vertical position; [0097] Moving both arms forward (or back); [0098] Pushing one wrist forward while pulling the other back. [0099] Simulating turning a steering wheel (each direction); [0100] Shift weight onto one leg (each leg); [0101] Shift weight forward (or back); [0102] Rotating torso right (or left); [0103] Bending forward (or leaning back).; para. [0085] With the wearable sensors 310 inserted into the sensor platter 600, the user 120 may calibrate the wearable sensors 310 by wearing the headset 130 and pointing the line of sight 132 toward the visual indication marks 640. As the locations of the wearable sensors 310 relative to the sensor platter 600 are known and the visual indication marks 640 have a known size and direction (as indicated by the arrow), the headset 130 can compute the location of each wearable sensor 310 relative to the headset 130, and can thus establish the initial 3-D locations of all of the wearable sensors 310 at once.; para. [0086] It is expected that the measurement circuitry 314 within the wearable sensors 310 may drift over time. Such drift may degrade proper rendering of avatar movements and may cause errors to arise in interpreting control commands.[0087] FIG. 7 shows an example method 700 for detecting the need to retrain wearable sensors 310 and for initiating such retraining.[0088] At 710, first measurements are made of wearable sensor locations by querying the measurement circuitry 314 in the wearable sensors 310 and applying correction vectors, e.g., as described in connection with the method 500 above.”; para. [0042] Positioned at multiple locations in the physical space 110 are respective stationary sensors 140 (e.g., 140a-140d). Each of the sensors 140 is coupled to a server apparatus 150 using wired and/or wireless connections. The sensors 140 are configured to detect the presence of the user 120 and to provide detection information as inputs to the server apparatus 150. The server apparatus 150 is configured to process the inputs from the sensors 140 and to measure therefrom the 3-D locations of the user 120 in the physical space 110. The server apparatus 150 is further configured to store map data 160, which describes physical features of the physical space 110, such as locations, sizes, and orientations of walls, floors, ceilings, furniture, stairways, doors, windows, and the like.”)
Gullicksen fails to explicitly disclose
Zavesky further teaches wherein the system comprises instructions for one or more than one user(para. [0084], “As described herein, various embodiments provide mechanisms to facilitate active remodeling based on style and design preferences (e.g., model synthesis). [0085] “As described herein, various embodiments provide mechanisms that operate in the context of XR, digital twinning, and/or synthesis of environments with high interaction and personalization by the user. [0086]As described herein, various embodiments provide mechanisms that operate in the context of XR and/or digital twinning for construction.”)
As both Gullicksen and Zavesky are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gullicksen to include teaches wherein the system comprises instructions for one or more than one user to iteratively plot a plurality of reference points for the layout of an entire physical location to create an aligned digital twin of the physical location and any additional virtual features that are not present in the physical location, the instructions comprising: and fully mapping a digital twin of the physical location in the XR virtual environment by pressing a third XR hand controller button, issuing a third voice command, a third gesture command, or a combination thereof, to merge the points and any additional virtual penetrations, extrusions, or other virtual features, in the context of spatial synchronization, according to the teaching of Zavesky, in order to provide a high degree of accuracy of digital representation (Abstract and para. [0007] of Zavesky).
Regarding claim 6, Gullicksen discloses The system of claim 2, wherein the system comprises a library of objects and assets that can be quickly added to the digital twin or shared XR virtual environment. (para. [0008], “AR (augmented reality) allows users to continue to see their physical environments through display screens while additional content is superimposed. AR thus provides a vehicle for adding synthetic content to users' normal views of their environments. An example of AR technology is Google Glass.”; para. [0049], “In some examples, additional map data 160 may derive from a software application. For example, a game or other application may define particular rooms, realms, spaces, and other artificial content, which the server apparatus 150 receives and may integrate into the map data 160 at suitable locations and orientations relative to the physical space 110.”; para.[0057], “Using the WAN port, the switch/router 240 may connect to one or more public servers 260. These may include on-line stores (e.g., for buying games) and various servers to support vault-based communications. The switch/router 240 also supports communication over the WAN/Internet 250 with similarly-configured networks of other users, e.g., to support multi-player games or other applications across different local networks and locations.”; para. [0050] In some examples, the user operates a software program to define custom features, such as coverings, objects, walls, and the like, which the server apparatus 150 may locate relative to the map data 160 at user-defined locations. Also, as will be described in more detail below, input from other physical spaces may also be received and located relative to the map data 160, e.g., to enable the user 120 to visualize features in a distinct physical spaces in a manner that is aligned with the user's own physical space 110.”)
Regarding claim 7, Gullicksen discloses The system of claim 2, wherein the digital twin or shared XR virtual environment can be configured to support various gaming and other fantasy or real location layouts. (para. [0007], “The user may control a VR application, such as a game, virtual walk-through, or other application using a hand-held controller. A well-known example of a VR headset is the Oculus Rift, available from Oculus VR, LLC.”; para. [0014] “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0109], “Attention to this point has been focused on activities involving a single user 120. However, many applications, such as games, may involve multiple users. For example, multiple users may occupy the physical space 110 shown in FIG. 1, with each user's location tracked by the sensors 140 and server apparatus 150, i.e., in the same manner as described for the user 120. Each user may create a skeletal model 420, which may be rendered as an avatar 430 (FIG. 4). In some examples, each user in the physical space 110 sees all the other users as avatars, which may be superimposed over the user's physical bodies.”
Regarding claim 8, Gullicksen discloses The system of claim 2, wherein the system further comprises instructions to import or access saved, persistent, user-created content, and third-party content into the digital twin or XR virtual environment. (para. [0020], “The computer program product stores instructions which, when executed on control circuitry of a data storage system, cause the data storage system to perform a method of managing virtual content to be displayed to users via headsets, such as the method described above.“; para. [0053], “The vault appliance 220 is a secure hub for storing and dispatching rights. Such rights may include content rights for accessing particular content, communication rights for establishing communications with another party, and action rights for performing actions on particular devices or elements. For example, the vault appliance 220 may securely store the map data 160, or portions thereof, and may securely control the release of such map data 160 using content rights and/or communication rights. Further details of the vault appliance 220 may be found in U.S. patent application Ser. No. 15/347,551, filed Nov. 9, 2016 and entitled “VAULT APPLIANCE FOR IDENTITY VERIFICATION AND SECURE DISPATCH OF RIGHTS,” the contents and teachings of which are incorporated by reference herein.”; para. [0063], “As shown in FIG. 4, the training procedure 412 measures 3-D positions 410 of the wearable sensors 310 as the user 120 executes a set of predefined movements. The result of the training procedure 412 is a 3-D skeletal model 420 of the user, which estimates the user's joint locations and limb lengths. Based on the skeletal model 420, a rendering step 422 may generate a 3-D avatar 430 of the user 120. In some examples, the application server 230 (e.g., a game hub) directs the training procedure 412 and stores the skeletal model 420 of the user 120. However, generation of the avatar 430 from the skeletal model 420 may be initiated by particular applications. For example, users may represent themselves with different avatars 430 in different games, but with all such avatars 430 based on the same body geometries. The application server 230 and particular applications may support multiple users, with each user having his or her own skeletal model and avatar.”; para. [0115] “Synthetic content may be supplied by users, by games, and/or by third-party software, for example. Sharing of content may go in any direction. For example, the second user 120-2 may design a virtual object and place that object in a particular location within the play zone, sharing that object back to other users via respective server apparatus.”
Regarding claim 9, Gullicksen discloses The system of Claim 8, wherein the system further comprises instructions to add virtual elements to the existing XR virtual environment and modify or remove elements of the XR virtual environment, using mapping methods and save the resulting digital twin. (para. [0020], “The computer program product stores instructions which, when executed on control circuitry of a data storage system, cause the data storage system to perform a method of managing virtual content to be displayed to users via headsets, such as the method described above.“; para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0111], “In the example shown, the second server apparatus 150-2 has obtained shared map data 160a from the first server apparatus 150 (FIG. 1), e.g., over the WAN/Internet 250. The shared map data 160a includes portions of the map data 160. Sharing may be accomplished, for example, by the owner of the vault appliance 230 in the first physical space 110 granting a content right to the second user 120-2 to access particular features in the physical space 110. Here, map data describing features of the first room 110a are shared, such as walls 112a, whereas map data describing other portions of the physical space 110 are not shared, such as data describing the room 110b and walls 112b (e.g., the first user 120 may wish to keep the room 110b private). In general, map data 160 may be shared at any desired level of granularity. By obtaining the shared map data 160a, a second headset 130-2 worn by the second user 120-2 is able to render objects that are physical in the first physical space 120 as virtual objects in the second physical space 110-2. Thus, the second user 120-2 is able to visualize the room 110a and walls 112a as holograms 110a-V and 112a-V, respectively.”; para. [0115], “Synthetic content may be supplied by users, by games, and/or by third-party software, for example. Sharing of content may go in any direction. For example, the second user 120-2 may design a virtual object and place that object in a particular location within the play zone, sharing that object back to other users via respective server apparatus.”; para. [0121] “As shown in FIG. 12A, the second user 120-2 in the second physical space 110-2 has left the play zone. For example, the second user 120-2 has crossed from a first side 1210 of the virtual wall 112a-V to a second side 1220. Here, when the second user 120-2 leaves the play zone, the avatar 430-2 of the second user 120-2 stays behind, i.e., on the first side 1210 of the virtual wall 112a-V. The avatar 430-2 may remain in this position until the second user 120-2 returns to the same location, at which point the avatar 430-2 resumes tracking the movements of the second user 120-2.”; para. [0122], “FIG. 12B shows the same situation from the viewpoint of the first user 120 in the first physical space 110. The first user 120 sees, through the headset 130, the avatar 430-2 of the second user 120-2, but the avatar 430-2 appears to be standing still—until the second user returns such that tracking can resume.”; para. [0123], “Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”
Gullicksen fails to explicitly disclose
Zavesky further teaches (para. [0084], “As described herein, various embodiments provide mechanisms to facilitate active remodeling based on style and design preferences (e.g., model synthesis). [0085] “As described herein, various embodiments provide mechanisms that operate in the context of XR, digital twinning, and/or synthesis of environments with high interaction and personalization by the user. [0086]As described herein, various embodiments provide mechanisms that operate in the context of XR and/or digital twinning for construction.”)
As both Gullicksen and Zavesky are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gullicksen to include wherein the system further comprises instructions to add virtual elements to the existing XR virtual environment and modify or remove elements of the XR virtual environment, using mapping methods and save the resulting digital twin, in the context of spatial synchronization, according to the teaching of Zavesky, in order to provide a high degree of accuracy of digital representation (Abstract and para. [0007] of Zavesky).
Regarding claim 10, Gullicksen discloses The system of claim 2, wherein the system further comprises instructions to scale, rotate, translate, tilt, and/or orient the digital twin or shared XR virtual environment, of an inside physical location or of an outside physical object, and outside physical location or another 3D model, to whatever size, orientation, and position selected by the user in the XR virtual environment. (para. [0020], “The computer program product stores instructions which, when executed on control circuitry of a data storage system, cause the data storage system to perform a method of managing virtual content to be displayed to users via headsets, such as the method described above.“; para. [0012] “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0041], “FIG. 1 shows an example environment 100 in which embodiments of the improved technique hereof can be practiced. Here, a user 120 is seen moving through a physical space 110. The physical space 110 includes rooms 110 a and 110 b and walls 112 a and 112 b. The user 120 wears a three-dimensional imaging headset 130, such as an AR (augmented reality) or MR (mixed reality) headset, which is capable of displaying holograms via display elements positioned in front of the user's eyes. The display elements in the headset 130 are transparent or translucent, such that the user 120 is able to look directly through the display elements to see the immediate surroundings. In the usual way, the headset 130 may project holograms, via the display elements, that appear fixed in relation to physical features in the physical space 110, such as floors, ceilings, walls, and other physical objects.”; [0048], “The map data 160 may incorporate information from a variety of sources. For example, the user may direct the server apparatus 150 to enter a learning mode, whereupon the user proceeds to walk around a perimeter of the physical space 110 while wearing the headset 130 (or carrying some other wireless device detectable by the sensors 140). As the user does so, the server apparatus 150 receives inputs from the sensors 140, generates user locations therefrom, and follows the user's movements to define the perimeter of the space. In some examples, input from one or more cameras in the headset 130 augments the location information to create a more precise map of the physical space 110, e.g., the precise locations of walls, floors, ceilings, and the like, as imaged by the camera(s) and aligned with the measured locations. One should appreciate that the physical space 110 may span multiple rooms, stories, and even outdoor spaces, provided such spaces are within range of the sensors 140.”; para. [0049] “In some examples, additional map data 160 may derive from a software application. For example, a game or other application may define particular rooms, realms, spaces, and other artificial content, which the server apparatus 150 receives and may integrate into the map data 160 at suitable locations and orientations relative to the physical space 110.”; para. [0050], “In some examples, the user operates a software program to define custom features, such as coverings, objects, walls, and the like, which the server apparatus 150 may locate relative to the map data 160 at user-defined locations. Also, as will be described in more detail below, input from other physical spaces may also be received and located relative to the map data 160, e.g., to enable the user 120 to visualize features in a distinct physical spaces in a manner that is aligned with the user's own physical space 110.”; para. [0123], “Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”; para. [0124], “FIG. 13 shows an example of a different play zone, which is laid out as a simple maze 1310. Here, the walls of the maze are set up as barriers, such that users must stay within the designated paths of the maze to avoid becoming separated from their avatars. FIG. 13 also shows various uses of virtual interconnects. For example, the system provides virtual interconnects 118a and 118b at maze decision points. A user 120 entering these virtual interconnects 118a and 118b may rotate the perspective, as described in connection with FIGS. 11A and 11B. The user may also specify a rate of movement. For example, the user may specify accelerated movement, such that each meter the user walks in physical space is translated to N meters of movement in virtual space—through the maze 1310.”; para. [0125], “In an example, virtual interconnects 118c and 118d specify a 1:1 scale movement, such that every meter the user walks in physical space is mapped 1:1 to a meter in virtual space. Virtual interconnects 118e and 118f may require the user 120 to enter a VR realm to cross the indicated distance. The VR realm may present particular challenges that the user must overcome to cross the indicated distance.”; para. [0126], “FIG. 14 shows an example spot 1410 that users may step into to enter a VR realm. A game or other software application may place such “VR Anywhere” spots 1410 at desired locations within a play zone. Once inside a VR Anywhere spot 1410, the user 120 is able perform VR commands to navigate a synthetic VR space, e.g., using the above-described physical movements mapped to control commands. The VR space is not required to have any relationship to the physical space in which the user is located. The VR Anywhere spot 1410 is preferably a small area, e.g., just large enough to safely accommodate the user 120 executing the predefined movements. In some examples, VR control commands are enabled only when the user 120 remains within the VR Anywhere spot 1410 and are disabled if the user steps out of the VR Anywhere spot 1410. In some examples, the system checks local map data 160 and locations of other users to ensure that no mapped objects are within some predetermined radius of the VR Anywhere spot 1410, to protect the user 120 (and other users) from harm that may result from sudden movements of the user's head, arms, legs, or torso”)
Regarding claim 11, Gullicksen discloses The system of claim 2, wherein the system executes instructions for spatial synchronization of one or more than one user located in a physical location using a controller synchronization method, the controller synchronization method comprising instructions operable on a processor for: (para. [0020], “The computer program product stores instructions which, when executed on control circuitry of a data storage system, cause the data storage system to perform a method of managing virtual content to be displayed to users via headsets, such as the method described above.“; [0059] FIG. 3 shows a schematic view of the user 120 wearing the headset 130 as well as various wearable sensors 310. The headset 130 and the wearable sensors 310 track the user's movements and enable the server apparatus 250 to construct a skeletal model of the user 120. The resulting skeletal model has two main uses. First, it enables the server apparatus 150 to track the user's body position and movements, such that the server apparatus 150 can construct a sharable model of the user that moves in synchronization with the user. Second, it enables the server apparatus 150 and/or the headset 130 to map user movements to a motion-based set of control commands. For example, the server apparatus 150 and/or headset 130 may interpret particular movements of the user's head, arms, legs, and/or torso as control commands, which are interpreted to initiate particular activities.”; para. [0113], “In addition to receiving shared map data 160a, the second server apparatus 150-2 also receives, over the WAN/Internet 250, a real-time representation of the avatar 430 of the first user 120, including its location and orientation relative to the play zone. With this arrangement, the second user 120-2 is able to see the avatar 430 of the first user 120 move within the second physical space 110-2 (movement shown by arrow 1010) in a manner that mirrors the movement of the first user 120 within the first physical space 110. In a like manner, the first user 120, within the first physical space 110, sees an avatar of the second user 120-2 displayed and moving in real time in synchronization with the second user 120-2. This arrangement is extendible to any number of users, with each user's headset rendering the holograms 114, 116, and 118 and the avatars of the other users, with the avatars moving in real time to reflect movements of the users they represent. Also, each user's headset, with the possible exception of the first user's (where it may not be necessary), renders a virtual rendition of the shared portions of the physical space 110. Owing to the global perspective afforded by the stationary sensors and the computerized apparatus in each physical space, the position and orientation of each avatar relative to the play zone matches the position and orientation of the respective physical user in that user's local play zone.”
a. placing a controller in a predefined location by a first user; ([0007] VR (virtual reality) enables users to enter immersive, virtual environments without leaving the comfort and safety of their physical environments. Users of VR technology can move through virtual spaces and interact with rendered content, such as 3-D (three dimensional) representations of objects, scenery, and other users. At the center of VR technology is the VR headset, which displays images onto opaque screens placed in front of a user's eyes, allowing the user to visualize displayed content in three dimensions. The VR headset typically includes accelerometers and other sensors, which provide input to a 3-D rendering engine, such that the user can change the displayed view of the virtual environment by making natural head movements. The VR headset typically also includes speakers and a microphone, which allow users to hear sounds in the virtual environment and to communicate with others. The user may control a VR application, such as a game, virtual walk-through, or other application using a hand-held controller. A well-known example of a VR headset is the Oculus Rift, available from Oculus VR, LLC.; para. [0081], “At 520, an initial 3-D location of each of the wearable sensors 310 is established relative to the headset 130. For example, the user 120 may place each wearable sensor 310 adjacent to an object having known dimensions. The user 120 may then point the line of sight 132 of the headset 130 toward the object and direct the headset 130 to compute a precise location in space of the wearable sensor 310 relative to the headset's own location. This location is also relative to the physical space 110, as it is based on the headset's location as measured by the accurate antenna/LPS system. The headset 130 may simultaneously query the wearable sensor 310 to obtain its self-reported location as indicated by the measurement circuitry 314 within the wearable sensor. This self-reported location may be used as a baseline going forward. For example, the headset 130 may compute a 3-D correction vector based on a difference between the initial location of the wearable sensor 310 as reported by the headset 130 and the self-reported location as reported by the wearable sensor 310.”; para. [0105], “FIGS. 9A and 9B depict an example arrangement for displaying holograms through the headset 130 relative to body appendages of the user. FIG. 9A shows an example user view 900 through the headset 130. The user's arm 910 can be seen (with wrist sensor 310a), as well as a controller 920, which is shaped like a sword. The controller 920 may include buttons and other controls for enhancing game play. A hologram 930, which represents some virtual object in the background, is also visible in the user's view 900. Maintaining the hologram 930 in the background relative to the user's arm 910 and the controller 920 is a difficult problem for conventional MR systems but is readily managed here, as shown in FIG. 9B.”
b. identifying a first point by pressing a first button on the controller, issuing a first voice command, or a first gesture command by the first user; (para. [0081], “At 520, an initial 3-D location of each of the wearable sensors 310 is established relative to the headset 130. For example, the user 120 may place each wearable sensor 310 adjacent to an object having known dimensions. The user 120 may then point the line of sight 132 of the headset 130 toward the object and direct the headset 130 to compute a precise location in space of the wearable sensor 310 relative to the headset's own location. This location is also relative to the physical space 110, as it is based on the headset's location as measured by the accurate antenna/LPS system. The headset 130 may simultaneously query the wearable sensor 310 to obtain its self-reported location as indicated by the measurement circuitry 314 within the wearable sensor. This self-reported location may be used as a baseline going forward. For example, the headset 130 may compute a 3-D correction vector based on a difference between the initial location of the wearable sensor 310 as reported by the headset 130 and the self-reported location as reported by the wearable sensor 310.”; para. [0082] At 530, the user 120 may place the wearable sensor 310 on the body, such as on the wrist. As the user moves, the headset 130 may again query the wearable sensor 310 for its self-reported location, correcting the measured values by applying the 3-D vector. Similar activities may be performed for other wearable sensors.”; para. [0083], “FIG. 6 shows an example sensor platter 600 that may be used for establishing initial locations of the wearable sensors 310. The sensor platter 600 has a substrate 610, such as a printed circuit board, and docking locations 612a-612f for receiving wearable sensors 310. The sensor platter 600 may also have a grip region 620 and a charging connector 630. Visual identification marks 640 are printed on the substrate 610 to facilitate the measurement of distance and perspective by the headset 130.”; para, “[0093], “Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows:”; para. [0105], “FIGS. 9A and 9B depict an example arrangement for displaying holograms through the headset 130 relative to body appendages of the user. FIG. 9A shows an example user view 900 through the headset 130. The user's arm 910 can be seen (with wrist sensor 310a), as well as a controller 920, which is shaped like a sword. The controller 920 may include buttons and other controls for enhancing game play. A hologram 930, which represents some virtual object in the background, is also visible in the user's view 900. Maintaining the hologram 930 in the background relative to the user's arm 910 and the controller 920 is a difficult problem for conventional MR systems but is readily managed here, as shown in FIG. 9B.”)
c. placing a second controller in the same or different predefined location, by a second user; (para. [0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0081], “At 520, an initial 3-D location of each of the wearable sensors 310 is established relative to the headset 130. For example, the user 120 may place each wearable sensor 310 adjacent to an object having known dimensions. The user 120 may then point the line of sight 132 of the headset 130 toward the object and direct the headset 130 to compute a precise location in space of the wearable sensor 310 relative to the headset's own location. This location is also relative to the physical space 110, as it is based on the headset's location as measured by the accurate antenna/LPS system. The headset 130 may simultaneously query the wearable sensor 310 to obtain its self-reported location as indicated by the measurement circuitry 314 within the wearable sensor. This self-reported location may be used as a baseline going forward. For example, the headset 130 may compute a 3-D correction vector based on a difference between the initial location of the wearable sensor 310 as reported by the headset 130 and the self-reported location as reported by the wearable sensor 310.”; para. [0082], “At 530, the user 120 may place the wearable sensor 310 on the body, such as on the wrist. As the user moves, the headset 130 may again query the wearable sensor 310 for its self-reported location, correcting the measured values by applying the 3-D vector. Similar activities may be performed for other wearable sensors.”; para. [0111], “In the example shown, the second server apparatus 150-2 has obtained shared map data 160a from the first server apparatus 150 (FIG. 1), e.g., over the WAN/Internet 250. The shared map data 160a includes portions of the map data 160. Sharing may be accomplished, for example, by the owner of the vault appliance 230 in the first physical space 110 granting a content right to the second user 120-2 to access particular features in the physical space 110. Here, map data describing features of the first room 110a are shared, such as walls 112a, whereas map data describing other portions of the physical space 110 are not shared, such as data describing the room 110b and walls 112b (e.g., the first user 120 may wish to keep the room 110b private). In general, map data 160 may be shared at any desired level of granularity. By obtaining the shared map data 160a, a second headset 130-2 worn by the second user 120-2 is able to render objects that are physical in the first physical space 120 as virtual objects in the second physical space 110-2. Thus, the second user 120-2 is able to visualize the room 110a and walls 112a as holograms 110a-V and 112a-V, respectively.”; para. [0116], “In some examples, a game or other software application may generate the play zone, which need not correspond to features in any physical space. For example, the game may define the play zone and share that play zone with all users. The server apparatus at each location may instantiate the play zone locally. Users can then view features of the play zone and avatars of other users through their respective headsets. Games or other software applications may also produce computer-generated avatars, which move through the play zone and can be viewed at corresponding locations by all users.”
d. identifying a second point by pressing a second button on the second controller, issuing a second voice command or a second gesture command by the second user; (para. [0021], “Further embodiments include methods of generating skeletal models of users, methods of interpreting user movements as control commands, methods of calibrating positions and/or orientations of wearable sensors, methods of reorienting virtually-rendered spaces via virtual interconnects, methods of teleporting users between virtual spaces via virtual interconnects, and methods of entering and exiting VR realms. Still further embodiments are directed to wearable sensors, sensor platters, MR headsets, game consoles, location positioning systems, and security vaults, for implementing any of the above methods, as well as to computer-readable media from which software instructions for carrying out such methods may be installed. Some embodiments involve multiple components, which may be deployed over a computer network.”; para. [0081], “At 520, an initial 3-D location of each of the wearable sensors 310 is established relative to the headset 130. For example, the user 120 may place each wearable sensor 310 adjacent to an object having known dimensions. The user 120 may then point the line of sight 132 of the headset 130 toward the object and direct the headset 130 to compute a precise location in space of the wearable sensor 310 relative to the headset's own location. This location is also relative to the physical space 110, as it is based on the headset's location as measured by the accurate antenna/LPS system. The headset 130 may simultaneously query the wearable sensor 310 to obtain its self-reported location as indicated by the measurement circuitry 314 within the wearable sensor. This self-reported location may be used as a baseline going forward. For example, the headset 130 may compute a 3-D correction vector based on a difference between the initial location of the wearable sensor 310 as reported by the headset 130 and the self-reported location as reported by the wearable sensor 310.”; para. [0082], “At 530, the user 120 may place the wearable sensor 310 on the body, such as on the wrist. As the user moves, the headset 130 may again query the wearable sensor 310 for its self-reported location, correcting the measured values by applying the 3-D vector. Similar activities may be performed for other wearable sensors.”; para.[0085], “With the wearable sensors 310 inserted into the sensor platter 600, the user 120 may calibrate the wearable sensors 310 by wearing the headset 130 and pointing the line of sight 132 toward the visual indication marks 640. As the locations of the wearable sensors 310 relative to the sensor platter 600 are known and the visual indication marks 640 have a known size and direction (as indicated by the arrow), the headset 130 can compute the location of each wearable sensor 310 relative to the headset 130, and can thus establish the initial 3-D locations of all of the wearable sensors 310 at once”; para. [0093] “Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows:”; para. [0105], “FIGS. 9A and 9B depict an example arrangement for displaying holograms through the headset 130 relative to body appendages of the user. FIG. 9A shows an example user view 900 through the headset 130. The user's arm 910 can be seen (with wrist sensor 310a), as well as a controller 920, which is shaped like a sword. The controller 920 may include buttons and other controls for enhancing game play. A hologram 930, which represents some virtual object in the background, is also visible in the user's view 900. Maintaining the hologram 930 in the background relative to the user's arm 910 and the controller 920 is a difficult problem for conventional MR systems but is readily managed here, as shown in FIG. 9B.”
e. synchronizing both the first user and the second user in an XR virtual environment bounded by a location and apparatus, enabling both the first user and the second user to move about the location and apparatus and the XR virtual environment freely; and (para. [0011], “In contrast with conventional VR, AR, and MR systems, an improved technique for rendering virtual content to a user stores map data of features in a physical environment of the user and measures the location of the user with stationary sensors placed at respective locations within the environment. A server provides the location of the user and portions of the map data to a headset worn by the user. The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data.”; para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0013], “In examples where applications involve multiple users, the server may track locations of users in the same physical space and maintain a global perspective of all of them, such that the server maintains all user locations even when the user have no line of sight to one another.”; para. [0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0059], “FIG. 3 shows a schematic view of the user 120 wearing the headset 130 as well as various wearable sensors 310. The headset 130 and the wearable sensors 310 track the user's movements and enable the server apparatus 250 to construct a skeletal model of the user 120. The resulting skeletal model has two main uses. First, it enables the server apparatus 150 to track the user's body position and movements, such that the server apparatus 150 can construct a sharable model of the user that moves in synchronization with the user. Second, it enables the server apparatus 150 and/or the headset 130 to map user movements to a motion-based set of control commands. For example, the server apparatus 150 and/or headset 130 may interpret particular movements of the user's head, arms, legs, and/or torso as control commands, which are interpreted to initiate particular activities.”; para. [0113], “In addition to receiving shared map data 160a, the second server apparatus 150-2 also receives, over the WAN/Internet 250, a real-time representation of the avatar 430 of the first user 120, including its location and orientation relative to the play zone. With this arrangement, the second user 120-2 is able to see the avatar 430 of the first user 120 move within the second physical space 110-2 (movement shown by arrow 1010) in a manner that mirrors the movement of the first user 120 within the first physical space 110. In a like manner, the first user 120, within the first physical space 110, sees an avatar of the second user 120-2 displayed and moving in real time in synchronization with the second user 120-2. This arrangement is extendible to any number of users, with each user's headset rendering the holograms 114, 116, and 118 and the avatars of the other users, with the avatars moving in real time to reflect movements of the users they represent. Also, each user's headset, with the possible exception of the first user's (where it may not be necessary), renders a virtual rendition of the shared portions of the physical space 110. Owing to the global perspective afforded by the stationary sensors and the computerized apparatus in each physical space, the position and orientation of each avatar relative to the play zone matches the position and orientation of the respective physical user in that user's local play zone.”; para. [0122], “FIG. 12B shows the same situation from the viewpoint of the first user 120 in the first physical space 110. The first user 120 sees, through the headset 130, the avatar 430-2 of the second user 120-2, but the avatar 430-2 appears to be standing still—until the second user returns such that tracking can resume.”; para. [0123], “Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”)
f. repeating steps c-e to add additional users. (para. [0059], “FIG. 3 shows a schematic view of the user 120 wearing the headset 130 as well as various wearable sensors 310. The headset 130 and the wearable sensors 310 track the user's movements and enable the server apparatus 250 to construct a skeletal model of the user 120. The resulting skeletal model has two main uses. First, it enables the server apparatus 150 to track the user's body position and movements, such that the server apparatus 150 can construct a sharable model of the user that moves in synchronization with the user. Second, it enables the server apparatus 150 and/or the headset 130 to map user movements to a motion-based set of control commands. For example, the server apparatus 150 and/or headset 130 may interpret particular movements of the user's head, arms, legs, and/or torso as control commands, which are interpreted to initiate particular activities.”; para. [0091], “At 740, the system determines whether the first measurement for any wearable sensor 310 falls outside the respective 3-D bounding region for that wearable sensor. If so, the system initiates a wearable-sensor-retraining operation at the first opportunity. For example, the system may wait until the user 120 enters a designated region, such as the region indicated by object 118 in FIG. 1. The user 120 may previously have been instructed to assume a neutral body position when standing within that region, such as standing up straight with arms at the sides. For example, assuming this position may be pre-arranged as port of normal game play. Upon detecting that the user has entered the region indicated by object 118, the system measures the locations of the wearable sensors 310 using the measurement circuitry 314 (as done in the first measurements, above), and compares the sensor locations with expected locations based on the skeletal model 420. The system then generates a new correction vector for each wearable sensor 310 and applies the new correction vectors going forward.”; para. [0116], “In some examples, a game or other software application may generate the play zone, which need not correspond to features in any physical space. For example, the game may define the play zone and share that play zone with all users. The server apparatus at each location may instantiate the play zone locally. Users can then view features of the play zone and avatars of other users through their respective headsets. Games or other software applications may also produce computer-generated avatars, which move through the play zone and can be viewed at corresponding locations by all users.”)
Regarding claim 12, Gullicksen discloses The system of claim 2, wherein after the users are synchronized, the system provides real-time tracking, free-roaming, manipulation and interaction with some virtual elements, and social interaction of the one or more than one user in both the physical location and the XR virtual environment in a single shared XR virtual environment, wherein the system is scalable in any instance for any amount of users located anywhere. (para. [0011] – [0013], “In contrast with conventional VR, AR, and MR systems, an improved technique for rendering virtual content to a user stores map data of features in a physical environment of the user and measures the location of the user with stationary sensors placed at respective locations within the environment. A server provides the location of the user and portions of the map data to a headset worn by the user. The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data. Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors. In examples where applications involve multiple users, the server may track locations of users in the same physical space and maintain a global perspective of all of them, such that the server maintains all user locations even when the user have no line of sight to one another.”; para. [0044], “In example operation, the server apparatus 150 receives a stream of inputs from the sensors 140 and generates measured locations of the user 120 as the user moves around in the physical space 110. In this example, the location of the user 120 is taken as the location of the headset 130 worn by the user 120. The server apparatus 150 sends the measured user location, information about holograms, and map data 160 (or portions thereof) to the headset 130. The headset 130 processes the user location and map data 160 to render the holograms. For example, the headset 130 applies hologram information and map data 160 to generate a virtual model of objects within its line of sight 132, based on the headset's current 3-D location and orientation. The headset 130 then renders the holograms that fall within the headset's field of view. The user 120 is thus able to see virtual objects 114, 116, and 118 placed in their proper locations and orientations within the physical space 110.”; para. [0115], “Synthetic content may be supplied by users, by games, and/or by third-party software, for example. Sharing of content may go in any direction. For example, the second user 120-2 may design a virtual object and place that object in a particular location within the play zone, sharing that object back to other users via respective server apparatus.”; para. [0011], “In contrast with conventional VR, AR, and MR systems, an improved technique for rendering virtual content to a user stores map data of features in a physical environment of the user and measures the location of the user with stationary sensors placed at respective locations within the environment. A server provides the location of the user and portions of the map data to a headset worn by the user. The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data.”; para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0109], “Attention to this point has been focused on activities involving a single user 120. However, many applications, such as games, may involve multiple users. For example, multiple users may occupy the physical space 110 shown in FIG. 1, with each user's location tracked by the sensors 140 and server apparatus 150, i.e., in the same manner as described for the user 120. Each user may create a skeletal model 420, which may be rendered as an avatar 430 (FIG. 4). In some examples, each user in the physical space 110 sees all the other users as avatars, which may be superimposed over the user's physical bodies.”; para. [0110], “FIG. 10 shows another multi-user scenario. Here, a second user 120-2 is located within a second physical space 110-2, which is discontinuous and remote from the physical space 110 shown in FIG. 1 (hereinafter, the “first” physical space). The second physical space 110-2 is configured similarly to the first, and includes its own sensors 140a-2 through 140d-2 and server apparatus 150-2. The second physical space 110-2 may have a different shape from the first physical space 110. The “play zone” is an area designated for game play or other activity. Furniture 1020 and 1022 in the second physical space 110-2 may be moved out of the way to clear space for the play zone.”; para. [0123], “Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”; para. [0124], “FIG. 13 shows an example of a different play zone, which is laid out as a simple maze 1310. Here, the walls of the maze are set up as barriers, such that users must stay within the designated paths of the maze to avoid becoming separated from their avatars. FIG. 13 also shows various uses of virtual interconnects. For example, the system provides virtual interconnects 118a and 118b at maze decision points. A user 120 entering these virtual interconnects 118a and 118b may rotate the perspective, as described in connection with FIGS. 11A and 11B. The user may also specify a rate of movement. For example, the user may specify accelerated movement, such that each meter the user walks in physical space is translated to N meters of movement in virtual space—through the maze 1310.”; para. [0125], “In an example, virtual interconnects 118c and 118d specify a 1:1 scale movement, such that every meter the user walks in physical space is mapped 1:1 to a meter in virtual space. Virtual interconnects 118e and 118f may require the user 120 to enter a VR realm to cross the indicated distance. The VR realm may present particular challenges that the user must overcome to cross the indicated distance.”; para. [0126], “FIG. 14 shows an example spot 1410 that users may step into to enter a VR realm. A game or other software application may place such “VR Anywhere” spots 1410 at desired locations within a play zone. Once inside a VR Anywhere spot 1410, the user 120 is able perform VR commands to navigate a synthetic VR space, e.g., using the above-described physical movements mapped to control commands. The VR space is not required to have any relationship to the physical space in which the user is located. The VR Anywhere spot 1410 is preferably a small area, e.g., just large enough to safely accommodate the user 120 executing the predefined movements. In some examples, VR control commands are enabled only when the user 120 remains within the VR Anywhere spot 1410 and are disabled if the user steps out of the VR Anywhere spot 1410. In some examples, the system checks local map data 160 and locations of other users to ensure that no mapped objects are within some predetermined radius of the VR Anywhere spot 1410, to protect the user 120 (and other users) from harm that may result from sudden movements of the user's head, arms, legs, or torso.”; [0127], “FIG. 15 shows an example method 1500 of managing virtual content to be displayed to users via three-dimensional imaging headsets, such as AR and MR headsets. The method 1500 may be carried out, for example, by the server apparatus 150 described in connection with FIG. 2. The various acts of method 1500 may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in orders different from that illustrated, which may include performing some acts simultaneously.”)
Regarding claim 13, Gullicksen discloses The system of claim 2, wherein the system performs spatial synchronization of one or more than one user in a physical location using a headset synchronization method, the headset synchronization method comprising instructions operable on a processor for: (para. [0059], “FIG. 3 shows a schematic view of the user 120 wearing the headset 130 as well as various wearable sensors 310. The headset 130 and the wearable sensors 310 track the user's movements and enable the server apparatus 250 to construct a skeletal model of the user 120. The resulting skeletal model has two main uses. First, it enables the server apparatus 150 to track the user's body position and movements, such that the server apparatus 150 can construct a sharable model of the user that moves in synchronization with the user. Second, it enables the server apparatus 150 and/or the headset 130 to map user movements to a motion-based set of control commands. For example, the server apparatus 150 and/or headset 130 may interpret particular movements of the user's head, arms, legs, and/or torso as control commands, which are interpreted to initiate particular activities.”; para. [0113], “In addition to receiving shared map data 160a, the second server apparatus 150-2 also receives, over the WAN/Internet 250, a real-time representation of the avatar 430 of the first user 120, including its location and orientation relative to the play zone. With this arrangement, the second user 120-2 is able to see the avatar 430 of the first user 120 move within the second physical space 110-2 (movement shown by arrow 1010) in a manner that mirrors the movement of the first user 120 within the first physical space 110. In a like manner, the first user 120, within the first physical space 110, sees an avatar of the second user 120-2 displayed and moving in real time in synchronization with the second user 120-2. This arrangement is extendible to any number of users, with each user's headset rendering the holograms 114, 116, and 118 and the avatars of the other users, with the avatars moving in real time to reflect movements of the users they represent. Also, each user's headset, with the possible exception of the first user's (where it may not be necessary), renders a virtual rendition of the shared portions of the physical space 110. Owing to the global perspective afforded by the stationary sensors and the computerized apparatus in each physical space, the position and orientation of each avatar relative to the play zone matches the position and orientation of the respective physical user in that user's local play zone.”)
a. stepping on a first predefined point and staring straight ahead by a first user; (para. [0081], “At 520, an initial 3-D location of each of the wearable sensors 310 is established relative to the headset 130. For example, the user 120 may place each wearable sensor 310 adjacent to an object having known dimensions. The user 120 may then point the line of sight 132 of the headset 130 toward the object and direct the headset 130 to compute a precise location in space of the wearable sensor 310 relative to the headset's own location. This location is also relative to the physical space 110, as it is based on the headset's location as measured by the accurate antenna/LPS system. The headset 130 may simultaneously query the wearable sensor 310 to obtain its self-reported location as indicated by the measurement circuitry 314 within the wearable sensor. This self-reported location may be used as a baseline going forward. For example, the headset 130 may compute a 3-D correction vector based on a difference between the initial location of the wearable sensor 310 as reported by the headset 130 and the self-reported location as reported by the wearable sensor 310.”; para. [0085], “With the wearable sensors 310 inserted into the sensor platter 600, the user 120 may calibrate the wearable sensors 310 by wearing the headset 130 and pointing the line of sight 132 toward the visual indication marks 640. As the locations of the wearable sensors 310 relative to the sensor platter 600 are known and the visual indication marks 640 have a known size and direction (as indicated by the arrow), the headset 130 can compute the location of each wearable sensor 310 relative to the headset 130, and can thus establish the initial 3-D locations of all of the wearable sensors 310 at once.”; para. [0126] FIG. 14 shows an example spot 1410 that users may step into to enter a VR realm. A game or other software application may place such “VR Anywhere” spots 1410 at desired locations within a play zone. Once inside a VR Anywhere spot 1410, the user 120 is able perform VR commands to navigate a synthetic VR space, e.g., using the above-described physical movements mapped to control commands. The VR space is not required to have any relationship to the physical space in which the user is located. The VR Anywhere spot 1410 is preferably a small area, e.g., just large enough to safely accommodate the user 120 executing the predefined movements. In some examples, VR control commands are enabled only when the user 120 remains within the VR Anywhere spot 1410 and are disabled if the user steps out of the VR Anywhere spot 1410. In some examples, the system checks local map data 160 and locations of other users to ensure that no mapped objects are within some predetermined radius of the VR Anywhere spot 1410, to protect the user 120 (and other users) from harm that may result from sudden movements of the user's head, arms, legs, or torso.”)
b. synchronizing the first user by the first user pressing a button on a first controller, using a first verbal command, using a first gesture command, or a combination thereof; (para. [0059], “FIG. 3 shows a schematic view of the user 120 wearing the headset 130 as well as various wearable sensors 310. The headset 130 and the wearable sensors 310 track the user's movements and enable the server apparatus 250 to construct a skeletal model of the user 120. The resulting skeletal model has two main uses. First, it enables the server apparatus 150 to track the user's body position and movements, such that the server apparatus 150 can construct a sharable model of the user that moves in synchronization with the user. Second, it enables the server apparatus 150 and/or the headset 130 to map user movements to a motion-based set of control commands. For example, the server apparatus 150 and/or headset 130 may interpret particular movements of the user's head, arms, legs, and/or torso as control commands, which are interpreted to initiate particular activities.”; para. [0093], “Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows:”; para. [0105], “FIGS. 9A and 9B depict an example arrangement for displaying holograms through the headset 130 relative to body appendages of the user. FIG. 9A shows an example user view 900 through the headset 130. The user's arm 910 can be seen (with wrist sensor 310a), as well as a controller 920, which is shaped like a sword. The controller 920 may include buttons and other controls for enhancing game play. A hologram 930, which represents some virtual object in the background, is also visible in the user's view 900. Maintaining the hologram 930 in the background relative to the user's arm 910 and the controller 920 is a difficult problem for conventional MR systems but is readily managed here, as shown in FIG. 9B.”)
c. moving away from the first predefined point by the first user; (para. [0091], “At 740, the system determines whether the first measurement for any wearable sensor 310 falls outside the respective 3-D bounding region for that wearable sensor. If so, the system initiates a wearable-sensor-retraining operation at the first opportunity. For example, the system may wait until the user 120 enters a designated region, such as the region indicated by object 118 in FIG. 1. The user 120 may previously have been instructed to assume a neutral body position when standing within that region, such as standing up straight with arms at the sides. For example, assuming this position may be pre-arranged as port of normal game play. Upon detecting that the user has entered the region indicated by object 118, the system measures the locations of the wearable sensors 310 using the measurement circuitry 314 (as done in the first measurements, above), and compares the sensor locations with expected locations based on the skeletal model 420. The system then generates a new correction vector for each wearable sensor 310 and applies the new correction vectors going forward.”; para. [0123], “Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”; para. [0126], “FIG. 14 shows an example spot 1410 that users may step into to enter a VR realm. A game or other software application may place such “VR Anywhere” spots 1410 at desired locations within a play zone. Once inside a VR Anywhere spot 1410, the user 120 is able perform VR commands to navigate a synthetic VR space, e.g., using the above-described physical movements mapped to control commands. The VR space is not required to have any relationship to the physical space in which the user is located. The VR Anywhere spot 1410 is preferably a small area, e.g., just large enough to safely accommodate the user 120 executing the predefined movements. In some examples, VR control commands are enabled only when the user 120 remains within the VR Anywhere spot 1410 and are disabled if the user steps out of the VR Anywhere spot 1410. In some examples, the system checks local map data 160 and locations of other users to ensure that no mapped objects are within some predetermined radius of the VR Anywhere spot 1410, to protect the user 120 (and other users) from harm that may result from sudden movements of the user's head, arms, legs, or torso.”)
d. stepping on the first predefined point or a second predefined point and staring straight ahead by a second user; (para. [0081], “At 520, an initial 3-D location of each of the wearable sensors 310 is established relative to the headset 130. For example, the user 120 may place each wearable sensor 310 adjacent to an object having known dimensions. The user 120 may then point the line of sight 132 of the headset 130 toward the object and direct the headset 130 to compute a precise location in space of the wearable sensor 310 relative to the headset's own location. This location is also relative to the physical space 110, as it is based on the headset's location as measured by the accurate antenna/LPS system. The headset 130 may simultaneously query the wearable sensor 310 to obtain its self-reported location as indicated by the measurement circuitry 314 within the wearable sensor. This self-reported location may be used as a baseline going forward. For example, the headset 130 may compute a 3-D correction vector based on a difference between the initial location of the wearable sensor 310 as reported by the headset 130 and the self-reported location as reported by the wearable sensor 310.”; para. [0085], “With the wearable sensors 310 inserted into the sensor platter 600, the user 120 may calibrate the wearable sensors 310 by wearing the headset 130 and pointing the line of sight 132 toward the visual indication marks 640. As the locations of the wearable sensors 310 relative to the sensor platter 600 are known and the visual indication marks 640 have a known size and direction (as indicated by the arrow), the headset 130 can compute the location of each wearable sensor 310 relative to the headset 130, and can thus establish the initial 3-D locations of all of the wearable sensors 310 at once.”; para. [0126], “FIG. 14 shows an example spot 1410 that users may step into to enter a VR realm. A game or other software application may place such “VR Anywhere” spots 1410 at desired locations within a play zone. Once inside a VR Anywhere spot 1410, the user 120 is able perform VR commands to navigate a synthetic VR space, e.g., using the above-described physical movements mapped to control commands. The VR space is not required to have any relationship to the physical space in which the user is located. The VR Anywhere spot 1410 is preferably a small area, e.g., just large enough to safely accommodate the user 120 executing the predefined movements. In some examples, VR control commands are enabled only when the user 120 remains within the VR Anywhere spot 1410 and are disabled if the user steps out of the VR Anywhere spot 1410. In some examples, the system checks local map data 160 and locations of other users to ensure that no mapped objects are within some predetermined radius of the VR Anywhere spot 1410, to protect the user 120 (and other users) from harm that may result from sudden movements of the user's head, arms, legs, or torso.”)
e. synchronizing the second user by the second user pressing a button on a second controller, using a second verbal command, using a second gesture command, or a combination thereof; (para. [0059], “FIG. 3 shows a schematic view of the user 120 wearing the headset 130 as well as various wearable sensors 310. The headset 130 and the wearable sensors 310 track the user's movements and enable the server apparatus 250 to construct a skeletal model of the user 120. The resulting skeletal model has two main uses. First, it enables the server apparatus 150 to track the user's body position and movements, such that the server apparatus 150 can construct a sharable model of the user that moves in synchronization with the user. Second, it enables the server apparatus 150 and/or the headset 130 to map user movements to a motion-based set of control commands. For example, the server apparatus 150 and/or headset 130 may interpret particular movements of the user's head, arms, legs, and/or torso as control commands, which are interpreted to initiate particular activities.”; para. [0093], “Once the wearable sensors 310 have been trained, users may issue control commands by moving their limbs and/or torso in predetermined ways. As shown in FIG. 8, the method 800 may detect predetermined patterns of changes in locations and/or orientations of the wearable sensors (step 810) based on output from the measurement circuitry 314 in the wearable sensors 310. For example, the measurement circuitry 314 in the wearable sensors 310 reports changes in position and/or orientation of each sensor, which the headset 130 and/or server apparatus 150 processes to detect predetermined patterns. For each predetermined pattern detected, the system may map that pattern to a respective control command and then execute the mapped-to control command (step 820). Some examples of predetermined patterns of movement are as follows:”; para. [0105], “FIGS. 9A and 9B depict an example arrangement for displaying holograms through the headset 130 relative to body appendages of the user. FIG. 9A shows an example user view 900 through the headset 130. The user's arm 910 can be seen (with wrist sensor 310a), as well as a controller 920, which is shaped like a sword. The controller 920 may include buttons and other controls for enhancing game play. A hologram 930, which represents some virtual object in the background, is also visible in the user's view 900. Maintaining the hologram 930 in the background relative to the user's arm 910 and the controller 920 is a difficult problem for conventional MR systems but is readily managed here, as shown in FIG. 9B.”)
g. positionally synchronizing the first user and the second user in an XR virtual environment and in the selected physical environment; wherein both the first user and the second user are able to move about the physical location and the XR virtual environment freely; and (para. [0011], “In contrast with conventional VR, AR, and MR systems, an improved technique for rendering virtual content to a user stores map data of features in a physical environment of the user and measures the location of the user with stationary sensors placed at respective locations within the environment. A server provides the location of the user and portions of the map data to a headset worn by the user. The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data.”; para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0013], “In examples where applications involve multiple users, the server may track locations of users in the same physical space and maintain a global perspective of all of them, such that the server maintains all user locations even when the user have no line of sight to one another.”; para. [0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0113], “In addition to receiving shared map data 160a, the second server apparatus 150-2 also receives, over the WAN/Internet 250, a real-time representation of the avatar 430 of the first user 120, including its location and orientation relative to the play zone. With this arrangement, the second user 120-2 is able to see the avatar 430 of the first user 120 move within the second physical space 110-2 (movement shown by arrow 1010) in a manner that mirrors the movement of the first user 120 within the first physical space 110. In a like manner, the first user 120, within the first physical space 110, sees an avatar of the second user 120-2 displayed and moving in real time in synchronization with the second user 120-2. This arrangement is extendible to any number of users, with each user's headset rendering the holograms 114, 116, and 118 and the avatars of the other users, with the avatars moving in real time to reflect movements of the users they represent. Also, each user's headset, with the possible exception of the first user's (where it may not be necessary), renders a virtual rendition of the shared portions of the physical space 110. Owing to the global perspective afforded by the stationary sensors and the computerized apparatus in each physical space, the position and orientation of each avatar relative to the play zone matches the position and orientation of the respective physical user in that user's local play zone.”; para. [0122], “FIG. 12B shows the same situation from the viewpoint of the first user 120 in the first physical space 110. The first user 120 sees, through the headset 130, the avatar 430-2 of the second user 120-2, but the avatar 430-2 appears to be standing still—until the second user returns such that tracking can resume.”; para. [0123], “Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”)
Regarding claim 14, Gullicksen discloses The system of claim 13, further comprising the step of displaying in the headset a cross hair graphic (116 in FIG. 11B) and orienting the headset using the cross hair graphic to a specified marker in the physical environment to enhance precision. (para. [0044], “In example operation, the server apparatus 150 receives a stream of inputs from the sensors 140 and generates measured locations of the user 120 as the user moves around in the physical space 110. In this example, the location of the user 120 is taken as the location of the headset 130 worn by the user 120. The server apparatus 150 sends the measured user location, information about holograms, and map data 160 (or portions thereof) to the headset 130. The headset 130 processes the user location and map data 160 to render the holograms. For example, the headset 130 applies hologram information and map data 160 to generate a virtual model of objects within its line of sight 132, based on the headset's current 3-D location and orientation. The headset 130 then renders the holograms that fall within the headset's field of view. The user 120 is thus able to see virtual objects 114, 116, and 118 placed in their proper locations and orientations within the physical space 110.”; para. [0085], “With the wearable sensors 310 inserted into the sensor platter 600, the user 120 may calibrate the wearable sensors 310 by wearing the headset 130 and pointing the line of sight 132 toward the visual indication marks 640. As the locations of the wearable sensors 310 relative to the sensor platter 600 are known and the visual indication marks 640 have a known size and direction (as indicated by the arrow), the headset 130 can compute the location of each wearable sensor 310 relative to the headset 130, and can thus establish the initial 3-D locations of all of the wearable sensors 310 at once.”
Gullicksen discloses displaying in the headset a cross hair graphic and orienting headset to specified marker. Therefore, it would be obvious to include the step of displaying in the headset a cross hair graphic and orienting the headset using the cross hair graphic to a specified marker in the physical environment to enhance precision, in the context of spatial synchronization, in order to measure location of wearable sensors accurately (para. [0079] of Gullicksen).
Regarding claim 15, Gullicksen discloses The system of claim 2, wherein after the one or more than one user is synchronized, the system enables real-time tracking, free-roaming, manipulation of and interaction with virtual elements, and social interaction of the one or more than one user in both the physical location and the remote XR virtual environment in a single shared XR virtual environment, wherein the system is scalable in any instance for any amount of users located anywhere. (para. [0044], “In example operation, the server apparatus 150 receives a stream of inputs from the sensors 140 and generates measured locations of the user 120 as the user moves around in the physical space 110. In this example, the location of the user 120 is taken as the location of the headset 130 worn by the user 120. The server apparatus 150 sends the measured user location, information about holograms, and map data 160 (or portions thereof) to the headset 130. The headset 130 processes the user location and map data 160 to render the holograms. For example, the headset 130 applies hologram information and map data 160 to generate a virtual model of objects within its line of sight 132, based on the headset's current 3-D location and orientation. The headset 130 then renders the holograms that fall within the headset's field of view. The user 120 is thus able to see virtual objects 114, 116, and 118 placed in their proper locations and orientations within the physical space 110.”; para. [0115], “Synthetic content may be supplied by users, by games, and/or by third-party software, for example. Sharing of content may go in any direction. For example, the second user 120-2 may design a virtual object and place that object in a particular location within the play zone, sharing that object back to other users via respective server apparatus.”; para. [0011], “In contrast with conventional VR, AR, and MR systems, an improved technique for rendering virtual content to a user stores map data of features in a physical environment of the user and measures the location of the user with stationary sensors placed at respective locations within the environment. A server provides the location of the user and portions of the map data to a headset worn by the user. The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data.”; para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0109], “Attention to this point has been focused on activities involving a single user 120. However, many applications, such as games, may involve multiple users. For example, multiple users may occupy the physical space 110 shown in FIG. 1, with each user's location tracked by the sensors 140 and server apparatus 150, i.e., in the same manner as described for the user 120. Each user may create a skeletal model 420, which may be rendered as an avatar 430 (FIG. 4). In some examples, each user in the physical space 110 sees all the other users as avatars, which may be superimposed over the user's physical bodies.”; para. [0110], “FIG. 10 shows another multi-user scenario. Here, a second user 120-2 is located within a second physical space 110-2, which is discontinuous and remote from the physical space 110 shown in FIG. 1 (hereinafter, the “first” physical space). The second physical space 110-2 is configured similarly to the first, and includes its own sensors 140a-2 through 140d-2 and server apparatus 150-2. The second physical space 110-2 may have a different shape from the first physical space 110. The “play zone” is an area designated for game play or other activity. Furniture 1020 and 1022 in the second physical space 110-2 may be moved out of the way to clear space for the play zone.”; para. [0123], “Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”; para. [0124], “FIG. 13 shows an example of a different play zone, which is laid out as a simple maze 1310. Here, the walls of the maze are set up as barriers, such that users must stay within the designated paths of the maze to avoid becoming separated from their avatars. FIG. 13 also shows various uses of virtual interconnects. For example, the system provides virtual interconnects 118a and 118b at maze decision points. A user 120 entering these virtual interconnects 118a and 118b may rotate the perspective, as described in connection with FIGS. 11A and 11B. The user may also specify a rate of movement. For example, the user may specify accelerated movement, such that each meter the user walks in physical space is translated to N meters of movement in virtual space—through the maze 1310.”; para. [0125], “In an example, virtual interconnects 118c and 118d specify a 1:1 scale movement, such that every meter the user walks in physical space is mapped 1:1 to a meter in virtual space. Virtual interconnects 118e and 118f may require the user 120 to enter a VR realm to cross the indicated distance. The VR realm may present particular challenges that the user must overcome to cross the indicated distance.”; para. [0126], “FIG. 14 shows an example spot 1410 that users may step into to enter a VR realm. A game or other software application may place such “VR Anywhere” spots 1410 at desired locations within a play zone. Once inside a VR Anywhere spot 1410, the user 120 is able perform VR commands to navigate a synthetic VR space, e.g., using the above-described physical movements mapped to control commands. The VR space is not required to have any relationship to the physical space in which the user is located. The VR Anywhere spot 1410 is preferably a small area, e.g., just large enough to safely accommodate the user 120 executing the predefined movements. In some examples, VR control commands are enabled only when the user 120 remains within the VR Anywhere spot 1410 and are disabled if the user steps out of the VR Anywhere spot 1410. In some examples, the system checks local map data 160 and locations of other users to ensure that no mapped objects are within some predetermined radius of the VR Anywhere spot 1410, to protect the user 120 (and other users) from harm that may result from sudden movements of the user's head, arms, legs, or torso.”; [0127], “FIG. 15 shows an example method 1500 of managing virtual content to be displayed to users via three-dimensional imaging headsets, such as AR and MR headsets. The method 1500 may be carried out, for example, by the server apparatus 150 described in connection with FIG. 2. The various acts of method 1500 may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in orders different from that illustrated, which may include performing some acts simultaneously.”)
Regarding claim 16, Gullicksen discloses The system of claim 2, wherein the system further comprises instructions for tracking physical objects using one or more than one motion tracking technology, and having the physical objects appear in the XR virtual environment with the correct features. (para. [0020], “The computer program product stores instructions which, when executed on control circuitry of a data storage system, cause the data storage system to perform a method of managing virtual content to be displayed to users via headsets, such as the method described above.“; para. [0013] “In examples where applications involve multiple users, the server may track locations of users in the same physical space and maintain a global perspective of all of them, such that the server maintains all user locations even when the user have no line of sight to one another.”; para. [0014], “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0015], “In some examples, the server may filter shared map information sent to other users to protect privacy. The server may also apply artificial content, e.g., coverings, colors, textures, etc., to provide simplified and/or stylized renderings of physical features in the play zone.”; para. [0041], “FIG. 1 shows an example environment 100 in which embodiments of the improved technique hereof can be practiced. Here, a user 120 is seen moving through a physical space 110. The physical space 110 includes rooms 110a and 110b and walls 112a and 112b. The user 120 wears a three-dimensional imaging headset 130, such as an AR (augmented reality) or MR (mixed reality) headset, which is capable of displaying holograms via display elements positioned in front of the user's eyes. The display elements in the headset 130 are transparent or translucent, such that the user 120 is able to look directly through the display elements to see the immediate surroundings. In the usual way, the headset 130 may project holograms, via the display elements, that appear fixed in relation to physical features in the physical space 110, such as floors, ceilings, walls, and other physical objects.”; para. [0043], “The server apparatus 150 specifies holograms (virtual objects) relative to the map data 160. For example, each hologram has a location, dimensions, and an orientation that is aligned with a coordinate system of the map data 160. Example holograms 114, 116, and 118 are shown. These holograms are not physical features but rather virtual features that the headset 130 may render such that they appear to be present in the indicated locations from the perspective of the user 120 through the headset 130. The server apparatus 150 may further specify holograms as artificial coverings (e.g., skins) on physical objects present in the physical space 110, such as on walls 112a and 112b. In some examples, virtual objects are constructed around physical objects. For example, hologram 114 may be shown as a garden whereas in fact it covers a physical object, such as a sofa.”; para. [0044], “In example operation, the server apparatus 150 receives a stream of inputs from the sensors 140 and generates measured locations of the user 120 as the user moves around in the physical space 110. In this example, the location of the user 120 is taken as the location of the headset 130 worn by the user 120. The server apparatus 150 sends the measured user location, information about holograms, and map data 160 (or portions thereof) to the headset 130. The headset 130 processes the user location and map data 160 to render the holograms. For example, the headset 130 applies hologram information and map data 160 to generate a virtual model of objects within its line of sight 132, based on the headset's current 3-D location and orientation. The headset 130 then renders the holograms that fall within the headset's field of view. The user 120 is thus able to see virtual objects 114, 116, and 118 placed in their proper locations and orientations within the physical space 110.”; para. [0050], “In some examples, the user operates a software program to define custom features, such as coverings, objects, walls, and the like, which the server apparatus 150 may locate relative to the map data 160 at user-defined locations. Also, as will be described in more detail below, input from other physical spaces may also be received and located relative to the map data 160, e.g., to enable the user 120 to visualize features in a distinct physical spaces in a manner that is aligned with the user's own physical space 110.”; para. [0113], “In addition to receiving shared map data 160a, the second server apparatus 150-2 also receives, over the WAN/Internet 250, a real-time representation of the avatar 430 of the first user 120, including its location and orientation relative to the play zone. With this arrangement, the second user 120-2 is able to see the avatar 430 of the first user 120 move within the second physical space 110-2 (movement shown by arrow 1010) in a manner that mirrors the movement of the first user 120 within the first physical space 110. In a like manner, the first user 120, within the first physical space 110, sees an avatar of the second user 120-2 displayed and moving in real time in synchronization with the second user 120-2. This arrangement is extendible to any number of users, with each user's headset rendering the holograms 114, 116, and 118 and the avatars of the other users, with the avatars moving in real time to reflect movements of the users they represent. Also, each user's headset, with the possible exception of the first user's (where it may not be necessary), renders a virtual rendition of the shared portions of the physical space 110. Owing to the global perspective afforded by the stationary sensors and the computerized apparatus in each physical space, the position and orientation of each avatar relative to the play zone matches the position and orientation of the respective physical user in that user's local play zone.)
Regarding claim 17, Gullicksen discloses The system of claim 2, wherein the one or more than one user can quickly switch content with avatars of inhabiting users, to effect a different XR virtual environment experience or scenario in the same physical room, within the same XR virtual environment platform, or imported from a different XR virtual environment platform, all in the original physical location, creating new XR content, an XR virtual environment, or scenario easily and quickly; wherein the new XR virtual environment, content, or scenario also includes an entire 3D dataset. (para. [0014] “In some examples, applications can span multiple physical spaces, with each physical space mapped and equipped with sensors. For example, a user can share the map of a play zone with other users in other locations. The other users' headsets may render the sharing user's play zone as a virtual space, and users can interact in the same play zone via avatars. For example, the sharing user sees the other users' avatars in the sharing user's own physical space, while each remote user sees the sharing user and the other remote users as avatars in a virtual space, which resembles the physical play zone.”; para. [0015], “In some examples, the server may filter shared map information sent to other users to protect privacy. The server may also apply artificial content, e.g., coverings, colors, textures, etc., to provide simplified and/or stylized renderings of physical features in the play zone.”; para. [0043], “The server apparatus 150 specifies holograms (virtual objects) relative to the map data 160. For example, each hologram has a location, dimensions, and an orientation that is aligned with a coordinate system of the map data 160. Example holograms 114, 116, and 118 are shown. These holograms are not physical features but rather virtual features that the headset 130 may render such that they appear to be present in the indicated locations from the perspective of the user 120 through the headset 130. The server apparatus 150 may further specify holograms as artificial coverings (e.g., skins) on physical objects present in the physical space 110, such as on walls 112a and 112b. In some examples, virtual objects are constructed around physical objects. For example, hologram 114 may be shown as a garden whereas in fact it covers a physical object, such as a sofa.”; para. [0008], “AR (augmented reality) allows users to continue to see their physical environments through display screens while additional content is superimposed. AR thus provides a vehicle for adding synthetic content to users' normal views of their environments. An example of AR technology is Google Glass.”; para. [0049], “In some examples, additional map data 160 may derive from a software application. For example, a game or other application may define particular rooms, realms, spaces, and other artificial content, which the server apparatus 150 receives and may integrate into the map data 160 at suitable locations and orientations relative to the physical space 110.”; para. [0057], “Using the WAN port, the switch/router 240 may connect to one or more public servers 260. These may include on-line stores (e.g., for buying games) and various servers to support vault-based communications. The switch/router 240 also supports communication over the WAN/Internet 250 with similarly-configured networks of other users, e.g., to support multi-player games or other applications across different local networks and locations.”; para. [0011], “In contrast with conventional VR, AR, and MR systems, an improved technique for rendering virtual content to a user stores map data of features in a physical environment of the user and measures the location of the user with stationary sensors placed at respective locations within the environment. A server provides the location of the user and portions of the map data to a headset worn by the user. The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data.”; para.[0018], “In some examples, the server may generate virtual content in the form of scenes, where a “scene” includes a collection of holograms and virtual interconnects. A “hologram” is a synthesized 3-D image, and a “virtual interconnect” is an identified space in the play zone that a user may enter to perform an action. Such actions may include, for example, rotating a virtual play zone (e.g., to accommodate a physical space inconsistent with the virtual play zone), teleporting to a different location in the virtual play zone, or entering a VR play area, e.g., an entirely synthetic VR realm.”
Regarding claim 18, Gullicksen discloses The system of claim 2, wherein the system further comprises real-time monitoring of game sessions and user interactions, with event logging (enter/leave), casting (render), session recording (maintain/resume avatar) and other functions. (para. [0012], “Advantageously, the improved technique benefits from a perspective of the user's environment that is more global than what could normally be achieved using an AR or MR headset alone, as such headsets are generally limited to local perspectives from vantage points of the headsets themselves. The global perspective provides advantages for many software applications, particularly for games. For example, a user can set up a play zone within the user's own house or apartment, with boundaries and features of the play zone mapped and stored on the server. An application may provide virtual content superimposed on physical features in the play zone, with the play zone potentially spanning multiple rooms, stories, and/or outdoor spaces, provided they are mapped and within range of the sensors.”; para. [0017], “In some examples, the wearable sensors and skeletal model allow the user to convey control commands by performing predetermined movements. For example, the server maps certain movements of the user's arms, legs, and/or torso to particular commands, e.g., for controlling an application, such as a game, which commands the server may execute upon detection. Because the server detects user movements based on input from the wearable sensors, there is no need for user movements to be within the field of view of the headset.”; para. [0041], “FIG. 1 shows an example environment 100 in which embodiments of the improved technique hereof can be practiced. Here, a user 120 is seen moving through a physical space 110. The physical space 110 includes rooms 110a and 110b and walls 112a and 112b. The user 120 wears a three-dimensional imaging headset 130, such as an AR (augmented reality) or MR (mixed reality) headset, which is capable of displaying holograms via display elements positioned in front of the user's eyes. The display elements in the headset 130 are transparent or translucent, such that the user 120 is able to look directly through the display elements to see the immediate surroundings. In the usual way, the headset 130 may project holograms, via the display elements, that appear fixed in relation to physical features in the physical space 110, such as floors, ceilings, walls, and other physical objects.”; para. [0075], “In addition, location and position of joints may also be enhanced by taking into account the user's view of the body as projected by the headset 130, e.g., by receiving user feedback. One should appreciate that the order of the above movements can be varied in any sensible way.; para. [0120], “FIGS. 12A and 12B show an example arrangement for encouraging users to stay within the play zone and to avoid moving to locations that are off-limits. For example, the second physical space 110-2 (FIG. 10) is larger than the play zone, such that the second user 120-2 is able to physically leave the play zone. During normal game play, this freedom may present an unfair advantage for the second user 120-2. At minimum, it is inconsistent with the wish to keep all players in the game.”; para. [0121], “As shown in FIG. 12A, the second user 120-2 in the second physical space 110-2 has left the play zone. For example, the second user 120-2 has crossed from a first side 1210 of the virtual wall 112a-V to a second side 1220. Here, when the second user 120-2 leaves the play zone, the avatar 430-2 of the second user 120-2 stays behind, i.e., on the first side 1210 of the virtual wall 112a-V. The avatar 430-2 may remain in this position until the second user 120-2 returns to the same location, at which point the avatar 430-2 resumes tracking the movements of the second user 120-2.”; para. [0122], “FIG. 12B shows the same situation from the viewpoint of the first user 120 in the first physical space 110. The first user 120 sees, through the headset 130, the avatar 430-2 of the second user 120-2, but the avatar 430-2 appears to be standing still—until the second user returns such that tracking can resume.”; para. “[0123] Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”; para. [0019], “Certain embodiments are directed to a method of managing virtual content to be displayed to users via three-dimensional imaging headsets. The method includes measuring locations of a user in a physical space as the user moves through the physical space, by a server apparatus receiving inputs from multiple stationary sensors positioned at respective sensor locations within the physical space and processing the inputs to generate the locations of the user. The method further includes storing map data that describes a map of the physical space and specifying a set of holograms that have apparent locations that are defined relative to the map data. The method still further includes providing the measured locations of the user and at least a portion of the map data to a headset worn by the user, to enable the headset to render the set of holograms at the apparent locations relative to the map data and from a user perspective based on the measured locations of the user.”)
Regarding claim 19, Gullicksen does not explicitly disclose The system of claim 2, wherein the system comprises a default automatic standard genZavesky profile, is generated for every new user, with prompts to customize the profile.
Zavesky more explicitly teaches The system of claim 2, wherein the system comprises a default automatic standard genZavesky profile, is generated for every new user, with prompts to customize the profile. (para. [0021], “One or more aspects of the subject disclosure include 2D and/or 3D generative models, e.g., 2D-GAN and/or 3D-GAN (generative adversarial network), for fast remodeling synthesis based on scanning of current room and design preference (e.g. styles). The GAN models and/or suggestions can follow, for example, constraints of existing house framework. In one example, mechanisms can allow interactive user modifications to be quickly observed and/or incorporated in the generation of a new design without costly remodeling, without human proposal and without drafting, etc.”; para. [0040], “Still referring to the additional discussion of the operation of the system according to an embodiment, the system can receive preference inputs from user (see, e.g., Profiles 280 of FIG. 2B). These profiles can be stored, for example, in a database. In one example, the system can utilize explicit feedback from description (e.g., semantic and vocal description of different components). In addition, a break down by object and location through identification can be implemented. In addition (or alternatively), the system can show “like” or preference for seen properties from geo tracking and/or from clicking on a browser. In addition, a user can optionally bootstrap preferences with gestures (e.g., relative sizes, etc.). In another embodiment, implicit feedback can be based on historical living and/or visited locations. In addition, social similarity (e.g., friends) and/or understanding of environment that you've taken pictures of can optionally be implemented. In addition, the system can optionally be used to locally move objects, visual appearance of room, etc. without actual 3D wall modifications. In addition, a user could optionally input a cost limit which can restrict generated model specification and/or revisions.”; para. [0041], “Still referring to the additional discussion of the operation of the system according to an embodiment, the system can receive input of a user's target environment to remodel. In one example, a model can either come from fixed CAD model or on-site SLAM (Simultaneous Localization and Mapping) for geometry scanning and object detection. In addition, multiple rooms can optionally be integrated for a more cohesive (whole home remodel) model.”; para. [0042], “Still referring to the additional discussion of the operation of the system according to an embodiment, the system can provide synthesis with a user's preferred remodel with interaction. In one example, a mechanism can enable a user to interact with gestures and/or speech to remodel the components and explicitly trigger different regeneration components. In addition, the more a user interacts with the system, the more the system can assume what components the user wants to run automatically (and/or have those components ingested into the profile). In addition, a user can use gesture plus XR (e.g., augmented reality and/or virtual reality) to understand which wall or limited set of geometry to change (e.g. virtual line drawing, rotation, zooming, gazing, etc.).”; para. [0043], “Still referring to the additional discussion of the operation of the system according to an embodiment, the system can provide a mechanism to swap in with different vendors' styles. For example, specific teams and preference for a vendor's contribution. In another example, competitive analysis can be performed of different price structures (e.g., good, better, best) to render appearance through vendor catalog.”; para. [0044], “Referring now to FIG. 2C, various steps of a method 2000 according to an embodiment are shown. As seen in this FIG. 2C, step 2002 comprises storing, in a database, a decorating style preference of a user. Next, step 2004 comprises receiving, from user equipment of the user, one or more images depicting an environment in which remodeling is desired. Next, step 2006 comprises generating, via a machine learning process, a first model to present by the user equipment, the generating the first model being based upon the decorating style preference and the one or more images, the first model comprising a first remodeling proposal for the environment. Next, step 2008 comprises sending, to the user equipment, the first model, the sending of the first model facilitating display by the equipment of a first depiction of the environment as proposed by the first remodeling proposal. Next, step 2010 comprises receiving, from the user equipment, feedback information regarding the first remodeling proposal. Next, step 2012 comprises generating, via the machine learning process, a second model to present by the user equipment, the generating the second model being based upon the decorating style preference, the one or more images, and the feedback information, the second model comprising a second remodeling proposal for the environment. Next, step 2014 comprises sending, to the user equipment, the second model, the sending of the second model facilitating display by the user equipment of a second depiction of the environment as proposed by the second remodeling proposal.”; para. [0064], “As described herein, various embodiments can perform “style transfer” (e.g., “style transfer” such as provided by certain conventional ML/GAN techniques). In one specific embodiment, the “style transfer” can be used to understand contents from a picture and push those styles (e.g., objects, colors, etc.) into a design generator engine.”; para. [0065], “As described herein, various embodiments can provide for a “digital supply chain”. For instance, during a remodel (or when the remodel is complete), a user may have designed custom furniture and/or structural components. These custom designed elements can be sent via digital supply chain to vendors for “on-demand” construction and/or delivery.”; para. [0066], “As described herein, various embodiments can instead of (or in addition to) learning how to remodel a target environment, learn preferences and/or geometrical limitations, and send this information to a vendor to bootstrap a new shopping experience (e.g., which door, window, lamp, TV is too large?).”; para. [0069], “As described herein, various embodiments can provide monetization for companies (e.g., furniture, engineering, and design companies) which could sponsor items within a remodel, create their own set of models (e.g., for a style), and/or quickly interact with users remotely (e.g., where one party is onsite and the other party is offsite making changes).”; para. [0070], “As described herein, various embodiments can provide a better user experience (e.g., with cost-controlled suggestions that are quickly available and/or with ML models trained from a wider set of styles or examples (instead of a single designer's input)).”; para. [0138], “In one or more embodiments, information regarding use of services can be generated including services being accessed, media consumption history, user preferences, and so forth. This information can be obtained by various methods including user input, detecting types of communications (e.g., video content vs. audio content), analysis of content streams, sampling, and so forth. The generating, obtaining and/or monitoring of this information can be responsive to an authorization provided by the user. In one or more embodiments, an analysis of data can be subject to authorization from user(s) associated with the data, such as an opt-in, an opt-out, acknowledgement requirements, notifications, selective authorization based on types of data, and so forth.”; para. [0140], “As will be readily appreciated, one or more of the embodiments can employ classifiers that are explicitly trained (e.g., via a gen Zavesky training data) as well as implicitly trained (e.g., via observing UE behavior, operator preferences, historical information, receiving extrinsic information). For example, SVMs can be configured via a learning or training phase within a classifier constructor and feature selection module. Thus, the classifier(s) can be used to automatically learn and perform a number of functions, including but not limited to determining according to predetermined criteria which of the acquired cell sites will benefit a maximum number of subscribers and/or which of the acquired cell sites will add minimum value to the existing communication network coverage, etc.”)
As both Gullicksen and Zavesky are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gullicksen to include wherein the system comprises a default automatic standard genZavesky profile, is generated for every new user, with prompts to customize the profile, in the context of spatial synchronization, according to the teaching of Zavesky, in order to enable new users to customize their profile based on a default automatic standard generic profile more efficiently without entering all required data.
Regarding claim 20, Gullicksen discloses The system of claim 19, (para. [0120], “FIGS. 12A and 12B show an example arrangement for encouraging users to stay within the play zone and to avoid moving to locations that are off-limits. For example, the second physical space 110-2 (FIG. 10) is larger than the play zone, such that the second user 120-2 is able to physically leave the play zone. During normal game play, this freedom may present an unfair advantage for the second user 120-2. At minimum, it is inconsistent with the wish to keep all players in the game.”; para. [0121], “As shown in FIG. 12A, the second user 120-2 in the second physical space 110-2 has left the play zone. For example, the second user 120-2 has crossed from a first side 1210 of the virtual wall 112a-V to a second side 1220. Here, when the second user 120-2 leaves the play zone, the avatar 430-2 of the second user 120-2 stays behind, i.e., on the first side 1210 of the virtual wall 112a-V. The avatar 430-2 may remain in this position until the second user 120-2 returns to the same location, at which point the avatar 430-2 resumes tracking the movements of the second user 120-2.”; para. [0122], “FIG. 12B shows the same situation from the viewpoint of the first user 120 in the first physical space 110. The first user 120 sees, through the headset 130, the avatar 430-2 of the second user 120-2, but the avatar 430-2 appears to be standing still—until the second user returns such that tracking can resume.”; para. [0123], “Similar behavior may apply when a user crosses any hologram that represents a barrier. For example, holograms may be used to represent barriers that users must stay within to participate in a game. Anytime a user crosses a barrier, such as a virtual wall, or otherwise steps out of bounds, the system may freeze that user's avatar, forcing the user to return to the place where the barrier was crossed. This behavior has the effect of enforcing virtual barriers to prevent cheating, e.g., by allowing users to take short cuts.”; para. [0113], “In addition to receiving shared map data 160a, the second server apparatus 150-2 also receives, over the WAN/Internet 250, a real-time representation of the avatar 430 of the first user 120, including its location and orientation relative to the play zone. With this arrangement, the second user 120-2 is able to see the avatar 430 of the first user 120 move within the second physical space 110-2 (movement shown by arrow 1010) in a manner that mirrors the movement of the first user 120 within the first physical space 110. In a like manner, the first user 120, within the first physical space 110, sees an avatar of the second user 120-2 displayed and moving in real time in synchronization with the second user 120-2. This arrangement is extendible to any number of users, with each user's headset rendering the holograms 114, 116, and 118 and the avatars of the other users, with the avatars moving in real time to reflect movements of the users they represent. Also, each user's headset, with the possible exception of the first user's (where it may not be necessary), renders a virtual rendition of the shared portions of the physical space 110. Owing to the global perspective afforded by the stationary sensors and the computerized apparatus in each physical space, the position and orientation of each avatar relative to the play zone matches the position and orientation of the respective physical user in that user's local play zone.”)
Gullicksen fails to disclose
Zavesky more explicitly teaches (customer’s ability to instruct design) and (para. [0021], “One or more aspects of the subject disclosure include 2D and/or 3D generative models, e.g., 2D-GAN and/or 3D-GAN (generative adversarial network), for fast remodeling synthesis based on scanning of current room and design preference (e.g. styles). The GAN models and/or suggestions can follow, for example, constraints of existing house framework. In one example, mechanisms can allow interactive user modifications to be quickly observed and/or incorporated in the generation of a new design without costly remodeling, without human proposal and without drafting, etc.”; para. [0040], “Still referring to the additional discussion of the operation of the system according to an embodiment, the system can receive preference inputs from user (see, e.g., Profiles 280 of FIG. 2B). These profiles can be stored, for example, in a database. In one example, the system can utilize explicit feedback from description (e.g., semantic and vocal description of different components). In addition, a break down by object and location through identification can be implemented. In addition (or alternatively), the system can show “like” or preference for seen properties from geo tracking and/or from clicking on a browser. In addition, a user can optionally bootstrap preferences with gestures (e.g., relative sizes, etc.). In another embodiment, implicit feedback can be based on historical living and/or visited locations. In addition, social similarity (e.g., friends) and/or understanding of environment that you've taken pictures of can optionally be implemented. In addition, the system can optionally be used to locally move objects, visual appearance of room, etc. without actual 3D wall modifications. In addition, a user could optionally input a cost limit which can restrict generated model specification and/or revisions.”; para. [0081], “As described herein, various embodiments provide mechanisms that can apply all style and demographic understanding by the analysis of other personal media (e.g., social media, user-generated content, personal web logs, stories, etc.) into the generation of home preferences.”; para. [0140], “As will be readily appreciated, one or more of the embodiments can employ classifiers that are explicitly trained (e.g., via a genZavesky training data) as well as implicitly trained (e.g., via observing UE behavior, operator preferences, historical information, receiving extrinsic information). For example, SVMs can be configured via a learning or training phase within a classifier constructor and feature selection module. Thus, the classifier(s) can be used to automatically learn and perform a number of functions, including but not limited to determining according to predetermined criteria which of the acquired cell sites will benefit a maximum number of subscribers and/or which of the acquired cell sites will add minimum value to the existing communication network coverage, etc.”; para. [0003], “Further, customers are often absent from the design process: both in the proposal of a specific style (the customer often cannot provide examples or specific instances) and in the modification of the plans proposed by a designer (the customer often cannot express simple movement of a wall (or spatial discomfort) with words alone to achieve the desired effect).”; para. [0004], “Further still, while certain conventional machine learning techniques could learn the styles of a user, conventional machine learning techniques are not typically incorporated in the design process and instead may recommend only high-level preferences, colors, etc.”; para. [0045], “In various examples, the decorating style preference of a user can comprise: one or more high-level textual/conceptual descriptions (e.g. pre-existing labels from a taxonomy of visual descriptors, such as “beach”, “mountain”, “forest”), one or more example photos, one or more pieces of signature furniture, one or more lighting pieces, one or more architecture elements, one or more specific features and/or characteristics (e.g. colors, furnishings, geometry, materials, etc.).”; para. [0056], “As described herein, various embodiments can provide for visiting (that is, viewing) a plan from an AR-constructed environment via use of a VR mechanism in an off-site location (e.g., someone locally first helps to design and interact with a physical environment, thereby creating an AR-environment; subsequently, the AR-environment (or model) is then conveyed to a VR mechanism (thereby allowing remote remodeling review).”; para. [0058], “As described herein, various embodiments can provide real-time interaction with one or more users for design instruction. In one example, this can be implemented using an understanding of 3D objects, remodeling steps, and constraints of the environment.”)
As both Gullicksen and Zavesky are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gullicksen to include wherein the customized profile is managed, accretes and incrementally auto-updates a log of the user’s behavior data from each return visit, using artificial intelligence and machine learning methods to create an incrementally refined model of the user, to incorporate in real-time dynamic XR experience creation for the user and others; wherein the artificial intelligence and machine learning sets are auto-adjusted to suit the user’s skill level and are synchronized across all users in the system, in the context of spatial synchronization, according to the teaching of Zavesky, in order to provide virtual environment by artificial intelligence based on users or customers (para. [0004] of Zavesky).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hyorim Park whose telephone number is (571)272-3859. The examiner can normally be reached Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Hyorim Park/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights