DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
2. The information disclosure statement (IDS) submitted on December 23, 2024 is considered by the examiner.
Drawings
3. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: Paragraph 108-110 refers to reference markers 210A and 210B in Figure 24 but they do not exist in Figure 24. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
4. The disclosure is objected to because of the following informalities:
Paragraph 147, the incorrect figure is referenced. Reference to Figure 18 should be to Figure 16 instead.
Paragraph 159 and 162, the incorrect figure is referenced. Reference to Figure 19 should be to Figure 17 instead. Markers 144, 146, and 148 are in Figure 17, not in Figure 19.
Paragraph 161 and 162, the incorrect figure is referenced. Reference to Figure 20 should be to Figure 18 instead.
Paragraph 175, the incorrect figure is referenced. Reference to Figure 21 should be to Figure 19 instead.
Paragraph 201, the incorrect figure is referenced. Reference to Figure 22 should be to Figure 20 instead.
Paragraph 202, the incorrect figure is referenced. Reference to Figure 23 should be to Figure 21 instead.
Paragraph 214, the incorrect figure is referenced. Reference to Figure 24 should be to Figure 22 instead.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
5. The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
6. Claim 26-35 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 26 and 35 claim "identifying a second pair of spatial elements having a spatial relationship corresponding to the spatial relationship of the first pair of spatial elements in a second physical space of the plurality of physical spaces". However, the Examiner cannot find support for identifying corresponding spatial relationships in the specification. The specification does not mention identifying a pair of spatial elements with a spatial relationship that corresponds to another spatial relationship in a second physical space. Instead, Paragraph 203 teaches identifying sematic mappings which correlates objects in one space to with objects in another space based on their semantic meaning, behavior, actions and interactions but not their spatial relationships. Thus, claim 26 is rejected.
Claims 27-34 are also rejected by dependency on claim 26. Claims will be examined as best understood.
Claim Rejections - 35 USC § 102
7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
9. Claim(s) 1-4, 6, 8-14, and 25 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ferrao et al. ("Environment-Aware Rendering and Interaction in Web-Based Augmented Reality" -- cited in IDS), hereinafter referred to as Ferrao.
10. Regarding claim 1, Ferrao teaches a method for generating XR content comprising (Abstract teaches a "novel framework for web-based environment-aware rendering and interaction in augment reality based on WebXR … solution allows for realistic rendering of 3D elements"): performing a spatial abstraction of a first physical space having a first physical arrangement by identifying and representing portions of the first physical arrangement as a first group of one or more digital components (Section 3.6 Paragraph 1 teaches creating "a simplified model of the real environment" which teaches a spatial abstraction. Section 3.6 Paragraphs 2-3 and Figure 12 teaches “the cannon.js rigid body physics engine was used. … a reconstruction of the real-world environment by projecting the depth data into a probabilistic voxel occupancy grid". An example of the process can be seen in Figure 13 which teaches the spatial abstraction is done in a physical space with a first physical arrangement. The voxel occupancy grid teaches representing a portion of the physical arrangement in a first physical space as digital components),
wherein the first group of one or more digital components is represented as code suitable for use by an XR generation system (XGS) to generate XR content for a plurality of physical spaces and a plurality of physical arrangements but that each include corresponding portions that may each be represented by the one or more digital components (Section 3.7 teaches "the solution presented is distributed openly as a library and focuses on providing an abstraction layer for web AR" and "Figure 14 presents the code required to add an object in a 3D scene". It also teaches "The 'ARApp' object provides access to all features … It checks for the existence of all WebXR features required in the device and manages the rendering of 3D objects and the physics simulation." The physics simulation teaches the portions of the physical space and arrangements are represented as digital components as disclosed in Section 3.6. The AR app teaches an XR generation system which generates XR content for any physical space. Thus, this teaches the physical space and its digital components are represented as code in the programming library to generate XR content; Page 8, Paragraph 3 teaches "combining real and virtual objects using the method proposed … for indoor and outdoor scenarios." This teaches using the XGS to generate XR content for a plurality of physical space and arrangements; Section 4, Paragraph 4 teaches running the method with different number of objects with teaches a plurality of physical arrangements);
with an XGS, generating XR content for the corresponding portions of at least one physical space (Section 3.7 teaches "the solution presented is distributed openly as a library and focuses on providing an abstraction layer for web AR" and "Figure 14 presents the code required to add an object in a 3D scene". It also teaches "The 'ARApp' object provides access to all features … It checks for the existence of all WebXR features required in the device and manages the rendering of 3D objects and the physics simulation." The AR app teaches an XR generation system which generations XR content for the corresponding portions of at least one physical space; Figure 13 discloses an example of a rendered ball which is the XR content generated for a corresponding portion of a physical space).
11. Regarding claim 2, Ferrao teaches the limitations of claim 1. Ferrao further teaches the method wherein the at least one physical space comprises the first physical space, such that the XR content is generated for the first physical space and the first physical arrangement (Page 3 Paragraph 1 and Figure 2 teach "The solution uses physically based shading, combined with environment information that results in the rendering of 3D objects that are consistent with the real-world surroundings as demonstrated in Figure 2"; Section 3.6 and Figure 13 discloses an example of a ball which is the XR content generated for a first physical space and first physical arrangement).
12. Regarding claim 3, Ferrao teaches the limitations of claim 1. Ferrao further teaches the method wherein the at least one physical space does not comprise the first physical space, such that the XR content is generated for a physical space and a physical arrangement other than the first physical space and the first physical arrangement (Page 8 Paragraph 3 teaches "combining real and virtual objects using the method proposed … for indoor and outdoor scenarios." This teaches using the XGS to generate XR content for a plurality of physical space and arrangements. The indoor and outdoor scenarios teach being able to generate XR content for a physical space and physical arrangement other than the first physical space and first physical arrangement).
13. Regarding claim 4, Ferrao teaches the limitations of claim 1. Ferrao further teaches the method further comprising providing the XR content to a first user (Abstract teaches the method of rendering the 3D objects or XR content provides "a fluid user experience". Thus, the XR content is provided to a first user).
14. Regarding claim 6, Ferrao teaches the limitations of claim 1. Ferrao further teaches the method wherein XR content is generated for at least a first physical space and a second and different physical space (Page 8 Paragraph 3 teaches "combining real and virtual objects using the method proposed … for indoor and outdoor scenarios." This teaches using the XGS to generate XR content for a plurality of physical space and arrangements. The indoor and outdoor scenarios teach being able to generate XR content for a first and second different physical space.).
15. Regarding claim 8, Ferrao teaches the limitations of claim 1. Ferrao further teaches the method further comprising providing one or more spatial modules that each govern an interaction with a digital component (Section 3.2 teaches "the object-picking module of the solution used for interaction with the 3D virtual environment from a 2D screen points using the 'hit-test' functionality" and "points obtained from the hit-test were stored internally by the API as environment anchors used to realign the scene … ensuring the objects placed in the scene were always correctly realigned". The object picking module teaches a spatial module governing an interaction with a digital component like ensuring objects remain placed in a scene; Section 3.3 teaches applying a spatial module of detecting geometry occlusion like in Figures 6 and 7 to govern the interactions of the XR content and the digital components; Section 3.6 teaches "For the physics simulation, the cannon.js rigid body physics engine was used" and "Figure 13 exemplifies the collisions between real and virtual objects". The physics simulation teaches a spatial module governing interactions with the digital components).
16. Regarding claim 9, Ferrao teaches the limitations of claim 8. Ferrao further teaches the method wherein the spatial modules comprise a library of pre-defined spatial modules that may each be selectively chosen and applied to each digital component when generating the XR content (Section 3.2 teaches the object-picking module also consists of sample measurement tools to evaluate distances, angles, and areas using the GUI toolkit. The sample measurement tools teaches a library of pre-defined spatial modules that can be applied to a digital component when generating XR content. Figure 3 shows an example of using the measurement tool on digital components in the physical space which also generates XR content; Section 3.6 teaches physics simulation and cannon.js being used and applied for the interactions between an XR content and real-world environment. Cannon.js is a library with more additional modules to define the physics and behavior when generating the XR content.
Page 17 Paragraph 2 also teaches with the physics simulation "the proposed simulation (Figure 25a) updated the simulation for all objects, updating our voxel-based model according to environment changes. After removing the cardboard box, the object fell to the ground." Thus, the cannon.js library and modules are applied to the digital components when generating the XR content).
17. Regarding claim 10, Ferrao teaches the limitations of claim 8. Ferrao further teaches the method wherein the XGS automatically selects and applies a spatial module to at least one digital component (Section 3.2 teaches the object-picking module that can be applied to at least one digital component as shown in Figure 3; Section 3.3 teaches automatically applying a spatial module of detecting geometry occlusion like in Figures 6 and 7 to govern the interactions of the XR content with the digital components; Section 3.6 teaches the physics simulation spatial module and "Figure 13 exemplifies the collisions between real and virtual objects". The physics simulation teaches a spatial module automatically applied to govern interactions with a digital component).
18. Regarding claim 11, Ferrao teaches the limitations of claim 8. Ferrao further teaches the method wherein at least one of the plurality of spatial modules governs a baseline physical function of the digital component (Section 3.6 teaches the physics simulation spatial module and "Figure 13 exemplifies the collisions between real and virtual objects". The collision and physics teaches a baseline physical function. The physics simulation teaches a spatial module governing the baseline physical function of a digital component; Section 3.3 teaches applying a spatial module of detecting geometry occlusion like in Figures 6 and 7 to govern the interactions of the XR content with the digital components. The geometry occlusion of a digital component in a scene teaches a baseline physical function; Page 17 Paragraph 2 also teaches with the physics simulation "the proposed simulation (Figure 25a) updated the simulation for all objects, updating our voxel-based model according to environment changes. After removing the cardboard box, the object fell to the ground." The act of falling after a box is removed teaches governing a baseline physical function of the digital component).
19. Regarding claim 12, Ferrao teaches the limitations of claim 8. Ferrao further teaches the method wherein at least one of the plurality of spatial modules governs a game mechanic function of the digital component (Section 3.2 teaches object picking can be combined with "dedicated controllers using the Gamepad API and motion-sensing-enabled controllers (e.g., Wii Remote) to further improve immersion". The Applicant does not define what a game mechanic function is within the claims so the object picking can be combined with gaming controllers to teach that the object picking spatial module governs a game mechanic function of the digital component).
20. Regarding claim 13, Ferrao teaches the limitations of claim 1. Ferrao further teaches the method further comprising using a scanner associated with the XGS to perform a three-dimensional scan of the first physical space prior to performing the spatial abstraction (Abstract teaches "can use monocular camera setups with depth data … or, when available, use higher-quality depth sensors (e.g. LIDAR, structured light) that provide a more accurate perception of the environment".), and wherein the spatial abstraction is performed on the three-dimensional scan (Section 3.6 Paragraph 1 teaches "Depth information was used to create a simplified model of the real environment" and Figure 12 teaches the camera projection and depth data collected from the sensors is done before performing the spatial abstraction. Thus, the spatial abstraction is performed on the 3D scan).
21. Regarding claim 14, Ferrao teaches the limitations of claim 1. Ferrao further teaches wherein the code comprises information related only to the one or more digital components and does not contain information related to a position or orientation of the first physical arrangement or a position or orientation of any other physical arrangement (Figure 14 teaches code required to add an object in a 3D scene does not contain information related to a position or orientation) such that the one or more digital components are usable for each of the plurality of physical spaces and plurality of physical arrangements (Page 8 Paragraph 3 teaches "combining real and virtual objects using the method proposed … for indoor and outdoor scenarios." This teaches using the XGS to generate XR content usable for a plurality of physical space and arrangements).
22. Regarding claim 25, Ferrao teaches a system for generating and providing XR content to a user comprising: an XR generation system (XGS) configured to (Abstract teaches a "novel framework for web-based environment-aware rendering and interaction in augment reality based on WebXR … solution allows for realistic rendering of 3D elements"): perform a spatial abstraction of a first physical space having a first physical arrangement by identifying and representing portions of the first physical arrangement as a first group of one or more digital components (Section 3.6 Paragraph 1 teaches creating "a simplified model of the real environment" which teaches a spatial abstraction. Section 3.6 Paragraphs 2-3 and Figure 12 teaches “the cannon.js rigid body physics engine was used. … a reconstruction of the real-world environment by projecting the depth data into a probabilistic voxel occupancy grid". An example of the process can be seen in Figure 13 which teaches the spatial abstraction is done in a physical space with a first physical arrangement. The voxel occupancy grid teaches representing a portion of the physical arrangement in a first physical space as digital components),
wherein the first group of one or more digital components is represented as code suitable for use by an XR generation system (XGS) to generate XR content for a plurality of physical spaces and a plurality of physical arrangements but that each include corresponding portions that may each be represented by the one or more digital components (Section 3.7 teaches "the solution presented is distributed openly as a library and focuses on providing an abstraction layer for web AR" and "Figure 14 presents the code required to add an object in a 3D scene". It also teaches "The 'ARApp' object provides access to all features … It checks for the existence of all WebXR features required in the device and manages the rendering of 3D objects and the physics simulation." The physics simulation teaches the portions of the physical space and arrangements are represented as digital components as disclosed in Section 3.6. The AR app teaches an XR generation system which generates XR content for any physical space. Thus, this teaches the physical space and its digital components are represented as code in the programming library to generate XR content; Page 8, Paragraph 3 teaches "combining real and virtual objects using the method proposed … for indoor and outdoor scenarios." This teaches using the XGS to generate XR content for a plurality of physical space and arrangements; Section 4, Paragraph 4 teaches running the method with different number of objects with teaches a plurality of physical arrangements);
using the code, generate XR content for the corresponding portions of at least one physical space (Section 3.7 teaches "the solution presented is distributed openly as a library and focuses on providing an abstraction layer for web AR" and "Figure 14 presents the code required to add an object in a 3D scene". It also teaches "The 'ARApp' object provides access to all features … It checks for the existence of all WebXR features required in the device and manages the rendering of 3D objects and the physics simulation." The AR app teaches an XR generation system which generations XR content for the corresponding portions of at least one physical space; Figure 13 discloses an example of a rendered ball which is the XR content generated for a corresponding portion of a physical space);
and an XR system configured to output the XR content to a user (Abstract teaches the method of rendering the 3D objects or XR content provides "a fluid user experience". Thus, the XR content is provided to a first user).
23. Claim(s) 26-29, 32-33, and 35 is/are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Agarawala et al. (U.S. Patent Application Publication No. 2019/0310761 A1 – cited in IDS), hereinafter Agarawala.
24. Regarding claim 26, Agarawala teaches a method for generating XR content comprising: providing a plurality of physical spaces (Abstract teaches being able to render AR content in a current and remote physical location which teaches a plurality of physical spaces), each with spatial elements having a known spatial relationship relative to one another including one or more of a position or orientation (Paragraph 17 teaches data or images having a relative position to each other or to the physical shape of the room. The data or images teach spatial elements with a spatial relationship relative to one another including a position; Paragraph 43 teaches “performing room scans including depth scans understanding the relative placement and images of physical objects to one another” which teaches spatial elements having a known spatial relationship relative to each other including their position.);
performing a spatial abstraction of a first physical space of the plurality of physical spaces by identifying and representing a first pair of the spatial elements, including the known spatial relationship, as a digital component (Paragraph 17 teaches data or images having a relative position to each other or to the physical shape of the room. The data or images teach spatial elements with a spatial relationship relative to one another including a position and are digital components since they are AR data objects; Paragraph 27 teaches “the AR system may map a grid or mesh to the various surfaces… a scan of a room may be performed”. Paragraph 43 teaches “performing room scans including depth scans understanding the relative placement and images of physical objects to one another”. Paragraph 232 teaches room scans can be saved and can include actual or approximated room or object dimensions. These paragraphs teach that the room scans perform a spatial abstraction of the space and representing surfaces or objects in the room as a mesh or grid. The mesh or grid represents the first pair of spatial elements as a digital component),
wherein the digital component comprises code suitable for use by an XR generation system (XGS) (Paragraph 27 teaches the digital components can be mesh for generating XR content by an AR system. The AR system teaches an XGS; Paragraph 35 teaches the AR system or XGS can read JSON or other metadata to arrange the XR content in the AR space. The JSON or other markup language teaches the digital component comprises code for use by an XGS; Paragraph 213-214 teaches using data structures and software to run the AR system. Thus, the digital component comprises code for the XGS) to generate XR content tailored for and representative of pairs of spatial elements in the plurality of physical spaces having a spatial relationship that corresponds to the spatial relationship of the first pair of spatial elements (Paragraph 8 teaches generating XR content that provides users “access to at least some of the same data, arranged relatively similarly (to the extent possible given their physical surroundings when used in AR mode)”. This teaches generating XR content tailored to the space which has pairs of spatial elements with spatial relationships as taught in Paragraph 232; Paragraph 55 teaches mapping AR display elements based on “relative location to each other” which teaches generating XR content for spatial elements like AR display elements with a spatial relationship and is rendered for viewing in the physical space),
including at least in a pair of physical spaces that have different physical arrangements (Abstract teaches being able to render AR content in a current and remote physical location which teaches a plurality of physical spaces; Paragraph 234 teaches aligning an AR environment to Room 2 which indicates it has a different physical alignment from Room 1 where the AR environment was initially saved in as taught in Paragraph 233. Thus, there are at least a pair of physical spaces with different physical arrangements);
identifying a second pair of spatial elements having a spatial relationship corresponding to the spatial relationship of the first pair of spatial elements in a second physical space of the plurality of physical spaces (Paragraph 55 teaches mapping display elements in one room to another room and displaying them based on relative locations. The relative locations of the display elements teaches a pair of spatial elements having a spatial relationship. The display elements shown in a current remote and another user's room teaches the spatial elements are a pair of first and second pair of spatial elements that have corresponding relationships in a first and second space),
with an XGS, generating XR content that is tailored for the second physical space of the plurality of physical spaces by using the digital component to represent the second pair of spatial elements (Figure 62 step 2610 and Paragraphs 258 teach opening an AR meeting space in a current physical location "wherein AR meeting space was previously configured for a remote physical location different from the current physical location". The remote physical location teaches a first physical space and the current physical location teaches a second physical space; Figure 62 step 6230 and Paragraphs 259-260 then teach setting an anchor area in the second physical space so that the spatial elements in the first physical space can be displayed relative to the anchor in the second physical space. The digital objects with relative locations to the anchor teach pairs of spatial elements with spatial relationships in a first and second physical space; Paragraph 261-262 and Figure 62 steps 6240 and 6250 teach rending the AR meeting space with the XR content of the spatial elements tailored for the second physical space).
25. Regarding claim 27, Agarawala teaches the limitations of claim 26. Agarawala further teaches the method wherein the second physical space comprises the first physical space, such that the XR content is generated for the first physical space and a first physical arrangement of the first physical space (Paragraph 53 teaches rendering display elements or XR content the same to multiple users co-located in the same room. The space the first user exists in can be considered a first physical space with a first physical arrangement. The space another user exists in can be considered a second physical space. In the event where the first and second user are co-located, this teaches the second space physical space comprises the first physical space and the XR content is generated the same; Paragraph 235 teaches Room 2 or the second physical space can be the same size as Room 1 and the XR content can be generated “without any size of appearance adjustments”. This teaches generating XR content for the first physical space and first physical arrangement for a second physical space).
26. Regarding claim 28, Agarawala teaches the limitations of claim 26. Agarawala further teaches the method wherein the second physical space does not comprise the first physical space, such that the XR content is generated for a physical space and a physical arrangement other than the first physical space and a first physical arrangement of the first physical space (Paragraph 18 teaches rendering AR data in a different physical room with a different physical arrangement that requires a room scan; Paragraph 233-234 teaches being able to open an AR meeting space in a new room, Room 2, which is different from Room 1. Room 1 teaches a first physical space and Room 2 teaches a second physical space. In Paragraph 234, it teaches adjusting the AR objects to align or fit with room 2 which teaches the second physical space has a physical arrangement other than the first physical space; Paragraph 237 teaches Room 2 is smaller than Room1 which teaches it has a different physical arrangement and space).
27. Regarding claim 29, Agarawala teaches the limitations of claim 26. Agarawala further teaches the method wherein XR content is generated for at least the pair of physical spaces that have different physical arrangements using the digital component (Paragraph 18 teaches rendering AR data in a different physical room with a different physical arrangement that requires a room scan. This teaches generating XR content for at least a pair of physical spaces with different physical arrangements since it teaches a new room and a previous room. The room scan creates a mesh as taught in paragraph 27 which can be considered the digital component which is used to generate the XR content; Paragraph 233-234 teaches being able to open an AR meeting space in a new room, Room 2, which is different from Room 1. Room 1 teaches a first physical space and Room 2 teaches a second physical space. In Paragraph 234, it teaches adjusting the AR objects to align or fit with room 2 which teaches the second physical space has a physical arrangement other than the first physical space; Paragraph 237 teaches Room 2 is smaller than Room1 which teaches it has a different physical arrangement and space).
28. Regarding claim 32, Agarawala teaches the limitations of claim 26. Agarawala further teaches the method further comprising: providing a two-part encoding that governs the XR content that is generated by the XGS for each of the plurality of physical spaces, the encoding formed by:
a first encoding half that is based on the first digital component and that lacks data related to any of the plurality of physical spaces (Paragraph 17-18 teaches saving the "relative positioning of data to other data". The saved information teaches a first encoding half based on the first digital component unrelated to the physical space; Paragraph 230 teaches saving relative alignment of digital canvases based on an anchor. This is information about a digital component not related to a physical space.);
and one or more second encoding halves that are each configured to pair with the first encoding half, and are each configured to store, retain, or allow retrieval of information about one of the plurality of physical spaces (Paragraph 17 teaches part of the saved information can also include "relative positioning of the data to the physical structure or shape of the room". This teaches a second part to the saved data or second encoding half that stores information about the physical space; Paragraph 232 teaches part of the saved meeting space includes saving a room scan of the room which has information about one of the plurality of physical spaces which pairs with the relative alignment information),
wherein, when the first encoding half is paired with one of the one or more second encoding halves to define an encoding pair, the XGS generates XR content that is determined based on the encoding pair (Paragraph 18 teaches the AR system or XGS renders the XR content based on this saved relative positioning information which includes the relative positioning to each other and the room These are the first and second halves; Paragraph 233 teaches the AR environment or XGS can generate XR content based on this saved meeting space including the relative alignment and room scan information).
29. Regarding claim 33, Agarawala teaches the limitations of claim 32. Agarawala further teaches the method wherein the encoding comprises at least one of: a base layer defining an abstracted geometry of one of the plurality of physical spaces; a dimensional layer for encoding one or more spatial relationships of the one physical space of the plurality of physical spaces; a semantic layer for providing semantic information of the one physical space of the plurality of physical spaces; or a mechanic layer for providing spatial mechanic information specifying at least one of materials or behavior of the one physical space of the plurality of physical spaces (Paragraph 17 teaches the saved session can include “relative positioning of data to other data” and "relative positioning of the data to the physical structure or shape of the room". The relative positionings teaches one or more spatial relationships. The Abstract teaches the method can be run on a current and remote physical location which teaches a plurality of physical spaces. Thus, this teaches a dimensional layer for encoding one or more spatial relationships of the one physical space of the plurality of physical spaces. The Applicant only requires the encoding to comprise of at least one of the layers so Agarawala teaching the dimensional layer teaches this claim).
30. Regarding claim 35, Agarawala teaches a system for generating and providing XR content to a user comprising: an XR generation system (XGS) configured to (Abstract teaches a system or XGS being able to render AR content):
perform a spatial abstraction of a first physical space of a plurality of physical spaces, each with spatial elements having a known spatial relationship relative to one another including one or more of a position or orientation (Paragraph 17 teaches data or images having a relative position to each other or to the physical shape of the room. The data or images teach spatial elements with a spatial relationship relative to one another including a position and are digital components since they are AR data objects; Paragraph 27 teaches “the AR system may map a grid or mesh to the various surfaces… a scan of a room may be performed”. Paragraph 43 teaches “performing room scans including depth scans understanding the relative placement and images of physical objects to one another”. Paragraph 232 teaches room scans can be saved and can include actual or approximated room or object dimensions. These paragraphs teach that the room scans perform a spatial abstraction of the space and representing surfaces or objects in the room as a mesh or grid; Paragraph 17 teaches data or images having a relative position to each other or to the physical shape of the room. The data or images teach spatial elements with a spatial relationship relative to one another including a position; Paragraph 43 teaches “performing room scans including depth scans understanding the relative placement and images of physical objects to one another” which teaches spatial elements having a known spatial relationship relative to each other including their position; Abstract teaches method can be performed in a current and remote physical location which teaches a plurality of physical spaces), by identifying and representing a first pair of the spatial elements, including the known spatial relationship, as a digital component (Paragraph 17 teaches data or images having a relative position to each other or to the physical shape of the room. The data or images teach spatial elements with a spatial relationship relative to one another including a position and are digital components since they are AR data objects; Paragraph 27 teaches “the AR system may map a grid or mesh to the various surfaces… a scan of a room may be performed”. Paragraph 43 teaches “performing room scans including depth scans understanding the relative placement and images of physical objects to one another”. Paragraph 232 teaches room scans can be saved and can include actual or approximated room or object dimensions. The mesh or grid represents the first pair of spatial elements as a digital component),
wherein the digital component comprises code suitable for use by an XR generation system (XGS) (Paragraph 27 teaches the digital components can be mesh for generating XR content by an AR system. The AR system teaches an XGS; Paragraph 35 teaches the AR system or XGS can read JSON or other metadata to arrange the XR content in the AR space. The JSON or other markup language teaches the digital component comprises code for use by an XGS; Paragraph 213-214 teaches using data structures and software to run the AR system. Thus, the digital component comprises code for the XGS) to generate XR content tailored for and representative of pairs of spatial elements in the plurality of physical spaces having a spatial relationship that corresponds to the spatial relationship of the first pair of spatial elements (Paragraph 8 teaches generating XR content that provides users “access to at least some of the same data, arranged relatively similarly (to the extent possible given their physical surroundings when used in AR mode)”. This teaches generating XR content tailored to the space which has pairs of spatial elements with spatial relationships as taught in Paragraph 232; Paragraph 55 teaches mapping AR display elements based on “relative location to each other” which teaches generating XR content for spatial elements like AR display elements with a spatial relationship and is rendered for viewing in the physical space),
including at least in a pair of physical spaces that have different physical arrangements (Abstract teaches being able to render AR content in a current and remote physical location which teaches a plurality of physical spaces; Paragraph 234 teaches aligning an AR environment to Room 2 which indicates it has a different physical alignment from Room 1 where the AR environment was initially saved in as taught in Paragraph 233. Thus, there are at least a pair of physical spaces with different physical arrangements);
identify a second pair of spatial elements having a spatial relationship corresponding to the spatial relationship of the first pair of spatial elements in a second physical space of the plurality of physical spaces (Paragraph 55 teaches mapping display elements in one room to another room and displaying them based on relative locations. The relative locations of the display elements teaches a pair of spatial elements having a spatial relationship. The display elements shown in a current remote and another user's room teaches the spatial elements are a pair of first and second pair of spatial elements that have corresponding relationships in a first and second space),
using the code (Paragraph 213-214 teaches using data structures and software to run the AR system. Thus, the digital component comprises code for the XGS and the XGS uses code to generate XR content), using the XGS to generate XR content that is tailored for the second physical space of the plurality of physical spaces by using the digital component to represent the second pair of spatial elements (Figure 62 step 2610 and Paragraphs 258 teach opening an AR meeting space in a current physical location "wherein AR meeting space was previously configured for a remote physical location different from the current physical location". The remote physical location teaches a first physical space and the current physical location teaches a second physical space; Figure 62 step 6230 and Paragraphs 259-260 then teach setting an anchor area in the second physical space so that the spatial elements in the first physical space can be displayed relative to the anchor in the second physical space. The digital objects with relative locations to the anchor teach pairs of spatial elements with spatial relationships in a first and second physical space; Paragraph 261-262 and Figure 62 steps 6240 and 6250 teach rending the AR meeting space with the XR content of the spatial elements tailored for the second physical space);
and an XR system configured to output the XR content to a user (Paragraph 6-8 teaches the AR system or XR system is used to augment meetings between users by displaying AR data. This teaches outputting XR content to a viewer).
Claim Rejections - 35 USC § 103
31. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
32. Claim(s) 5, 7, and 22-24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ferrao et al. ("Environment-Aware Rendering and Interaction in Web-Based Augmented Reality" -- cited in IDS), hereinafter Ferrao, as applied to claim 4, 6, and 1 above, and further in view of Agarawala et al. (U.S. Patent Application Publication No. 2019/0310761 A1 – cited in IDS), hereinafter Agarawala.
33. Regarding claim 5, Ferrao teaches the limitations of claim 4. However, Ferrao is not relied upon for the below claim language: the method further comprising providing the XR content to the first user and to a second user that are each located in a co-located at least one physical space.
Agarawala teaches the method further comprising providing the XR content to the first user and to a second user that are each located in a co-located at least one physical space (Paragraph 50 and Figure 9 teaches “multiple users who may be co-located in the same room … may see the various display elements”. This teaches providing XR content to a first and second user co-located in at least one physical space).
Ferrao and Agarawala are considered analogous to the claimed invention as because both are in the same field of generating extended reality content through analyzing the physical space. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the providing XR content to a first and second user in the same space taught by Agarawala in order to allow the users to share and view the same things to enable a collaborative environment (Agarawala Paragraph 50-53).
34. Regarding claim 7, Ferrao teaches the limitations of claim 6. However, Ferrao is not relied upon for the below claim language: the method further comprising providing the XR content to a first user located in the first physical space and a second user located in the second physical space.
Agarawala teaches the method further comprising providing the XR content to a first user located in the first physical space and a second user located in the second physical space (Paragraph 33 teaches a first user can access the XR content in their room “while another remote user accesses the room in a corresponding or mapped VR embodiment of the system.” This teaches displaying XR content to users in different locations. Thus, displaying the XR content to a first user and then a remote user teaches providing XR content to a first user in a first physical space and a second user located in a second physical space; Figure 63 and Paragraph 263 teaches providing XR content to a first and second user in different physical spaces).
Ferrao and Agarawala are considered analogous to the claimed invention as because both are in the same field of generating extended reality content through analyzing the physical space. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the providing XR content to a first and second user in different spaces taught by Agarawala in order to allow the users to share and view the same things to enable a collaborative environment (Agarawala Paragraph 54).
35. Regarding claim 22, Ferrao teaches the limitations of claim 1. However, Ferrao is not relied upon for the below claim language: the method further comprising: communicating data to a user located at a selected space of the at least one physical space; modifying the data or communication of the data communicated to the user based on the selected space.
Agarawala teaches the method further comprising: communicating data to a user located at a selected space of the at least one physical space; modifying the data or communication of the data communicated to the user based on the selected space (Paragraph 55 teaches “a user may be able to select one of the workspaces … the AR system may map the elements as close as possible based on relative location around the room, relative location to each other, and/or relative location to one or more users”. This teaches communicating the workspace data to a user in their room which is their selected room and modifying the communication of the workspace data based on their room; Paragraph 233 teaches “a user requests to open an AR meeting space in a new room” and Paragraph 234 teaches the AR environment may adjust the digital canvases to align or fit within a new Room 2. "This adjusted alignment of the saved meeting space … may be transmit to one or more AR devices 6103 (for different users 6102)" Thus, the digital canvases teaches data communicated to a user which is modified based on the selected space).
Ferrao and Agarawala are considered analogous to the claimed invention as because both are in the same field of generating extended reality content through analyzing the physical space. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the modifying data or communication of data based on the selected space taught by Agarawala in order to augment meetings between users and provide users with access to the same data, arranged relatively similarly, regardless of their physical location (Agarawala Paragraph 8).
36. Regarding claim 23, Ferrao in view of Agarawala teaches the limitations of claim 22. Ferrao further teaches the method wherein the data is the XR content generated by the XGS (Section 3.7 teaches "the solution presented is distributed openly as a library and focuses on providing an abstraction layer for web AR" and "Figure 14 presents the code required to add an object in a 3D scene". It also teaches "The 'ARApp' object provides access to all features … It checks for the existence of all WebXR features required in the device and manages the rendering of 3D objects and the physics simulation." The AR app teaches an XR generation system which generations XR content for the corresponding portions of at least one physical space).
37. Regarding claim 24, Ferrao in view of Agarawala teaches the limitations of claim 22. Ferrao is not relied upon for the below claim language: the method wherein the data or communication of the data is dynamically modified based on the selected space in real time.
Agarawala teaches the method wherein the data or communication of the data is dynamically modified based on the selected space in real time (Paragraph 234 teaches the AR environment may adjust the digital canvases to align or fit within a new Room 2. "This adjusted alignment of the saved meeting space … may be transmit to one or more AR devices 6103 (for different users 6102)" Thus, the digital canvases teaches data communicated to a user which is modified based on the selected space; Paragraph 235 teaches the modified data is dynamically modified and shown to a user's display in real-time; Paragraph 334 teaches that AR meeting spaces can update in real-time as changes to the space occur).
Ferrao and Agarawala are considered analogous to the claimed invention as because both are in the same field of generating extended reality content through analyzing the physical space. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the modifying data or communication of data based on the selected space taught by Agarawala in order to augment meetings between users and provide users with access to the same data, arranged relatively similarly, regardless of their physical location (Agarawala Paragraph 8).
38. Claim(s) 15-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ferrao et al. ("Environment-Aware Rendering and Interaction in Web-Based Augmented Reality" -- cited in IDS), hereinafter Ferrao, as applied to claim 1 above, and further in view of Andon et al. (U.S. Patent Application Publication No. 2021/0120054 A1 – cited in IDS), hereinafter Andon.
39. Regarding claim 15, Ferrao teaches the limitations of claim 1. However, Ferrao is not relied upon for the below claim language: the method further comprising: providing a plug comprising an encoding that is configured to store, retain, or allow retrieval of information; and providing one or more receptacles that are each configured to accept the plug and that are each comprised of an interaction layer that governs an interaction between (1) the at least one physical space, a user, or a first portion of the XR content and (2) a second portion of the XR content.
Andon teaches the method further comprising: providing a plug comprising an encoding that is configured to store, retain, or allow retrieval of information (Abstract teaches being able to add or remove functionality through visible layers. These visible layers teach plugs encoding information; Paragraph 105 teaches "a new visual layer (also referred to herein as a 'dimensional layer') that can be 'plugged in' to any session". Paragraph 106 also teaches "These dimensional layers may contain both the information and methods to enable anyone who engages with the system to experience, and optionally interact with, the content of each layer". The dimensional layer teaches a plug with an encoding storing information on the content); and providing one or more receptacles that are each configured to accept the plug and that are each comprised of an interaction layer that governs an interaction between (1) the at least one physical space, a user, or a first portion of the XR content and (2) a second portion of the XR content (Paragraph 105 teaches the ESVIE receives 3rd party application visual layers or plugs. ESVIE teaches a receptacle that accepts the plug; Paragraph 10 teaches “ESVIE system allows actual physical objects to be represented in a shared visual interaction environment … with users participating in the shared visual interaction environment allowed … to control the actual physical objects via interactions with their virtual avatars or other visual representations, optionally in accordance with access permissions or other restrictions”. The visual representation of the physical object can be considered the second portion of the XR content. This teaches that the receptacle ESVIE has an interaction layer governing interactions between a user and a second portion of the XR content; Paragraph 83 teaches the device with the ESVIE system can be a "spatially aware device" where "augmented objects exist in real-world geo-spatial coordinates and are shared with session participants". It teaches the receptacle ESVIE can enable the augmented objects to be interacted with "and have agent-based functionality. Interaction with the augmented objects is dictated by their unique behavior. For example, an object may be selectable and, upon selection, invoke a particular behavior that is experienced by all session participants". Thus, this teaches the receptacle, ESVIE, has an interaction layer providing the spatial mechanic information between the digital components (augmented objects) in the dimensional layer).
Ferrao and Andon are considered analogous to the claimed invention as because both are in the same field of generating extended reality content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the plug and receptacle taught by Andon in order to easily load in 3rd party applications and enable an augmented multisensory world (Andon Paragraphs 105-106).
40. Regarding claim 16, Ferrao teaches the limitations of claim 15. However, Ferrao is not relied upon for the below claim language: the method wherein the encoding comprises at least one of: a base layer defining an abstracted geometry of the at least one physical space and represented by the one or more digital components; a dimensional layer for encoding spatial relationships of the one or more digital components of the encoding; a semantic layer for providing semantic information of the one or more digital components of the encoding; or a mechanic layer for providing spatial mechanic information specifying at least one of materials or behavior of the one or more digital components of the encoding.
Andon teaches the method wherein the encoding comprises at least one of: a base layer defining an abstracted geometry of the at least one physical space and represented by the one or more digital components; a dimensional layer for encoding spatial relationships of the one or more digital components of the encoding; a semantic layer for providing semantic information of the one or more digital components of the encoding; or a mechanic layer for providing spatial mechanic information specifying at least one of materials or behavior of the one or more digital components of the encoding (Paragraph 8 teaches the visible layers or plugs can include “one or more virtual objects” and also “provide or remove access to a predefined type of functionality via the visual elements”. This teaches a mechanic layer within the plug encoding; Paragraph 60 teach "dimensional layers represent a collection of information … and methods that allow a participant to interact with the information" and Paragraph 61 teaches " dimensional layers … containing multiple virtual objects that the participant can interact with. Each dimensional layer can have its own associated toolset to allow the participant to … interact with the information and features of, that individual layer". This teaches a mechanic layer providing spatial mechanic information on the behavior of digital components; Paragraph 46 teaches the virtual objects can correspond to a real-world physical object and that the virtual object is able to interact with other virtual objects or other corresponding real-world objects. The virtual objects can teach digital components. Paragraph 46 also teaches that the " virtual object may be instantiated in a peer session dimensional layer, and optionally provide one or more of the following …. to be aware of their geo-temporal location and/or their location relative to a user; to sense the presence of users and/or other virtual objects relative to their location". The location of the virtual object relative to their location teaches a dimensional layer that encodes spatial relationships of the digital component. The interaction with other virtual objects teaches a mechanic layer specifying the behavior of the digital component. Thus, the layers teach plugs with encodings with a dimensional and mechanic layer. The Applicant only specifies “at least one of” the layers need to be taught so the base layer and semantic layer do not need to be taught).
Ferrao and Andon are considered analogous to the claimed invention as because both are in the same field of generating extended reality content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the plug and layers in its encoding taught by Andon in order to easily load in 3rd party applications and enable an augmented multisensory world (Andon Paragraphs 105-106).
41. Regarding claim 17, Ferrao teaches the limitations of claim 16. However, Ferrao is not relied upon for the below claim language: the method wherein the mechanic layer of the plug fully matches the mechanic layer of the one or more receptacles such that an entirety of the at least one of materials or behavior specified by the mechanic layer of the one or more receptacles is accessible to the plug.
Andon teaches the method wherein the mechanic layer of the plug fully matches the mechanic layer of the one or more receptacles such that an entirety of the at least one of materials or behavior specified by the mechanic layer of the one or more receptacles is accessible to the plug (Paragraph 78 teaches dimensional layers can be added to the ESVIE system and that "object methods for the dimensional layers are aware of data types communicated over the continuum and are capable of rendering that particular information type" and "interaction mechanisms are aware of the data types for that dimensional layer and the interaction mechanisms are specific for interacting with the appropriate data type". As taught in Paragraph 46, the dimensional layer plug also has a mechanic layer. The ESVIE system teaches a receptacle that can be connected to multiple dimensional layers or plugs through certain data types. The ESVIE system is a receptacle where it is able to receive the dimensional layer fully and understands the data types communicated. Since the ESVIE system understands the data type, this teaches that the mechanic layer of the receptacle matches the mechanic layer of the plug. Thus, the receptacle layer’s mechanic layer matches the mechanic layer such that the behaviors is accessible to the plug; Paragraph 94 and 104 teaches ESVIE is a receptacle that can receive plug-ins).
Ferrao and Andon are considered analogous to the claimed invention as because both are in the same field of generating extended reality content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the receptacle taught by Andon in order to easily load in 3rd party applications and enable an augmented multisensory world (Andon Paragraphs 105-106).
42. Regarding claim 18, Ferrao teaches the limitations of claim 16. However, Ferrao is not relied upon for the below claim language: the method wherein the mechanic layer of the plug partially matches the mechanic layer of the one or more receptacles such that less than an entirety of the at least one of materials or behavior specified by the mechanic layer of the one or more receptacles is accessible to the plug; or the mechanic layer of the one or more receptacles partially matches the mechanic layer of the plug such that less than an entirety of the at least one of materials or behavior specified by the mechanic layer of the plug is accessible to the one or more receptacles.
Andon teaches the method wherein the mechanic layer of the plug partially matches the mechanic layer of the one or more receptacles such that less than an entirety of the at least one of materials or behavior specified by the mechanic layer of the one or more receptacles is accessible to the plug; or the mechanic layer of the one or more receptacles partially matches the mechanic layer of the plug such that less than an entirety of the at least one of materials or behavior specified by the mechanic layer of the plug is accessible to the one or more receptacles (Paragraph 74 teaches "access to specific dimensional layers can be granted or limited based on permission". If the permissions for the mechanic layer of the plug do not match the receptacle or system, then the functionality will be limited).
Ferrao and Andon are considered analogous to the claimed invention as because both are in the same field of generating extended reality content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the plug partially matching the receptacle in order to still load in 3rd party applications and enable an augmented multisensory world (Andon Paragraphs 105-106).
43. Regarding claim 19, Ferrao teaches the limitations of claim 15. However, Ferrao is not relied upon for the below claim language: the method wherein at least one of the one or more receptacles comprises a mechanic layer for providing spatial mechanic information specifying at least one of materials or behavior of the one or more digital components of the encoding.
Andon teaches the method wherein at least one of the one or more receptacles comprises a mechanic layer for providing spatial mechanic information specifying at least one of materials or behavior of the one or more digital components of the encoding (Paragraph 94 and 104 teaches ESVIE is a receptacle that can receive plug-ins; Paragraph 83 teaches the device with the ESVIE system can be a "spatially aware device" where "augmented objects exist in real-world geo-spatial coordinates and are shared with session participants". It teaches the receptacle ESVIE can enable the augmented objects to be interacted with "and have agent-based functionality. Interaction with the augmented objects is dictated by their unique behavior. For example, an object may be selectable and, upon selection, invoke a particular behavior that is experienced by all session participants". Thus, this teaches the receptacle, ESVIE, has a mechanic layer providing the spatial mechanic information between the digital components (augmented objects) in the dimensional layer).
Ferrao and Andon are considered analogous to the claimed invention as because both are in the same field of generating extended reality content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the receptacle taught by Andon in order to allow participants to interact with and select objects (Andon Paragraphs 105-106).
44. Regarding claim 20, Ferrao teaches the limitations of claim 1. However, Ferrao is not relied upon for the below claim language: the method further comprising providing an interpreter configured to convert the one or more digital components to XR content suitable tailored for the at least one physical space.
Andon teaches the method further comprising providing an interpreter configured to convert the one or more digital components to XR content suitable tailored for the at least one physical space (Paragraph 10 teaches "ESVIE system allows actual physical objects to be represented in a shared visual interaction environment … with one or more such actual physical objects each having an associated visual layer for its visual representation… in this manner, physical objects may be introduced to and be interacted with in a shared visual interaction environment". The visual representation in a visual layer teaches a digital component for the actual physical object and also teaches converting the digital component to XR content by enabling it to be interacted with in a shared visual interaction environment since virtual avatars are able to interact with it. Thus, the ESVIE system teaches an interpreter able to convert digital components to XR content; Paragraph 46 teaches a virtual/augmented object can be implemented that represents a real-world object so that it has a "visual appearance in a layer and can provide 'intelligent' actions by interaction with other virtual objects". It also teaches that it controls "actions of that real-world object in the actual physical environment". This teaches converting the digital component or real-world object into XR content tailored for the at least one physical space.).
Ferrao and Andon are considered analogous to the claimed invention as because both are in the same field of generating extended reality content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the interpreter taught by Andon in order to create an interactive communication session that is immersive and interactive with shared situational awareness (Andon Paragraph 47).
45. Regarding claim 21, Ferrao teaches the limitations of claim 20. However, Ferrao is not relied upon for the below claim language: the method further comprising: with the interpreter, receiving a space-as-code (SAC) instruction for creating the XR content; in generating the XR content and in response to the SAC instruction, using to the interpreter to translate the one or more digital components to corresponding physical equivalents; and using the interpreter, identifying the corresponding physical equivalents in the at least one physical space and then generating the XR content based on the identified corresponding physical equivalents.
Andon teaches the method further comprising: with the interpreter, receiving a space-as-code (SAC) instruction for creating the XR content (Paragraph 17 teaches execution of the ESVIE system can create and manage sessions involving virtual objects that can correspond to real world objects. The claim does not define SAC instruction so it can be broadly interpreted as anything that creates XR content. ESVIE creating the session and virtual objects teaches receiving a SAC instruction for creating XR content; Paragraph 10 teaches "ESVIE system allows actual physical objects to be represented in a shared visual interaction environment … with one or more such actual physical objects each having an associated visual layer for its visual representation… in this manner, physical objects may be introduced to and be interacted with in a shared visual interaction environment". The visual representation in a visual layer teaches a digital component for the actual physical object and also teaches converting the digital component to XR content by enabling it to be interacted with in a shared visual interaction environment since virtual avatars are able to interact with it. The instruction to convert the physical object to have an XR equivalent teaches receiving a SAC instruction to create XR content);
in generating the XR content and in response to the SAC instruction, using to the interpreter to translate the one or more digital components to corresponding physical equivalents (Paragraph 20 teaches information can be provided that allows "information about types of allowed manipulations of the virtual object (e.g., to correspond to possible changes that may be performed to a corresponding physical object". This teaches the interpreter in ESVIE translates the virtual object to its physical equivalent);
and using the interpreter, identifying the corresponding physical equivalents in the at least one physical space and then generating the XR content based on the identified corresponding physical equivalents (Paragraph 46 teaches the virtual object has a corresponding real-world physical object and can generate interactions with the virtual object which corresponds to interactions with the corresponding real-world object. This teaches generating XR content based on the corresponding physical equivalent).
Ferrao and Andon are considered analogous to the claimed invention as because both are in the same field of generating extended reality content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Ferrao with the interpreter and SAC instruction taught by Andon in order to create an interactive communication session that is immersive and interactive with shared situational awareness (Andon Paragraph 47).
46. Claim(s) 30-31 is/are rejected under 35 U.S.C. 103 as being unpatentable over Agarawala et al. (U.S. Patent Application Publication No. 2019/0310761 A1 – cited in IDS), hereinafter Agarawala, as applied to claim 26 above, and further in view of Ferrao et al. ("Environment-Aware Rendering and Interaction in Web-Based Augmented Reality" -- cited in IDS), hereinafter Ferrao.
47. Regarding claim 30, Agarawala teaches the limitations of claim 26. Agarawala further teaches the method wherein the digital component comprises one or more digital components that are each associated with a pair of spatial elements having a known spatial relationship relative to one another including one or more of a position and orientation (Figure 62 step 6230 and Paragraphs 259-260 teach setting an anchor area in the second physical space so that the spatial elements in the remote first physical space can be displayed relative to the anchor in the second physical space. The digital objects with relative locations to the anchor teach pairs of spatial elements with spatial relationships relative to one another including position. Orientation is not required to be taught since the applicant uses "one or more"),
However, Agarawala is not relied upon for the below claim language: the method further comprising providing one or more spatial modules that are each reusable and that each govern an interaction between the XR content generated by the XGS and one of the one or more digital components.
Ferrao teaches the method further comprising providing one or more spatial modules that are each reusable and that each govern an interaction between the XR content generated by the XGS and one of the one or more digital components (Section 3.2 teaches "the object-picking module of the solution used for interaction with the 3D virtual environment from a 2D screen points using the 'hit-test' functionality" and "points obtained from the hit-test were stored internally by the API as environment anchors used to realign the scene … ensuring the objects placed in the scene were always correctly realigned". The object picking module which can be used for multiple objects teaches a reusable spatial module governing an interaction with a digital component like ensuring objects remain placed in a scene; Section 3.3 teaches applying a spatial module of detecting geometry occlusion like in Figures 6 and 7 to govern the interactions of the XR content with the digital components; Section 3.6 teaches "For the physics simulation, the cannon.js rigid body physics engine was used" and "Figure 13 exemplifies the collisions between real and virtual objects". The physics simulation teaches a reusable spatial module governing interactions with a digital component and XR content generated by an XGS).
Agarawala and Ferrao are considered analogous to the claimed invention as because both are in the same field of generating XR content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Agarawala with the reusable spatial modules taught by Ferrao in order to allow for a realistic rendering of 3D elements that can reflect changes in the environment (Ferrao Abstract and Section 6 Paragraph 1).
48. Regarding claim 31, Agarawala teaches the limitations of claim 26. However, Agarawala is not relied upon for the below claim language: the method wherein the code comprises information related only to the digital component and does not contain information related to a position or orientation of the first physical arrangement or a position or orientation of any other physical arrangement such that the digital component is usable for each of the plurality of physical spaces and plurality of physical arrangements.
Ferrao teaches the method wherein the code comprises information related only to the digital component and does not contain information related to a position or orientation of the first physical arrangement or a position or orientation of any other physical arrangement (Figure 14 teaches code required to add an object in a 3D scene does not contain information related to a position or orientation) such that the digital component is usable for each of the plurality of physical spaces and plurality of physical arrangements (Page 8 Paragraph 3 teaches "combining real and virtual objects using the method proposed … for indoor and outdoor scenarios." This teaches using the XGS to generate XR content usable for a plurality of physical space and arrangements).
Agarawala and Ferrao are considered analogous to the claimed invention as because both are in the same field of generating XR content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Agarawala with the code for the digital component taught by Ferrao in order to allow for a realistic rendering of 3D elements that can reflect changes in the environment (Ferrao Abstract and Section 6 Paragraph 1).
49. Claim(s) 34 is/are rejected under 35 U.S.C. 103 as being unpatentable over Agarawala et al. (U.S. Patent Application Publication No. 2019/0310761 A1 – cited in IDS), hereinafter Agarawala, as applied to claim 26 above, and further in view of Murphy et al. (U.S. Patent Application Publication No. 2020/0201514 A1), hereinafter Murphy.
Regarding claim 34, Agarawala teaches the limitations of claim 26. However, Agarawala is not relied upon for the below claim language: the method wherein each of the pairs of spatial elements have a semantic relationship relative to one another, the method further comprising performing a spatial abstraction on the first physical space of the plurality of physical spaces by identifying and representing the pair of spatial elements of the first physical space, including the known spatial relationship and the known semantic relationship, as the first digital component.
Murphy teaches the method wherein each of the pairs of spatial elements have a semantic relationship relative to one another (Paragraph 35 teaches UI content can have a semantic relationship relative to one another; Figure 2B and Paragraph 61 teaches a semantic relationship between spatial elements like the Painting Scene Object 222 and Information Surface 226),
the method further comprising performing a spatial abstraction on the first physical space of the plurality of physical spaces by identifying and representing the pair of spatial elements of the first physical space, including the known spatial relationship and the known semantic relationship, as the first digital component (Paragraph 5 teaches “data defining an augmented reality (AR) environment including a representation of a physical environment, identifying … relationships between a plurality of scene elements”. The representation of the physical environment teaches a spatial abstraction. Paragraph 35-39 teaches the semantic relationship provides information on the relative positioning of each spatial element to each other like “adjacent, on top of, coplanar”. This teaches a semantic and spatial relationship and is represented through a graph as seen in Figure 2B. Paragraph 37 then teaches the scene is represented as a semantic graph which teaches a spatial abstraction and a digital component).
Agarawala and Murphy are considered analogous to the claimed invention as because both are in the same field of generating extended reality content. Thus, it would have been obvious to a person holding ordinary skill in the art before the effective filing date to modify the method of generating XR content taught by Agarawala with the semantic relationships taught by Murphy in order to provide dynamic AR experiences across multiple platforms and devices (Murphy Paragraph 33) and allow the AR layout of objects to adapt to different scene conditions (Murphy Paragraph 106).
Conclusion
50. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Barnes et al. (U.S. Patent Application Publication No. 2021/0225727 A1) teaches semantic relationships between objects to generate an augmented reality.
Dolbier et al. (U.S. Patent Application Publication No. 20100020100 A1) teaches the virtual world consists of multiple different layers that can store different virtual objects.
Collart (U.S. Patent Application Publication No. 2020/0104522 A1) teaches rendering 3D objects in 3D spaces and creating a semantic map.
51. Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTINE Y AHN whose telephone number is (571)272-0672. The examiner can normally be reached M-F 9-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571)272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTINE YERA AHN/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights