Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 11-18 recite “the meta-space generation method of claim…” Since claims 11-18 depend from claim 10, which is directed to a meta-space generation apparatus, these claims should be amended to recite “the meta-space generation apparatus of claim…”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-10 and 12-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Keshavarzi et al. (Optimization and Manipulation of Contextual Mutual Spaces for Multi-User Virtual and Augmented Reality Interaction, 2020), in view of Schmalstieg et al. (Pub. No. US 20160179830), and further in view of Azmandian (Pub. No. US 2024/0108979).
Regarding claim 1, Keshavarzi discloses a method of generating a shared virtual space for multi-user interaction in which the remote users’ room spaces differ in layout and size (see Keshavarzi, Abstract; §1). Keshavarzi discloses generating a user space having a three-dimensional (3D) virtual space form based on a user space image captured through a user terminal, in that each user’s room is scanned, including an initial scan by a HoloLens head-mounted display and a scan with a Matterport camera, and the captured point cloud or mesh data is converted into a 3D model of labeled objects with bounding boxes (see Keshavarzi, §3.1; §6). Keshavarzi further discloses analyzing a structural feature of the user space in consideration of a background and objects that constitute the user space, in that semantic segmentation identifies the room enclosure (floors, walls, ceilings) and the objects within it, and a topological scene graph records the relationships and distances among the objects and between the objects and the room boundary (see Keshavarzi, §3; §3.1; §3.2).
Keshavarzi also discloses searching for a mutually shareable space that satisfies alignment conditions in which the structural feature is taken into consideration, in that standable and sittable spaces are calculated from the segmented objects and scene graph (see Keshavarzi, §3.3.1–3.3.2, Eqs. 1–5) and a genetic search finds the rigid-body translation and rotation to be applied to each room so that the intersection of the rooms’ available spaces is maximized (see Keshavarzi, §3.3.3, Eqs. 6–7; §4). Keshavarzi further discloses generating a mutually shareable space by aligning the rooms to that maximal mutual space and presenting the resulting mutual boundary to the users in augmented reality (see Keshavarzi, §3.3.3; §6; FIG. 6).
However, Keshavarzi does not disclose that the search is performed based on user space data of other users registered in a user space database.
In the same field of endeavor, Schmalstieg discloses a server that receives keyframes and associated metadata, including a user identification, from a plurality of client devices, stores the received keyframes in a 3D map database, and tags the stored keyframes in the 3D map database according to the received metadata (see Schmalstieg, FIG. 5, blocks 506, 511, 516, and accompanying description). Schmalstieg further discloses receiving from a client a request for map data that includes a metadata query, searching the 3D map database for map data according to the request, selecting two or more of the stored maps, and aligning the selected maps by computing a Seven Degrees of Freedom (7DOF) similarity transform between their coordinate systems, comprising a 3DOF rotation, a 3DOF translation, and a 1DOF scale (see Schmalstieg, FIG. 5, blocks 521, 526, 531, and accompanying description; description of stitching and the 7DOF similarity transform accompanying FIG. 2), and fusing the aligned maps into a single combined map (see Schmalstieg, description of Fusion 235; FIGS. 4A–4B, 6 and accompanying description). Schmalstieg further discloses using the resulting combined map to register shared augmented reality content across the clients whose maps were stitched together (see Schmalstieg, FIG. 11 and accompanying description).
Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to register each user’s space, as generated and analyzed in Keshavarzi, in a metadata-tagged database as disclosed by Schmalstieg, and to search that database for another user’s registered space using Schmalstieg’s query and alignment process, in order to identify a space satisfying the alignment conditions of Keshavarzi. The motivation is to locate a compatible mutual space automatically rather than by the manual process that Keshavarzi itself identifies as imprecise and labor intensive (see Keshavarzi, §1), using a database structure that Schmalstieg demonstrates is effective for combining independently generated maps from multiple users into a common coordinate system for shared, collaborative use (see Schmalstieg, description preceding FIG. 11).
The combination of Keshavarzi and Schmalstieg does not expressly disclose that the resulting mutually shareable space is generated in a metaverse.
In the same field of endeavor, Azmandian discloses a virtual reality embodiment in which the user is presented with a virtual space or “metaverse” and metaverse content through a head-mounted display (see Azmandian, par. 64).
Therefore, it would have been obvious to a PHOSITA before the effective filing date to present the mutually shareable space of the Keshavarzi-Schmalstieg combination as a metaverse, as taught by Azmandian, in order to provide the users with the immersive virtual space experience associated with that term.
Regarding claim 3, Keshavarzi discloses generating the shared space from the users’ rooms as set forth for claim 1. However, Keshavarzi does not disclose that the meta-space is generated to include all of the objects constituting the user space and the objects constituting the mutually shareable space, because the mutual space of Keshavarzi is the intersection of the rooms’ available spaces (see Keshavarzi, §3.3.3).
In the same field of endeavor, Azmandian discloses generating a merged instance of a virtual world that incorporates the in-game objects of both users’ instances, such that each user has access to all of the objects brought in from the other user’s instance (see Azmandian, FIG. 2, operation 213; FIG. 7 and associated description). Azmandian further discloses repositioning objects into open areas to resolve positional interference so that each object can be included in the merged instance (see Azmandian, FIGS. 8A–8B).
Therefore, it would have been obvious to a PHOSITA before the effective filing date to include the objects of both the user space and the mutually shareable space in the meta-space of Keshavarzi, repositioning them as needed within the aligned space, in order to preserve each user’s content in the shared space and give each user access to the other user’s objects.
Regarding claim 4, Keshavarzi discloses generating the meta-space by performing alignment on a position and an orientation between the user space and the mutually shareable space, in that a rigid-body motion comprising a translation and a rotation is applied to each room (see Keshavarzi, §3.3.3, Eqs. 6–7), and the search parameters allow free movement and orientation of each room (see Keshavarzi, §4).
However, Keshavarzi does not disclose alignment on a scale.
In the same field of endeavor, Schmalstieg discloses aligning the maps of different clients by computing a 7DOF similarity transform comprising a 3DOF translation, a 3DOF rotation, and a 1DOF scale, and applying the resulting scale factor when transforming the 3D structure of one client’s map into the coordinate system of another (see Schmalstieg, description of the 7DOF similarity transform accompanying FIG. 2; description of the scale factor and accompanying equations following FIG. 8).
Therefore, it would have been obvious to a PHOSITA before the effective filing date to include a scale adjustment among the alignment parameters of Keshavarzi, in addition to position and orientation, using the scale-factor computation of Schmalstieg, in order to align rooms of different sizes (see Keshavarzi, Abstract; §1) with a greater mutually usable area.
Regarding claim 5, Keshavarzi discloses that the mutually shareable space is a user space for each of the other users generated through user terminals thereof, in that each user scans his or her own room with a head-mounted display, the resulting mesh data is sent to a central server, and the room of each remote participant is aligned with the others to form the mutual space (see Keshavarzi, §6; §3.3.3; FIG. 6). Keshavarzi’s scan of a user’s room with a Matterport camera likewise generates a user space for that user (see Keshavarzi, §6).
Regarding claim 6, Keshavarzi in view of Azmandian teaches the meta-space of claim 3 for the reasons given above. Keshavarzi discloses that users may rearrange the furniture of their rooms to expand the mutual space, and that the updated mutual boundaries are presented to the users (see Keshavarzi, §3.4; §6).
However, Keshavarzi does not disclose that the meta-space is capable of being edited by multiple users who share it, with the edited information shared with those users.
In the same field of endeavor, Azmandian discloses that, within the merged instance, the players create and modify objects, that the objects created by each player during the merge are tracked, and that both players receive video streams of the merged instance (see Azmandian, FIG. 2, operations 215–217; FIG. 10 and associated description).
Therefore, it would have been obvious to a PHOSITA before the effective filing date to allow the users sharing the meta-space of Keshavarzi to edit it and to share the edits with one another, as in Azmandian, in order to promote interactivity and collaboration among the users of the shared space (see Azmandian, description preceding FIG. 2).
Regarding claim 7, Keshavarzi discloses that, when the objects in a room are moved, the standable space of the room changes and the mutual space is recalculated and expanded in successive steps to correspond to the changed space (see Keshavarzi, §3.4, Eqs. 8–10; §5; FIG. 5), and that the resulting standable, sittable and mutual boundaries are updated for display in the shared AR experience (see Keshavarzi, §6).
However, Keshavarzi does not expressly disclose that the update follows an edit made by a user of the shared meta-space.
Azmandian discloses that objects created or modified by the players within the merged instance are tracked and reflected in the merged instance presented to both players (see Azmandian, FIG. 10 and associated description; FIG. 2, operations 215–217).
Therefore, it would have been obvious to a PHOSITA before the effective filing date, when a user edits the meta-space so as to change an activity-allowed space in the user space or the mutually shareable space, to update the meta-space to correspond to the changed activity-allowed space, in order to keep the shared space free of conflicts between the users after the edit (see Keshavarzi, §7).
Regarding claim 8, Keshavarzi discloses obtaining object information about real objects included in the user space image based on a big data-based object database, in that the objects in the scanned room are identified using an object classifier derived from the Matterport3D dataset, a large-scale annotated RGB-D dataset of scanned building scenes having over one thousand object labels, which yields each object’s category and bounding box (see Keshavarzi, §3.1; §4). Keshavarzi also obtains further object information, such as dimensions and weights of each furniture type, from architectural standards (see Keshavarzi, §3.3.2; §3.4). Keshavarzi discloses mapping the objects to the user space in accordance with the object information, in that the labeled objects are converted into a topological scene graph that places the objects within the room and records their relationships to one another and to the room boundary (see Keshavarzi, §3.2; §4).
Regarding claim 9, Keshavarzi discloses that, when two or more mutually shareable spaces are present, the meta-space is generated by aligning the two or more mutually shareable spaces to the user space within a range in which the alignment conditions are satisfied. In particular, Keshavarzi treats m rooms, applying a rigid-body motion to each room such that the intersection of all of the rooms’ available spaces is maximized (see Keshavarzi, §3.3.3, Eqs. 6–7), evaluates groups of two, three and four rooms (see Keshavarzi, §4), and demonstrates the alignment of three rooms in an augmented reality experience (see Keshavarzi, §6; FIG. 6). Keshavarzi further states that its method can search a potentially unlimited number of spaces (see Keshavarzi, §2).
Schmalstieg likewise discloses stitching more than two independently created maps together, including stitching a first client’s map to a second map and that second map to a third client’s map so as to align all three maps within a single coordinate system (see Schmalstieg, FIGS. 4A–4B and accompanying description).
Azmandian likewise discloses extending its merge to any number of players, with multiple merge invitations presented to a given player (see Azmandian, description following FIG. 2, operation 217).
It would have been obvious to a PHOSITA before the effective filing date to align two or more shareable spaces to the user space in the combined system, in order to allow more than two users to share one meta-space.
Claims 10 and 12-18 recite similar limitations as respective claims 1 and 3-9, but it is directed to a meta-space generation apparatus. Since Keshavarzi’s method is a computer-implemented method, these claims could be rejected under the same rationales set forth in the rejections of their respective claims.
Claim(s) 2 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Keshavarzi in view of Schmalstieg and Azmandian as applied to respective claims 1 and 10, and further in view of Polichroniadis et al. (Pat. No. US 12,112,519).
Regarding claim 2, Keshavarzi discloses that the alignment conditions comprise a condition ensuring that an activity-allowed space, extracted in consideration of the structural feature, does not overlap between the user space and the mutually shareable space. Specifically, Keshavarzi extracts standable space as the room area remaining after the objects are subtracted (see Keshavarzi, §3.3.1, Eq. 1) and sittable space from the objects and their topological constraints (see Keshavarzi, §3.3.2, Eqs. 2–5), and takes the mutual space as the intersection of the transformed standable spaces (see Keshavarzi, §3.3.3, Eqs. 6–7), so that no remote participant is placed in a location that conflicts with any local user’s space (see Keshavarzi, §7). Keshavarzi also discloses aligning the room boundaries to maximize the overlap of the rooms (see Keshavarzi, §3.3; §3.3.3), which is a measure of how closely the background structures of the rooms match, and applies predetermined threshold criteria to its area metric, requiring an increase of 10% at each step and stopping when no further increase is possible (see Keshavarzi, §3.4, Eq. 10).
However, Keshavarzi does not expressly disclose that the similarity to the background of the user space must be equal to or higher than a reference level.
In the same field of endeavor, Polichroniadis discloses determining a similarity score between a local scene graph, representing the objects and their relationships within a user’s physical environment, and each of a set of stored scene graphs representing other environments, and discloses that, if the highest resulting similarity score does not meet a predetermined similarity threshold (for example, 80% where the score is calculated as a percentage), a determination is made that no match has been found, and the corresponding action based on that match is not carried out (see Polichroniadis, claim 9; description following FIG. 6, block 606).
Therefore, it would have been obvious to a PHOSITA before the effective filing date to require, in the combination of Keshavarzi and Schmalstieg, that the similarity score between the structural features, including the background, of the user space and of the mutually shareable space meet or exceed a predetermined reference level before that mutually shareable space is used, as taught by Polichroniadis, in order to avoid the poorly constrained results that Keshavarzi observes when a small room is fitted within a much larger one (see Keshavarzi, §5) and to ensure, consistent with Polichroniadis, that only a sufficiently similar space is selected for use.
Claim 11 recites similar limitations as claim 2, but it is directed to a meta-space generation apparatus. Since Keshavarzi’s method is a computer-implemented method, claim 11 could be rejected under the same rationale set forth in the rejection of claim 2.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHONG X NGUYEN whose telephone number is (571)270-1591. The examiner can normally be reached Mon-Fri 8am - 5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHONG X NGUYEN/ Primary Patent Examiner, Art Unit 2617
