DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment / Arguments
Claim Objections. Applicant’s amendment overcomes the claim objections.
103 Rejections. Applicant’s amendment included amendments for style. Regarding the added language to the independent claims, this does not distinguish the claims over the prior art.
Rather, the language added is respectfully more definitional and superfluous, basically reciting that a 3D voxel space comprises a plurality of voxels (Baillard teaches a 3D space, this was respectfully mapped in the last office action), and that the “voxel occupancy” identified voxels occupied. This is redundant claim language and does not distinguish the claims over the prior art.
The examiner is not persuaded that voxel occupancy and overall size of the object or boundary are distinguishable in the manner that Applicant wishes (Remarks, pages 11-12). Baillard teaches determining 3D objects in a scene (e.g. Figs. 3-5). This teaches Applicant’s claimed voxel occupancy that identifies voxels occupied. The rejection are maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 11 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Baillard (U.S. Patent App. Pub. No. 2022/0148278 A1; cited in IDS) in view of Couche (U.S. Patent App. Pub. No. 2024/0104876 A1).
Regarding claim 1:
Baillard teaches: a method for object placement in an extended reality (XR) environment (para. 16, method for placement of a virtual object in AR or MR), the method being executed by at least one processor in an XR apparatus (executed by Fig.8: 80 device, having a processor 82), the method comprising:
determining a voxel occupancy of a first position of a first object available in a virtual scene in a three-dimensional (3D) voxel space comprising a plurality of voxels, wherein the voxel occupancy of the first object identifies voxels among the plurality of voxels occupied by the first object (see Fig. 3, 3D scene, voxel occupancies are determined for a first position of a first object (in this example, the TV, TV stand, table, any of these can be a voxel occupancy of a first position of a first object that identify which voxels are occupied); …
determining a voxel occupancy for the second object for placing the second object in the second position in the 3D space, wherein the voxel occupancy of the second object identifies voxels, among the plurality of voxesl occupied by the second object at the second position (para. 103, for a candidate insertion area (position for placing the virtual object), the maximal size that the virtual object can reach inside the area is determined. This teaches “voxel occupancy” for the second object. See also Fig. 4 and claim 16, the “insertion size” of the virtual object also teaches this);
determining a voxel space in-between the first position of the first object and the second position of the second object based on the voxel occupancy of the first object and the voxel occupancy of the second object (see e.g. claim 17, para. 38-44); and
displaying at least one recommendation to place the second object in the virtual scene based on the voxel space in-between the first position of the first object and the second position of the second object (Fig. 6, the recommendation for the virtual object 71 (claimed “second object”) is displayed, in combination with para. 141, “if the user is not happy with the selected insertion point and scale, he can go to the next proposal…” which teaches/suggests that the Fig. 6 insertion is a recommendation displayed by the system, based on factors mapped above).
Regarding: receiving a user input including a second position for placing a second object in the virtual scene, consider the following.
In analogous art, Couche, relevant to 3D editing of virtual objects using voxels (paras. 2-3), teaches that it is known to receive user inputs indicating positions for placing virtual objects in a virtual scene (Couche, para 28, “In some examples, the user may move the input device to indicate (e.g., point to) virtual locations where voxels are to be deposited to form the virtual object”).
Modifying the applied references, such to include the user interactivity per Couche, in modifying or interacting with 3D scenes, per both references, is all of taught and suggested by the prior art, and would have been obvious and predictable to one of ordinary skill, with additional motivation to provide users with increased control and/or interactivity with environments. See MPEP §2143(A).
The prior art included each element recited in claim 1, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 11: see also claim 1.
Baillard teaches: an extended reality (XR) apparatus (Fig. 8: 80 device) for object placement in a virtual environment (Abstract, para. 146), comprising: memory (Fig. 8: 81, 83); at least one processor (Fig. 8: 82); a display (para. 58, device can be a HMD, phone or tablet, all of these having displays); and an object placement controller (e.g. para. 147, a microcontroller or DSP, or one aspect of the processor), communicatively coupled to the memory, the at least one processor, and the display, configured to: The functions of the apparatus correspond to the method of claim 1; the same rationale for rejection applies.
Regarding claim 16: see also claim 1.
Baillard teaches: a non-transitory computer-readable storage medium storing one or more instructions, that when executed by at least one processor (claim 36), cause an extended reality XR device (e.g. device of Fig. 8. See also Abstract) to.
The functions of the device correspond to the method of claim 1; the same rationale for rejection applies.
Claim(s) 2-4, 12-14 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Baillard in view of Couche and further in view of Lu (U.S. Patent App. Pub. No. 2024/0404075 A1).
Regarding claim 2:
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained: the method of claim 1, wherein the at least one recommendation comprises at least one alternate position to place the second object and an optimal orientation of the XR apparatus for placement of the second object in the virtual scene, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
Lu teaches that it is known to provide users with recommendations (see para. 199, which teaches a “recommend position indicator”, which can also indicate an orientation at which the device should be placed”). See also Figs. 7E-H. Modifying the applied references, such to incorporate recommendations for positions to place virtual objects, per Baillard and Couche, including optimal orientation of device, per Lu, is all of taught and suggested by the prior art, with additional motivation to assist users in viewability of interactive content.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 3:
It would have been obvious for one of ordinary skill in the art to have further modified the applied reference(-s), in view of same, to have obtained: the method of claim 2, wherein the at least one alternate position is beyond a current field of view of the XR apparatus, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
Lu teaches that it is known to provide indicators that some positions or placement are recommended to occur outside a current FOV of the device based on criteria of Lu (Figs. 7H, 7I). Baillard also teaches that distance between an insertion point and a user position, or region of interest, or another virtual object, are criteria for determining candidate insertion areas for virtual objects (paras. 38-41). Modifying the applied references, such that the insertion point, per Baillard, is recommended to be outside a current FOV, per Lu, in consideration of criterial of Baillard, is taught/suggested by the prior art, and would have been obvious and predictable to one of ordinary skill.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 4:
Lu and/or Couche teaches: the method of claim 2, wherein the method comprises displaying a directional indicia indicating the at least one alternate position to place the second object in the virtual scene (e.g. Figs. 7E-J, all examples of directional indicators indicating positions for placement) (see also Couche, Fig. 3H: arrows as indicators for direction are also known). Modifying the applied references, such to incorporate visual direction indicia per Lu or Couche, for where to place virtual objects, per Baillard and Couche, is all of taught and suggested by the prior art, with additional motivation to assist users in viewability of interactive content. See MPEP §2143(A).
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 12: see claim 2.
These claims are similar; the same rationale for rejection applies.
Regarding claim 13: see claim 3.
These claims are similar; the same rationale for rejection applies.
Regarding claim 14: see claim 4.
These claims are similar; the same rationale for rejection applies.
Regarding claim 17: see claim 2.
These claims are similar; the same rationale for rejection applies.
Regarding claim 18: see claim 3.
These claims are similar; the same rationale for rejection applies.
Regarding claim 19: see claim 4.
These claims are similar; the same rationale for rejection applies.
Claim(s) 5, 15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Baillard in view of Couche and further in view of Rakshit (U.S. Patent App. Pub. No. 2023/0062028 A1).
Regarding claim 5:
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained: the method of claim 1, wherein the displaying the at least one recommendation to place the second object in the virtual scene based on the voxel space comprises: determining whether the voxel space in-between the first position of the first object and the second position of the second object meets a voxel space threshold (Baillard, which teaches distance between an insertion and another virtual object are criteria for determining candidate insertion areas for virtual objects (paras. 38-41). This teaches Applicant’s claimed “threshold”);
based on the voxel space in-between the first position of the first object and the second position of the second object meeting the voxel space threshold (see mapping above to Baillard), determining the at least one recommendation to place the second object in the virtual scene based on a category of the second object (Rakshit, para. 13 for category of object being relevant for placement. See also para. 32 for placement recommendation teaching); and
displaying the at least one recommendation to place the second object in the virtual scene (mapped in claim 1),
and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 5, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 15: see claim 5.
These claims are similar; the same rationale for rejection applies.
Regarding claim 20: see claim 5.
These claims are similar; the same rationale for rejection applies.
Claim(s) 7 is rejected under 35 U.S.C. 103 as being unpatentable over Baillard in view of Couche and further in view of Rakshit and Goyal (U.S. Patent App. Pub. No. 2023/0234233 A1).
Regarding claim 7:
The applied references to claim 5 do not proactively teach claim 7. Consider the following.
In analogous art, Goyal teaches: the method of claim 5, wherein the method comprises determining the voxel space threshold using an AI model (e.g. paras. 106, training model with threshold distances to address object clutter in combination with para. 68-70, 3D models of scenes).
Modifying the applied references, such to determine a voxel space threshold using an AI model (the one in Goyal to prevent object clutter, for example), would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Allowable Subject Matter
Claims 6, 8, 9 and 10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the prior art does not disclose, and would not have rendered obvious, the combination of features as per the above listed claims, as shown below.
6. The method of claim 5, wherein the determining the at least one recommendation to place the second object in the virtual scene based on the category of the second object comprises: capturing a ray in one or more angles with respect to a current camera angle; estimating voxels covered in each of the one or more angles; determining a voxel occupancy for an optimal orientation of the XR apparatus in each of the one or more angles based on the voxels covered in each of the one or more angles; and determining the at least one recommendation to place the second object in the virtual scene based on voxel occupancy for the optimal orientation of the XR apparatus and the category of the second object.
8. The method of claim 7, wherein the method comprises: determining semantic parameters of a plurality of scenes, semantic parameters of objects in each scene of the plurality of scenes, a type of user interaction with objects in each scene of the plurality of scenes, and a correlation between voxel space and user interaction with objects in each scene of the plurality of scenes; and training the AI model based on the semantic parameters of the plurality of scenes, the semantic parameters of objects in each scene of the plurality of scenes, the type of user interaction with objects in each scene of the plurality of scenes, and the correlation between voxel space and user interaction.
9. The method of claim 1, wherein the determining the voxel occupancy of the first position of the first object available in the virtual scene comprises: displaying a 360 degree camera view of the virtual scene; generating a 3D voxel space based on a 3D spatial partitioning of the 360 degree camera view of the virtual scene, wherein the 3D voxel space comprises a plurality of voxels, and wherein each of the plurality of voxels represents whether at least one portion of the first object lies in the 3D voxel space; determining mesh coordinates of each voxel of the plurality of voxels and imaging sensor parameters of the XR apparatus; and determining the voxel occupancy of the first object available in the virtual scene based on the mesh coordinates of each voxel of the plurality of voxels and the imaging sensor parameters of the XR apparatus.
10. (Original) The method of claim 1, wherein the determining the voxel occupancy for the second object for placing the second object in the second position comprises: displaying a 360 degree camera view of the virtual scene;
generating a 3D voxel space based on a 3D spatial partitioning of the 360 degree camera view of the virtual scene, wherein the 3D voxel space comprises a plurality of voxels, and wherein each of the plurality of voxels represents whether at least one portion of the first object lies in the 3D voxel space;
determining a number of voxels of the plurality of voxels that are occupied by the second object in the virtual scene by performing a hit test for each voxel of the plurality of voxels based on the second position of the second object provided in the user input; and
determining the voxel occupancy for the second object in the second position based on the number of voxels that are occupied by the second object in the virtual scene.
The closest prior art is of record and discussed herein, in part. However, the prior art, alone or in varied combination, would not have rendered obvious the above claim features. That is to say, the prior art does not teach determining voxel occupancy for an optimal apparatus orientation in one or more angles of a ray captures, where the ray was used to estimate covered voxels, as per claim 6. Likewise, the reference of Goyal, while it does teach a model that can be trained using threshold distances to address object clutter and relevant to object placement (paras. 68-70, 106), and neural networks trained for semantic understanding (para. 204), the instant reference does not teach the above features of claim 8, as recited above. And regarding claims 9 and 10, the closest prior art does not teach the combined features as they relate to 360 degree cameras vies, spatial portioning thereof, mesh coordinates, and voxel occupancy (claim 9) and related features of 360 degree views per claim 10.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
* * * * *
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sarah Lhymn whose telephone number is (571)270-0632. The examiner can normally be reached M-F, 9:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Sarah Lhymn
Primary Examiner
Art Unit 2613
/Sarah Lhymn/Primary Examiner, Art Unit 2613
Prosecution Insights