Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9, 11-16 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a mental concept without significantly more. The claims recite the abstract idea of generating simulation environments based on user input, generating tasks, determining robot trajectories, generating simulation data, sampling objects and layouts, selecting environments and interpreting user input. This judicial exception is not integrated into a practical application because the additional elements of a computer, memory, and processors do not improve the functioning of a computer. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are generic computer parts. MPEP 2106.05(h).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 5, 14, 15 and 17-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 5 and 15 recite “based one or more predefined rules.” Amend to claim base ON one or more predefined rules.
Claim 14 recites “executed by at the least one processor…” Amend to claim by the at least one processor – at and the are swapped in the claim.
Claim 17 recites a CRM “comprising performing”. Amend to claim a CRM storing instructions that when executed by at least one processor to perform the steps of performing – similar to claim 11.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 5-7, 10, 11-13, 15, 17, 18 and 20 are rejected under 35 U.S.C. 102(a)(1) as being described by ProcTHOR: Large-Scale Embodied AI Using Procedural Generation by Deitke et al.
Deitke teaches claims 1, 11 and 20. A computer-implemented method for generating simulation data to train a machine learning model, the method comprising: (Deitke abs “a framework for procedural generation of Embodied AI environments.”)
generating a plurality of simulation environments based on a user input; and (Deitke p. 6 “Customizability. PROCTHOR supports many room, asset, material, and lighting specifications. With a few simple lines of specification, one can easily generate customized environments of interest.” Deitke fig. 2 p. 4 “Fig. 2 shows a high-level schematic of the procedure used by PROCTHOR to generate a scene. Given a room specification (e.g. house with 1 bedroom + 1 bathroom)“ The specification comes for a user.)
for each simulation environment included in the plurality of simulation environments:
generating a plurality of tasks for a robot to perform within the simulation environment, (Deitke p. 8 sec. 5 “Tasks. We now present results for models pre-trained on PROCTHOR-10K on several navigation and manipulation benchmarks to demonstrate the benefits of large-scale training. We consider ObjectNav (navigation towards a specific object category) in PROCTHOR, ARCHITECTHOR, RoboTHOR [27], HM3D [94], and AI2-iTHOR [63]. We also consider two manipulation-based tasks: ArmPoint Nav [33] and 1-phase Room Rearrangement [115]. In ArmPointNav, the agent moves an object using a robotic arm from a source location to a destination location specified in the 3D coordinate frame. In Room Rearrangement, the goal is to move objects or change their state to reach a target scene state.” The tasks for each environment are object nav and two manipulation based tasks.)
performing one or more operations to determine a plurality of robot trajectories for performing the plurality of tasks, and (The trajectories are what make up the task, e.g. Deitke p. 8 sec. 5 “In ArmPointNav, the agent moves an object using a robotic arm from a source location to a destination location specified in the 3D coordinate frame.”)
generating simulation data for training a machine learning model by performing one or more operations to simulate the robot moving within the simulation environment according to the plurality of trajectories. (Deitke abs “PROCTHOR enables us to sample arbitrarily large datasets of diverse, interactive, customizable, and performant virtual environments to train and evaluate embodied agents across navigation, interaction, and manipulation tasks.” Deitke p. 6 “This diversity of layouts, assets, materials, placements, and lighting enables the generation of arbitrarily large sets of houses– either statically generated and stored as a dataset or dynamically generated at each iteration of training. Scenes are efficiently represented in a JSON specification and are loaded into AI2-THOR at runtime, making the memory overhead of storing houses incredibly efficient. Moreover, the scene generation process is fully automatic and fast and PROCTHOR provides high framerates for training E-AI models…”)
Deitke teaches claims 2 and 12. The computer-implemented method of claim 1, wherein the simulation data comprises (i) one or more trajectories included in the plurality of trajectories that satisfy one or more goals of the plurality of tasks, and (ii) a plurality of rendered images of the simulation environment. (Deitke abs “PROCTHOR enables us to sample arbitrarily large datasets of diverse, interactive, customizable, and performant virtual environments to train and evaluate embodied agents across navigation, interaction, and manipulation tasks.” The trajectories are what make up the task, e.g. Deitke p. 8 sec. 5 “In ArmPointNav, the agent moves an object using a robotic arm from a source location to a destination location specified in the 3D coordinate frame. In Room Rearrangement, the goal is to move objects or change their state to reach a target scene state.”)
PNG
media_image1.png
144
570
media_image1.png
Greyscale
Deitke teaches claims 3 and 13. The computer-implemented method of claim 2, wherein the plurality of rendered images include at least two images rendered using at least one of different colors or different textures applied to one or more objects within the simulation environment. (Deitke fig. 8 below.)
PNG
media_image2.png
176
556
media_image2.png
Greyscale
Deitke teaches claims 5 and 15. The computer-implemented method of claim 1, wherein generating the plurality of simulation environments comprises:
performing one or more operations to sample at least one of (i) different layouts of objects from a plurality of predefined layouts, (ii) different objects from a plurality of predefined objects, or (iii) different sizes of objects, to generate a plurality of intermediate simulation environments; and (Deitke p. 8 sec. 4 “objects sampled via 18 different Semantic Asset groups. Examples of Semantic asset groups (SAG) are a Dining Table with 4 Chairs or Bed with 2 Pillows. Given our large asset library and SAGs, we can create 19.3 million combinations of group instantiations.”)
selecting the plurality of simulation environments from the plurality of intermediate simulation environments based one or more predefined rules. (Deitke p 32-33 “We sample an object or asset group that satisfies all of the previous conditions. If there are no objects or asset groups that satisfy all conditions, we continue to the next iteration and remove the selected rectangle from consideration. We slightly prioritize placing asset groups over standalone assets when possible. Once we have chosen an object or asset group, the bounding box with margin is then anchored to the corner of the rectangle, and hence to the corner of the room.”)
Deitke teaches claim 6. The computer-implemented method of claim 1, wherein at least two of the plurality of simulation environments are generated based on at least one of predefined layouts of objects or predefined three-dimensional (3D) models of objects. (Deitke p 32-33 “We sample an object or asset group that satisfies all of the previous conditions. If there are no objects or asset groups that satisfy all conditions, we continue to the next iteration and remove the selected rectangle from consideration. We slightly prioritize placing asset groups over standalone assets when possible. Once we have chosen an object or asset group, the bounding box with margin is then anchored to the corner of the rectangle, and hence to the corner of the room.”)
Deitke teaches claim 7. The computer-implemented method of claim 1, wherein generating the plurality of tasks comprises performing one or more operations to determine at least one of a plurality of goals or a plurality of sub-tasks associated with the plurality of tasks. (Deitke p. 8 sec. 5 “We consider ObjectNav (navigation towards a specific object category) in PROCTHOR, ARCHITECTHOR, RoboTHOR [27], HM3D [94], and AI2-iTHOR [63]. We also consider two manipulation-based tasks: ArmPoint Nav [33] and 1-phase Room Rearrangement [115]. In ArmPointNav, the agent moves an object using a robotic arm from a source location to a destination location specified in the 3D coordinate frame. In Room Rearrangement, the goal is to move objects or change their state to reach a target scene state.” The target scene stat is the goal.)
Deitke teaches claims 10 and 17. The computer-implemented method of claim 1, further comprising performing one or more operations to train a machine learning model based on the simulation data generated for the plurality of simulation environments. (Deitke p. 6 “This diversity of layouts, assets, materials, placements, and lighting enables the generation of arbitrarily large sets of houses– either statically generated and stored as a dataset or dynamically generated at each iteration of training. Scenes are efficiently represented in a JSON specification and are loaded into AI2-THOR at runtime, making the memory overhead of storing houses incredibly efficient.”)
Deitke teaches claim 18. The one or more non-transitory computer-readable media of claim 17, wherein the machine learning model is trained to control a physical robot. (Deitke p. 6 “This diversity of layouts, assets, materials, placements, and lighting enables the generation of arbitrarily large sets of houses– either statically generated and stored as a dataset or dynamically generated at each iteration of training. Scenes are efficiently represented in a JSON specification and are loaded into AI2-THOR at runtime, making the memory overhead of storing houses incredibly efficient.” Deitke Fig. 8, shown below.)
PNG
media_image2.png
176
556
media_image2.png
Greyscale
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over ProcTHOR: Large-Scale Embodied AI Using Procedural Generation by Deitke et al and Horizon: A Trajectory Optimization Framework for Robotic Systems by Ruscelli et al.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over ProcTHOR: Large-Scale Embodied AI Using Procedural Generation by Deitke et al and Evaluating Large Language Models Trained on Code by Chen et al.
Claims 9 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over ProcTHOR: Large-Scale Embodied AI Using Procedural Generation by Deitke et al and Webots.HPC: A Parallel Robotics Simulation Pipeline for Autonomous Vehicles on High Performance by Frachi.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over ProcTHOR: Large-Scale Embodied AI Using Procedural Generation by Deitke et al and AllenAct: A Framework for Embodied AI Research by Weihs et al.
Deitke teaches claims 4 and 14. The computer-implemented method of claim 1, further comprising performing one or more operations to [train] (Deitke p. 8 sec. 5 “In ArmPointNav, the agent moves an object using a robotic arm from a source location to a destination location specified in the 3D coordinate frame. …Our ArmPointNav model uses a simpler visual encoder with 3 convolutional layers; we found this more effective than the CLIP encoder. All models are trained with the AllenAct [116] framework, see the Appendix for training details.”)
Deitke doesn’t refine the trajectories.
However, Ruscelli teaches one or more operations to refine at least one robot trajectory. (Ruscelli abs “Horizon, an open-source framework for trajectory optimization tailored to robotic systems that implements a set of tools to simplify the process of dynamic motion generation.”)
Deitke, Ruscelli and the claims are all using robot trajectories. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to refine Deitke’s trajectories in order “to simplify the pipeline for optimal motion planning without
shadowing the underlying mechanics.” Ruscelli p. 2.
Deitke teaches claim 8. The computer-implemented method of claim 1, further comprising generating, (Deitke fig. 2 p. 4 “Fig. 2 shows a high-level schematic of the procedure used by PROCTHOR to generate a scene. Given a room specification (e.g. house with 1 bedroom + 1 bathroom)“)
Deitke doesn’t teach an LLM.
However, Chen teaches generating, via a large language model (LLM), an interpretation of the user input. (Chen title “large Language Models…” and Chen abs “we find that repeated sampling from the model is a surprisingly effective strategy for producing working solutions to difficult prompts.”)
Deitke, the claims and Chen are all interpreting user input. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to use an LLM to allow users to use something approaching natural language.
Deitke teaches claims 9 and 16. The computer-implemented method of claim 1, wherein at least two of the plurality of simulation environments are generated (Deitke p. 6 “Customizability. PROCTHOR supports many room, asset, material, and lighting specifications. With a few simple lines of specification, one can easily generate customized environments of interest.” Deitke fig. 2 p. 4 “Fig. 2 shows a high-level schematic of the procedure used by PROCTHOR to generate a scene. Given a room specification (e.g. house with 1 bedroom + 1 bathroom)“)
Deitke doesn’t teach generating in parallel.
However, Franchi teaches plurality of simulation environments are generated in parallel via a plurality of computing devices. (Franchi title “Webots.HPC: A Parallel Robotics Simulation Pipeline…” Franchi abs “a formalized parallel pipeline for running sequences of Webots simulations on powerful HPC resources.”)
Deitke, Franchi and the claim all simulate autonomous agents in an environment. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to simulate in parallel because “Such a pipeline would allow researchers to generate massive datasets from their simulations…” Franchi abs.
Deitke teaches claim 19. The one or more non-transitory computer-readable media of claim 17, wherein the one or more operations to train the machine learning model include one or more (Deitke sec. 5 p. 8 “Our ArmPointNav model uses a simpler visual encoder with 3 convolutional layers; we found this more effective than the CLIP encoder. All models are trained with the AllenAct [116] framework, see the Appendix for training details.)
Deitke doesn’t mention supervised learning.
However, Weihs teaches (Weihs p. 3 “AllenAct allows researchers to easily combine various losses while training models (for instance, use an external self-supervised loss while optimizing a PPO loss).”)
Deitke, Weihs, and the claims are all training on the environment data. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to use Allen act in Deitke because Deitke says all “maodels are trainined with AllenAct…” Deitke p. 8 sec. 5.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Austin Hicks whose telephone number is (571)270-3377. The examiner can normally be reached Monday - Thursday 8-4 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AUSTIN HICKS/Primary Examiner, Art Unit 2142
Prosecution Insights