DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is responsive to the Application No. 19/238,047 filled on 06/13/2025.
Claims 1-20 are presented for examination.
Drawing/Specification Objections
The drawing is objected to because of the following informalities: The drawings, Figs. 2C-4D are not of sufficient quality to permit examination. Accordingly, replacement drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to this Office action. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 15 and 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Analysis (Claim 15)
Step 1: Statutory Category-Yes
The claim 8 is directed to "An Apparatus”.
Step 2A-Prong 1: Judicially Exception Recited-Yes
The claim 15 recites the limitations of: An apparatus comprising: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform a method comprising: parsing, with a generative large language model (LLM), free-form instructions to extract intent and context information; creating a three-dimensional (3D) environmental model of a surrounding environment of the apparatus, wherein the 3D environmental model represents objects, object parts, and inter-object relationships with labels derived from one or more vision- language models (VLMs); and generating, with a closed-loop planner, at least one action based at least in part on the intent and context information and the 3D environmental model. The highlighted elements above are considered to be directed to mental processes and/or mathematical concepts, such as, steps above are steps encompass a mental process as it could be done by pen and paper based on observed/gathered intent and context information. Therefore, the highlighted limitation above, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in mental process but for the recitation of the above underlined limitations above, where the underline limitations are referred to at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform a method. That is, other than reciting the underlined limitations above, nothing in the claim elements precludes the step from practically being performed in the mental process and/or mathematical calculation.
Step 2A—Prong 2: Integrated into a Practical Application-No
The claim recites additional elements, such as, “at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform a method”, no more than mere instructions to apply the exception using a computer. Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does do not impose any meaningful limits on practicing the abstract idea. The claim is directed to the abstract idea.
Step 2B: Claim provides an Inventive Concept-No
Claims 15 is evaluated as to whether the claims as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
As discussed with respect to Step 2A Prong Two, the additional elements in the claim amount to no more than insignificant extra-solution activity. Under the 2019 PEG, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be re-evaluated in Step 2B. The additional elements, “at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform a method” is recited as high level of generality and merely automates parsing, creating and generating steps, therefore acting as a generic computer to perform the abstract idea, doesn’t use the judicial exception in a manner that impose a meaningful limit on the judicial exception, such that the claim is more than drafting effort designed to monopolize the exception. This additional limitation is no more than mere instruction to apply the exception using a computer. The claim is ineligible.
Dependent Claims 18-20 are determined to be directed to an abstract idea based on similar rationale since no feature in the claims alter the abstract idea.
Therefore, Claims 15 and 18-20 are ineligible UNDER 35 USC 101 and are rejected under 35 USC § 101.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-14 and 16-17 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1-3, 13 and 16 recite the phrase “one or more preconditions”, where the preconditions is not further described within the specification. Therefore, claim does not satisfy the written description requirement.
Claims 2-12 and 14 and 17 are also rejected by the virtue of their dependency on rejected base claim(s).
Examiner's Note
Examiner has cited particular paragraphs/ columns and line numbers or figures in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching
all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Applicant is reminded that the Examiner is entitled to give the broadest reasonable interpretation to the language of the claims. Furthermore, the Examiner is not limited to Applicants' definition which is not specifically set forth in the claims.
Claim Objections (having allowable subject matter)
Claims 1-20 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(a), 35 U.S.C. 101, and claims/drawing objections for informalities, set forth in this Office action.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 15, the closest prior art, Wannerberg et al. (US 2025/0252661 A1) teaches an apparatus comprising: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform a method (See Para. [0132]-[0133], discloses “computing device 1001 comprises processing/control circuitry 1006/10041004, as referred to herein, control circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer. In some embodiments, the control circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry 1004 executes instructions for the system or application stored in memory (e.g., storage 1008). Specifically, control circuitry 1004 may be instructed by the system or application to perform the functions discussed above and below) comprising:
parsing, with a generative large language model (LLM), free-form instructions to extract intent and context information (See Para. [0169], “extract a context using LLM”, and see Para. [0069]-[0070], discloses “The LLM may be configured to generate text in desired formats using text prompts, and/or visual elements (e.g., by visually inspecting pixels or voxels for use in a neural network), as input, and/or other modalities”);
creating a three-dimensional (3D) environmental model of a surrounding environment of the apparatus, wherein the 3D environmental model represents objects, object parts (See Para. [0073], “generate a 3D object similar to object 118, which more closely matches the determined context of the 3D environment”, and see Para. [0080], [0093], discloses “3D models for finding 3D object for optimum candidates”), and
generating, with a closed-loop planner, at least one action based at least in part on the intent and context information and the 3D environmental model (See Para. [0080], “the system may generate a 3D environment (e.g., including a VR scene) in which the world is being built based at least in part on user inputs. For example, the system may determine that the 3D environment shown at 602 has a natural language description and/or context that relates to a 17th-century interior. As shown at 602, the system may receive a user search for one or more 3D objects, or may automatically present a 3D object as a candidate for insertion (or automatically insert a 3D object) that is relevant to the context of the current 3D scene”).
Another prior arts, Wu et al. (NPL document, On the Safety Concern of Deploying LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities, this reference is from IDS) teaches, “vision-language models (VLMs) has enabled robots to perform various complex tasks by enhancing their capabilities for natural language processing and visual recognition” (See Introduction on page 1)
Another prior arts, Goyal et al. (US 2024/0371082 A1) discloses in Para. [0069], “the MLM(s) 106 may be trained to generate predictions 180 corresponding to a 3D object manipulation task. For example, the machine 800 may include a robot and predictions generated using the MLM(s) 106 may be used to perform one or more control operations for 3D manipulation of an object in the environment”, and in Para. [0256], “a system for performing operations using one or more vision language models (VLMs)”)
Another prior art, Takashi et al. (WO2025/164534 A1) discloses in para. [0105], “the vision-and-language model 60 determines whether the relationships between objects represented by visual data 70 match the relationships between objects represented by linguistic data 80.”, and in Para. [0109], “vision-and-language model 60 described above can be used to train the relation extraction model used to generate object relation information 40”, and/or see Para. [0107], “the second feature calculation model 64 of the trained vision-and-language model 60, in which text data representing the relationships between objects”.
Another prior art, Mohalik et al. (US 2021/0318906 A1) discloses in Para. [0043], “precondition P models the state of the system 12 that is required for the action to be performed.”)
Nevertheless, the cited reference fails to teach or suggest the claimed feature of “create inter-object relationships with labels derived from one or more vision- language models (VLMs)” and in combination with other limitations of claim 15.
Claims 16-20 depends directly or indirectly upon claim 15. Therefore claims 16-20 would be allowable by virtue of dependency.
Regarding claim 13, the closest prior art, Wannerberg et al. (US 2025/0252661 A1) teaches, a robotic system for planning and execution in an environment, the robotic system comprising: one or more robotic manipulators and/or actuators to interact with the environment and/or to move through the environment; one or more processors; and one or more non-transitory computer-readable media coupled to one or more of the processors and comprising instructions that, when executed by the one or more processors, cause the robotic system to perform a method (See Para. [0132]-[0133], discloses “computing device 1001 comprises processing/control circuitry 1006/10041004, as referred to herein, control circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer. In some embodiments, the control circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry 1004 executes instructions for the system or application stored in memory (e.g., storage 1008). Specifically, control circuitry 1004 may be instructed by the system or application to perform the functions discussed above and below) comprising:
receiving a natural language command (See Para. [0148], “control circuitry 1111 may be used to send and receive commands”);
processing the natural language command using a generative large language model to extract an intent and associated context (See Para. [0042], “extract the different 3D objects, and/or different types or classes of 3D objects, or portions thereof”, and/or see Para. [0169], “extract a context using LLM”, and see Para. [0069]-[0070], discloses “The LLM may be configured to generate text in desired formats using text prompts, and/or visual elements (e.g., by visually inspecting pixels or voxels for use in a neural network), as input, and/or other modalities”);
creating a three-dimensional (3D) open-vocabulary semantic scene graph of the environment, the scene graph comprising nodes for objects and object parts annotated with vision-language features and edges for their relationships, and associating the scene graph with the intent and associated context (See Para. [0082], “the collected semantic understanding of one or more 3D objects in the 3D environment may be created based on output from the previous step(s), which can be processed as text. For example, the system may combine the results of performing visual inspection of the 3D object (either as a whole scene or as individual 3D assets) and the metadata analysis from the one or more 3D content libraries to output text-based information into, e.g., a predetermined format that is suitable for use together with LLMs, and such text-based information may include any suitable detail, e.g., material and/or texture data of the one or more 3D objects in the 3D environment”);
creating, based at least in part on the scene graph, an execution plan comprising a sequence of actions to complete the natural language command, wherein creating the execution plan comprises operating in a closed-loop with iterative refinement (See Para. [0080], “the system may generate a 3D environment (e.g., including a VR scene) in which the world is being built based at least in part on user inputs. For example, the system may determine that the 3D environment shown at 602 has a natural language description and/or context that relates to a 17th-century interior. As shown at 602, the system may receive a user search for one or more 3D objects, or may automatically present a 3D object as a candidate for insertion (or automatically insert a 3D object) that is relevant to the context of the current 3D scene”) and predicate grounding by:
Another closest prior art, Yang et al. (CN116587265, attached English translate NPL document is used for claim mapping) teaches,
generating one or more preconditions for each plausible action of the plurality of plausible actions (See Para. [0007], “generate a set of preconditions and a set of expected effects”);
determining, for each plausible action, whether that plausible action meets or is likely to meet the one or more preconditions associated with that plausible action (See Para. [0010], “Determine whether the first observation meets the preconditions in the precondition set. If yes, execute the effect behavior; otherwise, repeat the execution of the precondition observation behavior set”);
determining the sequence of actions comprising a set of the plausible actions that meets or is likely to meet the one or more preconditions (See Para. [0076], “determine whether all the expected effects have been met”); and
controlling the one or more robotic manipulators and/or actuators to perform one or more actions of the sequence of actions based on the execution plan (See Para. [0092], “drive the robot's effectors to operate”, and see Para. [0053], “Autonomous robots accomplish their tasks by sequentially executing each effect behavior in a sequence.”).
Nevertheless, the cited prior arts as discussed above fail to teach or suggest the claimed limitations of “determining a plurality of plausible actions based at least in part on a current state of the robotic system and the intent and associated context; self-reflectively evaluating and revising one or more branches of the tree search when a feasibility threshold is not met; and generating executable code or tool calls corresponding to one or more actions in the sequence of actions”, and in combinations of other limitations of claim 13.
Claim 14 depends upon claim 13. Therefore claim 14 would be allowable by virtue of dependency.
Claim 1 is a method claim corresponding to the system claim 13 and having substantially the same technical features as claim 1, differing only in the category of invention. Therefore, the claim 1 is rejected for the same rationales set forth as above for claim 1.
Claims 2-12 depend directly or indirectly upon claim 1. Therefore claim 2-12 would be allowable by virtue of dependency.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to B M M HANNAN whose telephone number is (571)270-0237. The examiner can normally be reached MONDAY-FRIDAY at 8:30AM-5:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached at 5712705376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B M M HANNAN/Primary Examiner, Art Unit 3657
Prosecution Insights