DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Status of Claims
Claims 1-9 were originally filed on 12/13/2024 and claimed priority on JP2023-213740, which was filed on 12/19/2023.
Information Disclosure Statement
The Information Disclosure Statement filed on 12/13/2024 has been considered. An initialed copy of the Form 1449 is enclosed herewith.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, and 6-9 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al (US 20250196363 A1) in view of Rose et al (US 20240253224 A1) and Bromand et al (US 20200026489 A1) (Hereinafter referred to as Wang, Rose, and Bromand respectively)
Regarding Claims 1 and 8-9, Wang teaches a controller for controlling a robot based on an interaction with a user (See at least Wang Paragraphs 0046, 0062, and Figures 2 and 6, the system includes a digital processing apparatus to perform the operations, which is interpreted as a controller), a control method for controlling a robot based on an interaction with a user (See at least Wang Paragraph 0012), a non-transitory computer readable storage medium including a control program for controlling a robot based on an interaction with a user (See at least Wang Paragraphs 0046 and 0062), the control program being configured to, when executed by a computer (See at least Wang Paragraph 0046, the digital processing apparatus is interpreted as the computer), cause the computer to:
…
read out, from a prerequisite storage unit, a prerequisite including an operational procedure for an operation to be performed by the robot (See at least Wang Paragraphs 0069, 0113-0114, and Figure 2, the large language model acquires information from the database/prerequisite storage unit, which includes operational procedures/rules to be performed by the robot),
the controller comprising:
…
a prerequisite storage unit storing a prerequisite including an operational procedure for an operation to be performed by the robot (See at least Wang Paragraphs 0113-0114, the database is interpreted as the prerequisite storage unit, which includes operational procedures/rules to be performed by the robot), the operation by the robot being divided into a plurality of steps in the operational procedure (See at least Wang Paragraphs 0113-0114, the operational procedure is divided into a plurality of steps);
an input unit configured to receive, from the user, an instruction for the robot (See at least Wang Paragraphs 0093-0097 and Figures 2-3, the x-to-text converter is interpreted as the input unit, which receives the instruction “help me making ice tea” from the user);
an output unit configured to output, to the user, a reaction from the robot to the instruction (See at least Wang Paragraph 0101 and Figures 2-3, the text-to-x translator is interpreted as the output unit, which outputs the reaction “sure, I will give you the cup” to the user);…and
a processing unit (See at least Wang Paragraph 0046, the processing apparatus/unit performs the operations) configured to:
transmit, to a natural language processing system using a large language model, a prompt including the prerequisite, the data read out from the input (See at least Wang Paragraphs 0064-0069, 0101, 0113-0114, and Figure 2, the large language model acquires information from the database, which includes the prerequisite, and the input data from the user)…;
Wang fails to explicitly disclose read out, from a function list storage unit, a function list of a plurality of function modules each of which defines a predetermined action to be performed by the robot;
a function list storage unit storing a function list of a plurality of function modules each of which defines a predetermined action to be performed by the robot;
transmit, to a natural language processing system using a large language model, a prompt including the function list;
receive, from the natural language processing system, a response identifying one of the plurality of function modules in the function list; and
generate an action command for the robot to execute the identified one of the plurality of function modules based on the response.
However, Rose teaches read out, from a function list storage unit, a function list of a plurality of function modules each of which defines a predetermined action to be performed by the robot (See at least Rose Paragraphs 0032, 0049, 0051, and Figure 1, the instruction set is interpreted as the function list storage unit, which stores actions to be performed by the robot and is read by the LLM);
a function list storage unit storing a function list of a plurality of function modules each of which defines a predetermined action to be performed by the robot (See at least Rose Paragraphs 0032, 0049, 0051, and Figure 1, the instruction set is interpreted as the function list storage unit, which stores actions to be performed by the robot);
transmit, to a natural language processing system using a large language model, a prompt including the function list (See at least Rose Paragraphs 0005, 0049, 0051, and Figure 1, the instruction set/function list is transmitted to the LLM);
receive, from the natural language processing system, a response identifying one of the plurality of function modules in the function list (See at least Rose Paragraphs 0005, 0049, 0051, 0094 and Figure 1, the LLM outputs a task plan identifying/selecting functions from the function list/instruction set); and
generate an action command for the robot to execute the identified one of the plurality of function modules based on the response (See at least Rose Paragraphs 0051-0052, and Figure 1, the robot control signals are interpreted as the action command).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in Wang with Rose to transmit the function list to the natural language processing system using a large language model, and generate an action command for the robot to execute the identified one of the plurality of function modules based on the response. This modification, as taught by Rose, would allow the natural language processing system/LLM to select from a finite list of reusable work primitives, that when executed by a robot will cause or enable the robot to complete a task, thus allowing the natural language processing system/LLM to support a fully autonomous system that converts user input into a sequence of allowed Instructions that successfully performs the task (See at least Rose Paragraph 0049).
Modified Wang fails to disclose an input buffer configured to save data of the instruction input from the input unit.
However, Bromand teaches an input buffer configured to save data of the instruction input from the input unit (See at least Bromand Paragraphs 0029 and 0094, the system includes a buffer to save/store the utterance data/instruction input from the input unit/human-machine interface).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Wang with Bromand to have an input buffer configured to save data of the instruction input from the input unit. This modification, as taught by Bromand, would allow the system to utilize the instruction input/utterance data to determine an intent and a fulfillment strategy associated with the intent (See at least Bromand Paragraphs 0094-0099 and Figure 4), thus, improving the system’s capability to respond to user requests (See at least Bromand Paragraph 0004).
Regarding Claims 2-3, modified Wang fails to disclose receive, from the natural language processing system, the response including data to identify the one of the plurality of function modules and data of a parameter applied to the one of the plurality of function modules; and
generate the action command for the robot based on the data to identify the one of the plurality of function modules and the data of the parameter,
wherein the data of the parameter includes at least one of data relating to coordinates necessary for the robot to execute the predetermined action or data relating to wordings to be output to the user.
However, Rose teaches receive, from the natural language processing system, the response including data to identify the one of the plurality of function modules and data of a parameter applied to the one of the plurality of function modules (See at least Rose Paragraphs 0051-0052, 0113, and Figure 1, the LLM assigns parameters to the function modules/instruction set); and
generate the action command for the robot based on the data to identify the one of the plurality of function modules and the data of the parameter (See at least Rose Paragraphs 0051-0052, 0113, and Figure 1, the action command/control signals are generated based on the selection of the function module/instruction set and the parameters assigned),
wherein the data of the parameter includes at least one of data relating to coordinates necessary for the robot to execute the predetermined action or data relating to wordings to be output to the user (See at least Rose Paragraphs 0085, 0095-0101, 0104-0105, 0113, 0121-0125, and Figure 3, the parameter includes data relating to the position/coordinates necessary for the robot to execute the action).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Wang with Rose to generate the action command for the robot based on the data to identify the one of the plurality of function modules and the data of the parameter wherein the data of the parameter includes data relating to coordinates necessary for the robot to execute the predetermined action. This modification, as taught by Rose, would allow the system to autonomously generate the action command by populating the function modules/instruction set with parameters for the current task, which includes the positions/coordinates for the robot to execute the action (See at least Rose Paragraphs 0085, 0095-0101, 0104-0105, 0113, 0121-0125, and Figure 3), thus, improving the operability of the robot.
Regarding Claim 6, modified Wang teaches a robot system comprising: the controller according to claim 1 (See at least Wang Paragraphs 0046, 0062, and Figure 2, the robot system includes the controller/digital processing apparatus); and the robot (See at least Wang Paragraph 0062, and Figure 2).
Regarding Claim 7, modified Wang teaches a system comprising: the controller according to claim 1 (See at least Wang Paragraphs 0046, 0062, and Figure 2, the system includes the controller/digital processing apparatus); the robot (See at least Wang Paragraph 0062, and Figure 2); and the natural language processing system (See at least Wang Paragraphs 0066-0067 and Figure 2).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Rose and Bromand, and in further view of Hausman et al (US 20230311335 A1) (Hereinafter referred to as Hausman)
Regarding Claim 4, modified Wang fails to disclose the prerequisite includes information of an environment around the robot.
However, Hausman teaches the prerequisite includes information of an environment around the robot (See at least Hausman Paragraphs 0061-0064, and Figure 2a, the prerequisite includes scene descriptors of an environment around the robot).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Wang with Hausman to have the prerequisite include information of an environment around the robot. This modification, as taught by Hausman, would allow the LLM to utilize the environment information along with the operational procedure to determine the output (least Hausman Paragraphs 0061-0064, and Figure 2a), which would improve the awareness of the natural language processing system.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Rose and Bromand, and in further view of Sun et al (US 20140316570 A1) (Hereinafter referred to as Sun)
Regarding Claim 5, modified Wang fails to disclose the operational procedure is described in Unified Modeling Language.
However, Sun teaches the operational procedure is described in Unified Modeling Language (See at least Sun Paragraph 0030, the task presented with a finite state machine, which is interpreted as an operational procedure, is implemented with unified modeling language).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Wang with Sun to have the operational procedure be described in Unified Modeling Language. This modification, as taught by Sun, would allow the operational procedure to be in a standardized general-purpose modeling language (See at least Sun Paragraph 0030), thus, increasing the applicability of the operational procedure.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sokabe et al (US 20260042206 A1) teaches a robot processing natural language from a user to determine actions
Kollar (US 20230398696 A1) controlling a robot based on natural language input
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ESVINDER SINGH whose telephone number is (571)272-7875. The examiner can normally be reached Monday-Friday: 9 am-5 pm est.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached at 571-270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ESVINDER SINGH/Examiner, Art Unit 3657
Prosecution Insights