DETAILED ACTION
Status of the Application
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
This action is in response to the applicant’s filing on July 14, 2026. Claims 1, 4, 6, 9, 11 and 14 have been amended, no claims have been canceled, and claim 24 has been added. Thus, claims 1 – 7, 9 – 15 and 17 – 24 are pending and examined below.
Response to Arguments
Applicant’s arguments with respect to claims 1 – 7, 9 – 15 and 17 – 24 have been considered but are moot because the arguments do not apply to the new combination of references used in the current rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4 – 7, 10 – 13, 17, 18, 20 and 21 – 24 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication No. 2020/0376680 A1 to Wang (herein after “Wang publication") in view of U.S. Patent Application Publication No. 2024/0253212 A1 to Rose et al. (herein after “Rose et al. publication").
Note: Text written in bold typeface is claim language from the instant application.
Texts written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
As to claims 1, 20, 21 and 24,
the Wang publication discloses a robot (1, 20, 30, 50)(see ¶23 for “[a] robot system 1 shown in FIG. 1 [including] mobile robots 20, control devices 30”; see also ¶24) comprising:
one or more processors (see ¶23 for “control devices 30” which are known to have one or more processors; see ¶24, where “mobile robots 20 include movable trucks 50” and where “control devices 30 are housed on the insides of the trucks 50 . . . [and] . . . configured to move together with the mobile robots 20”; see also ¶49, where “control device 30 executes [similar to a processor] a work program to thereby output a driving control signal to the truck 50”)(Emphasis added); and
one or more memories storing one or more instructions that, when executed by the one or more processors, cause the one or more processors to:
wherein the robot is configured as an autonomous vehicle configured to move autonomously by executing the program (see ¶49, where “[examples] of the truck 50 include an AMR (Autonomous Mobile Robot) and an AGV (Automated Guided Vehicle)”; see also ¶51, where “[autonomous] movement of the mobile robot 20 according to this embodiment is realized by a publicly-known autonomous movement system included in the truck 50”).
The Wang publication, however, does not specifically disclose
one or more memories storing one or more instructions that, when executed by the one or more processors, cause the one or more processors to:
acquire information from a user;
generate input information for a generative model based on robot information and the acquired information;
acquire a program for controlling an operation of the robot, the program being generated by inputting the input information into the generative model; and
control the operation of the robot by executing the program, wherein the robot information includes at least information related to a software component for acquiring information related to an operating environment of the robot, and wherein the robot is configured as an autonomous vehicle configured to move autonomously by executing the program.
The Rose et al. publication discloses “[robot] control systems, methods, control modules and computer program products that leverage one or more large language model(s) (LLMs) in order to achieve at least some degree of autonomy . . . Robot control parameters and/or instructions may advantageously be specified in natural language (NL) and communicated with the LLM via an NL prompt or query. An NL response from the LLM may then be converted into robot control parameters and/or instructions. In this way, an LLM may be leveraged by the robot control system to enhance the autonomy of various operations and/or functions, including without limitation task planning, motion planning, human interaction, and/or reasoning about the environment.” (See Abstract.)
In particular, the Rose et al. publication disclose
a robot (400)(see FIG. 4 and ¶128 et seq.) comprising:
one or more processors (see ¶128 et seq.);
one or more memories storing one or more instructions that, when executed by the one or more processors, cause the one or more processors to (see ¶128 et seq.):
acquire information from a user (see ¶9 for “capturing . . . sensor data representing information about an environment of the robot body . . . [and] . . . providing a NL query . . . , the NL query including the NL description of at least one aspect of the environment, an NL description of a work objective, an NL description of an instruction set executable by the robot system, and an NL request for a task plan”, all of which are equivalent to acquiring robot information and/or instructional information input by the user)(Emphasis added);
generate input information for a generative model based on robot information and the acquired information (see ¶9 for “generating . . . a natural language (NL) description of at least one aspect of the environment based on the sensor data; providing a NL query to a large language model (LLM) module, the NL query including the NL description of at least one aspect of the environment, an NL description of a work objective, an NL description of an instruction set executable by the robot system, and an NL request for a task plan”)(Emphasis added);
acquire a program for controlling an operation of the robot, the
program being generated by inputting the input information into the generative model (see ¶9, where “receiving the task plan from the LLM module, the task plan expressed in NL; and executing the task plan by the robot system”); and
control the operation of the robot by executing the program (see ¶9),
wherein the robot information includes at least information related to a software component for acquiring information related to an operating environment of the robot (see ¶9 et seq.).
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to modify the Wang publication to acquire information from a user, generate input information for a generative model based on robot information and the acquired information, acquire a program for controlling an operation of the robot, the
program being generated by inputting the input information into the generative model, control the operation of the robot by executing the program, wherein the robot information includes at least information related to a software component for acquiring information related to an operating environment of the robot, as suggested by the Rose et al. publication, in order to deploy, harness, and/or generally use a large language model in the control of a robot to emulate the physical actions of a human or to perform a specific task.
As to claim 2,
according to applicant’s disclosure, “self-recognition information includes, for example, a hardware configuration . . . and the like of the robot . . . [and] the hardware configuration may include a configuration including a sensor, such as an inertial sensor, a 3D sensor, a LiDAR, or an ultrasonic sensor . . . .” (See ¶61 of applicant’s disclosure.)(Emphasis added.)
In ¶47, the Wang publication discloses that “[the] mobile robot 20 includes driving sections including not-shown motors configured to turn . . . the truck 50 and not-shown speed reducers. The mobile robot 20 includes a not-shown position sensor configured to detect rotation angles of rotation axes of the motors or the speed reducers. The driving sections and the position sensors are provided in, for example, the truck 50 . . . The driving sections and the position sensors are respectively communicably coupled to the control device 30.” (Emphasis added.)(See also “1.4 Hardware Configuration of the Robot System”.) Thus, the Wang publication is considered to disclose the robot information including information related to self-recognition of the robot.
As to claim 4,
the Wang publication, as modified by ¶9, ¶11, ¶16 and ¶18 of the Rose et al. publication, discloses the robot information including information related to the operating environment of the robot, and the information related to the operating environment including at least one of map information of the operating environment, sensor information measured in the operating environment, or information related to a target object of the operation of the robot.
As to claim 5,
the modified Wang publication discloses the invention substantially as claimed, except for
the information related to the target object includes information related to at least one of a position of the target object, a name of the target object, a relation between the target object and another target object, a size of the target object, a place where the target object is usually placed, or a handling rule of the target object.
In ¶90 – ¶92 and ¶94, the Rose et al. publication disclose generating NL description of at least one aspect of the environment based on the sensor data and the locations or placement of objects in said environment. Such disclosures suggest the information related to the target object includes information related to at least one of a position of the target object, a name of the target object, a relation between the target object and another target object, a size of the target object, a place where the target object is usually placed, or a handling rule of the target object.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify the Wang publication so that the information related to the target object includes information related to at least one of a position of the target object, a name of the target object, a relation between the target object and another target object, a size of the target object, a place where the target object is usually placed, or a handling rule of the target object, as suggested by the Rose et al. publication, in order to deploy, harness, and/or generally use a large language model in the control of a robot to emulate the physical actions of a human or to perform a specific task.
As to claim 6,
the modified Wang publication discloses the invention substantially as claimed, except for
causing the one or more processors to generate the input information based on information related to the user.
In ¶123, the Rose et al. publication discloses “using NL expressions (descriptions) (e.g., via a NL prompt, which may be entered directly in text by a user or may be spoken vocally by a user and converted to text by an intervening voice-to-text system) to control functions and operations of a robot, where an LLM module may provide an interface between the NL expressions and the robot control system.” (Emphasis added.)
Such disclosure suggests causing the one or more processors to generate the input information based on information related to the user.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify the Wang publication to cause the one or more processors to generate the input information based on information related to the user, as suggested by the Rose et al. publication, in order to deploy, harness, and/or generally use a large language model in the control of a robot to emulate the physical actions of a human or to perform a specific task.
As to claim 7,
the Wang publication, as modified by ¶123 of the Rose et al. publication, is considered to disclose the information related to the user including information related to at least one of a history of the acquired information, a habit of the user, or a preference of the user.
As to claim 10,
the modified Wang publication discloses the invention substantially as claimed, except for
the generation of the program being executed by one or more information processing devices communicably connected to the robot via a network.
In ¶135, the Rose et al. publication discloses that “[in] some implementations where the LLM module is not stored in memory 540, robot body 501 may access the LLM module through transceivers 550 and 581 via wireless signals 570.” Such disclosure suggests the generation of the program being executed by one or more information processing devices communicably connected to the robot via a network.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify the Wang publication so that the generation of the program is executed by one or more information processing devices communicably connected to the robot via a network, as suggested by the Rose et al. publication, in order to deploy, harness, and/or generally use a large language model in the control of a robot to emulate the physical actions of a human or to perform a specific task.
As to claim 11,
the modified Wang publication discloses the invention substantially as claimed, except for
acquiring the information by recognizing a voice of the user.
In ¶123, the Rose et al. publication discloses that “[various] implementations of the present systems, methods, control modules, and computer program products involve using NL expressions (descriptions) (e.g., via a NL prompt, which may be entered directly in text by a user or may be spoken vocally by a user and converted to text by an intervening voice-to-text system) to control functions and operations of a robot, where an LLM module may provide an interface between the NL expressions and the robot control system.”
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify the Wang publication to acquire the information by recognizing a voice of the user, as suggested by the Rose et al. publication, in order to deploy, harness, and/or generally use a large language model in the control of a robot to emulate the physical actions of a human or to perform a specific task.
As to claims 12 and 13,
the Wang publication, as modified by ¶123 of the Rose et al. publication, is considered to disclose the acquired information including instruction information for the robot provided by the user, and wherein the instruction information includes information related to a condition expected by the user.
As to claim 17,
the modified Wang publication discloses the invention substantially as claimed, except for
the robot starts the execution of the program in response to a predetermined trigger.
In ¶9, the Rose et al. publication discloses “providing a NL query to a large language model (LLM) module, the NL query including . . . an NL request for a task plan; receiving the task plan from the LLM module, the task plan expressed in NL; and executing the task plan by the robot system.” (Emphasis added.)
Such disclosure suggests the robot starting the execution of the program in response to a predetermined trigger.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify the Wang publication so that the robot starts the execution of the program in response to a predetermined trigger, as suggested by the Rose et al. publication, in order to deploy, harness, and/or generally use a large language model in the control of a robot to emulate the physical actions of a human or to perform a specific task.
As to claim 18,
the modified Wang publication discloses the invention substantially as claimed, except for
one or more sensors, wherein the robot is configured to acquire desired sensor information by executing the program.
In ¶123, the Rose et al. publication discloses that “certain acts of a method of operation of a robot system may be performed by peripheral components of the robot system that are communicatively coupled to the at least one processor, such as one or more physically actuatable components (e.g., arms, legs, end effectors, grippers, hands), one or more sensors (e.g., optical sensors, audio sensors, tactile sensors, haptic sensors), mobility systems (e.g., wheels, legs), communications and networking hardware (e.g., receivers, transmitters, transceivers), and so on.” (Emphasis added.)
Such disclosure suggests one or more sensors, wherein the robot is configured to acquire desired sensor information by executing the program.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify and/or provide the Wang publication with one or more sensors, wherein the robot is configured to acquire desired sensor information by executing the program, as suggested by the Rose et al. publication, in order to deploy, harness, and/or generally use a large language model in the control of a robot to emulate the physical actions of a human or to perform a specific task.
As to claim 22,
in ¶61 of applicant disclosure, applicant discloses that “self-recognition information includes . . . a basic policy . . . of the robot 200.” In ¶62, applicant discloses that “[the] basic policy is a role, concept, or the like of the robot 200. Examples of the basic policy include supporting the user's life in the home. Other examples of the basic policy may include supporting serving of meals at a restaurant, assisting a person in an office, selecting the shortest route for a given target, avoiding an obstacle if any, and the like.”
The Wang publication discloses that “a gripping hand 67 that grips a work target object is coupled to the distal end of the robot arm 60. The gripping hand 67 is replaceable. A suction hand, a magnetic hand, a screwing tool, an engaging tool, or the like may be used instead of the gripping hand 67.” (See ¶46.)(Emphasis added.) The tool coupled to the distal end of the robot arm 60 is used to manipulate a work target object. Thus, the basic policy, i.e. the role, the concept, or the like of the robot 20, is to emulate the physical actions of a human by manipulating a work target object.
As such, the Wang publication is considered disclose the information related to the self-recognition of the robot including information related to at least one of a role of the robot or a concept of the robot.
As to claim 23,
the Wang publication, as modified by the Rose et al. publication, is considered to disclose the information related to the software component including information related to an application programming interface that acquires data from one or more sensors included in the robot.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over the Wang publication in view of the Rose et al. publication, and further in view of U.S. Patent Application Publication No. 2023/0131458 A1 to Isonni (herein after “Isonni publication").
Note: Text written in bold typeface is claim language from the instant application.
Texts written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
As to claim 3,
the modified Wang publication discloses the invention substantially as claimed, except for
the information related to the software component including an execution condition of the software component.
Employing various APIs provided by the robot to execute not only the primary task but also subtasks for satisfying execution condition of the primary task is old and well-known, as demonstrated by the Isonni publication who discloses “a client device [that] interacts with the robot simulation server 130 through an application programming interface (API) running on a native operating system of the client device . . . .” (See ¶44; see also ¶60, where “the graphics engine includes one or more rendering APIs, including but not limited to Direct3D, OpenGL, Vulkan, DirectX, and the like.”)
Such disclosure suggests the information related to the software component including an execution condition of the software component.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify the Wang publication so that the information related to the software component includes an execution condition of the software component, as suggested by the Isonni publication, in order to allow different software applications to communicate with each other by defining rules for how they can request and exchange data.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over the Wang publication in view of the Rose et al. publication, and further in view of U.S. Patent Application Publication No. 2021/0352380 A1 to Duncan et al. (herein after “Duncan et al. publication").
Note: Text written in bold typeface is claim language from the instant application.
Texts written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
As to claim 9,
the modified the Wang publication discloses the invention substantially as claimed, except for
the at least one processor being configured to: acquire feedback information from the user with respect to a result of the operation of the robot, the operation of the robot being based on execution of the program; and generate second input information for the generation module, based on the feedback information.
Receiving feedback from the user in order to make corrections and/or adjustments is old and well-known, as demonstrated by the Roh et al. publication who discloses a method, apparatus, and system for maintaining and controlling robotic surgical tools. (See Abstract.) According to the Roh et al. publication, “the surgical system of FIG. 6 determines a surgical step to be performed by the surgical robot 602. Determining the surgical step is performed using a machine learning model (see FIG. 2) based on instructions received from a surgeon. For example, the tool control module 628 receives information describing a selected tool from the interface module 624 and receives inputs from a user and provides feedback to the user or performs an action in response to the user's instructions, such as adjusting the configuration of a tool.” (See ¶141.)(Emphasis added.)
Such disclosure suggests the at least one processor being configured to: acquire feedback information from the user with respect to a result of the operation of the robot, the operation of the robot being based on execution of the program; and generate second input information for the generation module, based on the feedback information.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify the Wang publication to acquire feedback information from the user with respect to a result of the operation of the robot, the operation of the robot being based on execution of the program, and generate second input information for the generation module, based on the feedback information, as suggested by the Roh et al. publication, in order to achieve a more favorable output as deemed necessary.
Claims 14 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over the Wang publication in view of the Rose et al. publication, and further in view of U.S. Patent Application Publication No. 2020/0338736 A1 to Hoshiyama (herein after “Hoshiyama publication").
Note: Text written in bold typeface is claim language from the instant application.
Texts written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
As to claims 14 and 15,
the modified the Wang publication discloses the invention substantially as claimed, except for
displaying a screen for editing the program on a display device; and editing the program based on an instruction from the user.
Employing an editing screen is old and well-known, as demonstrated by the Hoshiyama publication who discloses “a program editing section configured to create an editing screen of an operation program for the robot and display the editing screen on the display device”. (See Abstract; see also FIG. 4 and ¶19 et seq.) Such disclosure suggests displaying a screen for editing the program on a display device, and editing the program based on an instruction from the user.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to further modify the Wang publication to display a screen for editing the program on a display device, and edit the program based on an instruction from the user, as suggested by the Hoshiyama publication, in order to make changes to the program as necessary.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over the Wang publication in view of the Rose et al. publication, and further in view of U.S. Patent Application Publication No. 2007/0226949 A1 to Hahm et al. (herein after “Hahm et al. publication").
Note: Text written in bold typeface is claim language from the instant application.
Texts written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
As to claim 19,
the Wang publication discloses the invention substantially as claimed, except for
the robot being an autonomous vehicle configured to dock to a transferred body for transferring.
In FIG. 17 of the Hahm et al. publication, there is a perspective view schematically illustrating a robot (100) configured to dock to a movable docking station (200). In FIGS. 18 and 19, there are side sectional views, respectively, illustrating the configuration of a robot (100) and a docking station (200) of the robot system of FIG. 17. According to the Hahm et al. publication, FIGS 17 – 19 illustrates a configuration of the robot (100) having a movable first docking portion formed with a dust discharge hole and the docking station (200) having a movable second docking portion formed with a dust suction hole. (See ¶120 – ¶121.) When the robot (100) is docked with the docking station (200), the first and second docking portions (150b) and (280) are movable, to facilitate the docking operation. Such disclosure suggests the robot being configured to dock to a transferred body for transferring to facilitate the docking operation.
Based on a reasonable expectation of success, it would have been obvious to one having ordinary skill in the art before the time the invention was filed to modify and/or provide the Wang publication with a dock station so that the autonomous robot is configured to dock to a transferred body for transferring, as suggested by the Hahm et al. publication, in order to deploy the autonomous robot to different location to emulate the physical actions of a human and/or perform a specific task.
Conclusion
Examiner's Note(s): The Examiner has cited particular paragraphs or columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. SEE MPEP 2141.02 [R-07.2015] VI. PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, INCLUDING DISCLOSURES THAT TEACH AWAY FROM THE CLAIMS: A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
In addition, disclosures in a reference must be evaluated for what they would fairly teach one of ordinary skill in the art. See In re Snow, 471 F.2d 1400, 176 USPQ 328 (CCPA 1973) and In re Boe, 355 F.2d 961, 148 USPQ 507 (CCPA 1966). Specifically, in considering the teachings of a reference, it is proper to take into account not only the specific teachings of the reference, but also the inferences that one skilled in the art would reasonably have been expected to draw from the reference. See In re Preda, 401 F.2d 825, 159 USPQ 342 (CCPA 1968) and In re Shepard, 319 F.2d 194, 138 USPQ 148 (CCPA 1963). Likewise, it is proper to take into consideration not only the teachings of the prior art, but also the level of ordinary skill in the art. See In re Luck, 476 F.2d 650, 177 USPQ 523 (CCPA 1973). Specifically, those of ordinary skill in the art are presumed to have some knowledge of the art apart from what is expressly disclosed in the references. See In re Jacoby, 309 F.2d 513, 135 USPQ 317 (CCPA 1962).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RODNEY A. BUTLER whose telephone number is (313)446-6513. The examiner can normally be reached on weekdays, Monday through Friday, between 9 a.m. and 5 p.m.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached on weekdays, Monday through Friday, between 9 a.m. and 5 p.m. at (313) 446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Electronic Communications
Prior to initiating the first e-mail correspondence with any examiner, Applicant is responsible for filing a written statement with the USPTO in accordance with MPEP § 502.03 II. All received e-mail messages including e-mail attachments shall be placed into this application' s record.
/RODNEY A BUTLER/Primary Examiner, Art Unit 3666
