DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Effective Filing Date
Acknowledgement is made of applicant’s claim for foreign priority under 35 USC 119 (a)-(d) to application CN202410145231.5 filed 02/01/2024. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. As such, the effective filing date of the application is 02/01/2024.
Joint Inventors
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Status of Claims
The amendment filed 08/06/2026 has been entered. Claims 1-5, 7-15 and 17-20 have been amended. Claims 1-20 are now pending.
Response to Arguments
Applicant’s arguments with respect to the 35 USC 102 rejections filed in the Non-Final Office Action mailed 05/06/2026 have been fully considered but are moot because amendments to the claim language have necessitated new grounds of rejection set forth below.
Claim Interpretation under 35 USC § 112(f)
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“Gripping apparatus” in claims 1, 5, 6, 11, 15, 16, and 20. Paragraph [0020] of applicant’s specification states “where the gripping apparatus may be a structure with a gripping function, such as a gripper…” The “gripping apparatus” as claimed is therefore being interpreted as “a structure with a gripping function, such as a gripper.”
“Moving apparatus” in claims 1, 5-7, 11, 15-17, and 20. Paragraph [0020] of applicant’s specification states “the moving apparatus may refer to the bottom wheels of the intelligent robot…” The “moving apparatus” as claimed is therefore being interpreted as “bottom wheels of the intelligent robot.”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Park et al. (KR 20200133705 A), hereinafter Park, in view of Oleynik (US 20230031545 A1), hereinafter Oleynik.
Regarding claim 1, Park discloses:
A method of task execution, applicable to an intelligent robot, the method comprising:
obtaining a task (see at least [0036]: “For example, "transfer object A to user 1"-The work command may consist of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command “transfer object A to user 1”, the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.”)
determining object information required for completing the task based on the task, the object information comprising at least one object (see at least [0036]: “For example, "transfer object A to user 1"-The work command may consist of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command “transfer object A to user 1”, the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.”)
generating a plurality of sub-tasks to be executed for completing the task based on the object information, each sub-task comprising operation action information for a corresponding object (see at least [0041]: “The work command inference unit 120 may determine the work structure of the work command by further considering robot information, object information, human information, and the like. That is, the work command inference unit 120 may additionally generate a sub task required to perform the work command by further considering robot information, object information, human information, and the like. As shown in FIG. 3, the work command provided from the work command receiving unit 110 is "transfer object A to user 1", or object B is selected in consideration of information on an object located under object B. You can create additional subtasks to move. That is, the work command inference unit 120 considers information about a robot, an object, a human, and the like, and performs a sub-task for performing a work command that transmits the object A to the user 1 "Lifting object B, putting down object B, and Lifting A, passing object A to User 1, and “GoToHome” can be used.”)
and controlling a gripping apparatus and a moving apparatus to collaborate to execute the plurality of sub-tasks to complete the task (see at least [0048]: “According to the robot motion command, which is a combination of interaction types, the robot can perform the entire operation, and the object, human, and current information of the robot reflecting the result of the operation of the robot can be fed back to the database unit 130 and stored. And, it can be used to dynamically create an interaction type corresponding to the next work command.”)
Park does not explicitly disclose, but Oleynik, in the analogous endeavor of robotics teaches:
wherein the at least one object comprises at least one material or at least one tool to be utilized for completing the task, wherein determining the object information comprises determining the at least one material or the at least one tool based on a predetermined correspondence relationship table, and wherein the predetermined corresponding relationship table comprises a plurality of tasks and corresponding object information (see figures 29 and 30 which depict relationships between work objects or tools and given tasks or manipulations to be made based on the given work object or tool. See further [0060]-[0061]: “FIG. 29 is a table illustrating a database library structure of minimanipulations objects for use in the standardized robotic kitchen in accordance with the present disclosure. FIG. 30 is a table illustrating a database library structure of standardized objects for use in the standardized robotic kitchen in accordance with the present disclosure.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Park with the object relationship chart (described as a minimanipulation library) within the Oleynik reference. This is because, as stated in Oleynik [0026]: “Embodiments of the present disclosure are directed to the technical features relating to the ability of being able to create complex robotic humanoid movements, actions, and interactions with tools and the instrumented environment by automatically building movements for the humanoid; actions and behaviors of the humanoid based on a set of computer-encoded robotic movement and action primitives. The primitives are defined by motions/actions of articulated degrees of freedom that range in complexity from simple to complex, and which can be combined in any form in serial/parallel fashion. These motion-primitives are termed to be minimanipulations and each has a clear time-indexed command input-structure and output behavior/performance profile that is intended to achieve a certain function. Minimanipulations comprise a new way of creating a general programmable-by-example platform for humanoid robots. One or more minimanipulation electronic libraries provide a large suite of higher-level sensing-and-execution sequences that are common building blocks for complex tasks, such as cooking, taking care of the infirm, or other tasks performed by the next generation of humanoid robots.”
Regarding claim 2, the combination of Park and Oleynik teaches:
The method according to claim 1, further comprising: obtaining requirement information input by a related person (see at least [0034]: “The work order may be an order provided by a human. The work command may be provided through an input means such as a keyboard or a mouse, but is not limited thereto, and may be generated by recognizing a human voice or a human motion. The work command receiving unit 110 may include at least one of an input module for inputting a work command, a voice recognition module for recognizing a human voice, and a motion recognition module for recognizing a human motion. The work command received by the work command receiving unit 110 is provided to the work command inferring unit 120.”)
and determining the task based on the requirement information (see at least [0034]: “The work order may be an order provided by a human. The work command may be provided through an input means such as a keyboard or a mouse, but is not limited thereto, and may be generated by recognizing a human voice or a human motion. The work command receiving unit 110 may include at least one of an input module for inputting a work command, a voice recognition module for recognizing a human voice, and a motion recognition module for recognizing a human motion. The work command received by the work command receiving unit 110 is provided to the work command inferring unit 120.”)
Regarding claim 3, the combination of Park and Oleynik teaches:
The method according to claim 1, further comprising:
obtaining an execution process for completing the task; and presenting the object information and the execution process (see at least [0035]: “The work command inference unit 120 infers the work command and determines a work structure (Task Tree). The work order consists of at least one sub-task, and each sub-task consists of at least one or more interaction types. The work command inference unit 120 determines at least one or more sub-tasks constituting the work command, and determines at least one or more interaction types required to perform the determined sub-task.”)
Regarding claim 4, the combination of Park and Oleynik teaches:
The method according to claim 1, wherein generating the plurality of sub-tasks to be executed for completing the task based on the object information comprises:
obtaining a plurality of predetermined action operation templates (see at least [0035]: “The work command inference unit 120 infers the work command and determines a work structure (Task Tree). The work order consists of at least one sub-task, and each sub-task consists of at least one or more interaction types. The work command inference unit 120 determines at least one or more sub-tasks constituting the work command, and determines at least one or more interaction types required to perform the determined sub-task.”)
and inputting the plurality of action operation templates, the object information, and the task into a predetermined sub-task generation model to obtain the plurality of sub-tasks to be executed for completing the task (see at least [0036]: “For example, "transfer object A to user 1"-The work command may consist of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command “transfer object A to user 1”, the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.”)
Regarding claim 5, the combination of Park and Oleynik teaches:
The method according to claim 1, wherein controlling the gripping apparatus and the moving apparatus to collaborate to execute the plurality of sub-tasks to complete the task comprises:
for each sub-task of the plurality of sub-tasks, determining moving information of the moving apparatus and gripping information of the gripping apparatus based on the sub-task (see at least [0046]: “The robot motion command generator 140 may determine an order of robot tasks so that the robot performs a plurality of dynamically generated interaction types, and may generate a robot motion command. The robot motion command generation unit 140 may plan the motion of the robot based on the dynamically generated interaction type, and may plan a process in which each interaction type is performed as a whole. The robot motion command generation unit 140 may provide the generated robot motion command to the robot.”)
and executing the sub-task based on the moving information and the gripping information to complete the task (see at least [0048]: “According to the robot motion command, which is a combination of interaction types, the robot can perform the entire operation, and the object, human, and current information of the robot reflecting the result of the operation of the robot can be fed back to the database unit 130 and stored. And, it can be used to dynamically create an interaction type corresponding to the next work command.”)
Regarding claim 6, the combination of Park and Oleynik teaches:
The method according to claim 5, wherein executing the sub-task based on the moving information and the gripping information comprises:
controlling the moving apparatus to move based on the moving information, causing the moving apparatus to move to a corresponding object of the sub-task (see at least [0046]: “The robot motion command generator 140 may determine an order of robot tasks so that the robot performs a plurality of dynamically generated interaction types, and may generate a robot motion command. The robot motion command generation unit 140 may plan the motion of the robot based on the dynamically generated interaction type, and may plan a process in which each interaction type is performed as a whole. The robot motion command generation unit 140 may provide the generated robot motion command to the robot.”)
and controlling the gripping apparatus to grip the object based on the gripping information to execute the sub-task (see at least [0048]: “According to the robot motion command, which is a combination of interaction types, the robot can perform the entire operation, and the object, human, and current information of the robot reflecting the result of the operation of the robot can be fed back to the database unit 130 and stored. And, it can be used to dynamically create an interaction type corresponding to the next work command.”)
Regarding claim 7, the combination of Park and Oleynik teaches:
The method according to claim 6, wherein the moving information comprises:
a location of the corresponding object of the sub-task and an initial location of the moving apparatus (See at least [0044]: “As another example, even if the interaction type corresponding to the same MoveTo, the moving direction and speed may be different depending on the object and the object. In addition, when there is an obstacle that is expected to collide in the moving process, MovoTo must be created to avoid the obstacle. In addition, when the object A is handed to the user, the HandOver interaction type may be changed according to the location where the interaction with the human is performed. That is, interaction types such as MoveTo and HandOver must be dynamically generated to have different parameter values according to the work command and the surrounding environment. Accordingly, the work command inference unit 120 reads out information and an interaction type interface stored in the database unit 130, such as robots, objects, and humans, and provides a plurality of interactions constituting the currently provided work command. Types can be created dynamically.”)
and controlling the moving apparatus to move based on the moving information comprises: controlling the moving apparatus to move from the initial location to the location of the corresponding object of the sub-task (See at least [0044]: “As another example, even if the interaction type corresponding to the same MoveTo, the moving direction and speed may be different depending on the object and the object. In addition, when there is an obstacle that is expected to collide in the moving process, MovoTo must be created to avoid the obstacle. In addition, when the object A is handed to the user, the HandOver interaction type may be changed according to the location where the interaction with the human is performed. That is, interaction types such as MoveTo and HandOver must be dynamically generated to have different parameter values according to the work command and the surrounding environment. Accordingly, the work command inference unit 120 reads out information and an interaction type interface stored in the database unit 130, such as robots, objects, and humans, and provides a plurality of interactions constituting the currently provided work command. Types can be created dynamically.”)
Regarding claim 8, the combination of Park and Oleynik teaches:
The method according to claim 1, wherein determining the object information required for completing the task based on the task comprises:
obtaining user feature information of a related person and determining the object information required for completing the task based on the user feature information and the task (see at least [0040]: “The robot information refers to information related to a robot performing a task, and includes at least information on the position of the robot. Here, the object is an object included in an environment in which the robot and the human belong, and means an object capable of interacting with the robot and/or human. The object information includes location information of all objects located in the environment and physical information such as size and weight. The human information refers to information on the robot and a human capable of interacting with an object, and includes at least human location information and a location where the interaction with the human is performed (eg, the location of a hand).”)
Regarding claim 9, the combination of Park and Oleynik teaches:
The method according to claim 8, wherein determining the object information required for completing the task based on the user feature information and the task comprises:
determining an object preference of a user based on the user feature information and the task and determining the object information required for completing the task based on the object preference (see at least [0041]: “The work command inference unit 120 may determine the work structure of the work command by further considering robot information, object information, human information, and the like. That is, the work command inference unit 120 may additionally generate a sub task required to perform the work command by further considering robot information, object information, human information, and the like. As shown in FIG. 3, the work command provided from the work command receiving unit 110 is "transfer object A to user 1", or object B is selected in consideration of information on an object located under object B. You can create additional subtasks to move. That is, the work command inference unit 120 considers information about a robot, an object, a human, and the like, and performs a sub-task for performing a work command that transmits the object A to the user 1 "Lifting object B, putting down object B, and Lifting A, passing object A to User 1, and “GoToHome” can be used.”)
Regarding claim 10, the combination of Park and Oleynik teaches:
The method according to claim 1, wherein the task is based on an instruction input by a related person (see at least [0034]: “The work order may be an order provided by a human. The work command may be provided through an input means such as a keyboard or a mouse, but is not limited thereto, and may be generated by recognizing a human voice or a human motion. The work command receiving unit 110 may include at least one of an input module for inputting a work command, a voice recognition module for recognizing a human voice, and a motion recognition module for recognizing a human motion. The work command received by the work command receiving unit 110 is provided to the work command inferring unit 120.”)
Regarding claim 11, Park discloses:
An electronic device, comprising:
a processor and a memory configured to store executable instructions of the processor (see at least [0031]: “The interaction system 10 for human-robot interaction according to the embodiments and each device or unit constituting the same may have an aspect that is entirely hardware or partially hardware and partially software. For example, each component of the interaction system 10 for human-robot interaction refers to a combination of hardware and software driven by the hardware. The hardware may be a data processing device including a CPU (Central Processing Unit) or another processor. In addition, software driven by hardware may refer to an executing process, an object, an executable file, a thread of execution, a program, and the like. For example, the work command inference unit 120 may refer to a combination of hardware for inferring a work command and software for the same.”)
wherein the processor is configured to perform acts comprising:
obtaining a task (see at least [0036]: “For example, "transfer object A to user 1"-The work command may consist of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command “transfer object A to user 1”, the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.”)
determining object information required for completing the task based on the task, the object information comprising at least one object (see at least [0036]: “For example, "transfer object A to user 1"-The work command may consist of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command “transfer object A to user 1”, the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.”)
generating a plurality of sub-tasks to be executed for completing the task based on the object information, each sub-task comprising operation action information for a corresponding object (see at least [0041]: “The work command inference unit 120 may determine the work structure of the work command by further considering robot information, object information, human information, and the like. That is, the work command inference unit 120 may additionally generate a sub task required to perform the work command by further considering robot information, object information, human information, and the like. As shown in FIG. 3, the work command provided from the work command receiving unit 110 is "transfer object A to user 1", or object B is selected in consideration of information on an object located under object B. You can create additional subtasks to move. That is, the work command inference unit 120 considers information about a robot, an object, a human, and the like, and performs a sub-task for performing a work command that transmits the object A to the user 1 "Lifting object B, putting down object B, and Lifting A, passing object A to User 1, and “GoToHome” can be used.”)
and controlling a gripping apparatus and a moving apparatus to collaborate to execute the plurality of sub-tasks to complete the task (see at least [0048]: “According to the robot motion command, which is a combination of interaction types, the robot can perform the entire operation, and the object, human, and current information of the robot reflecting the result of the operation of the robot can be fed back to the database unit 130 and stored. And, it can be used to dynamically create an interaction type corresponding to the next work command.”)
Park does not explicitly disclose, but Oleynik, in the analogous endeavor of robotics teaches:
wherein the at least one object comprises at least one material or at least one tool to be utilized for completing the task, wherein determining the object information comprises determining the at least one material or the at least one tool based on a predetermined correspondence relationship table, and wherein the predetermined corresponding relationship table comprises a plurality of tasks and corresponding object information (see figures 29 and 30 which depict relationships between work objects or tools and given tasks or manipulations to be made based on the given work object or tool. See further [0060]-[0061]: “FIG. 29 is a table illustrating a database library structure of minimanipulations objects for use in the standardized robotic kitchen in accordance with the present disclosure. FIG. 30 is a table illustrating a database library structure of standardized objects for use in the standardized robotic kitchen in accordance with the present disclosure.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Park with the object relationship chart (described as a minimanipulation library) within the Oleynik reference. This is because, as stated in Oleynik [0026]: “Embodiments of the present disclosure are directed to the technical features relating to the ability of being able to create complex robotic humanoid movements, actions, and interactions with tools and the instrumented environment by automatically building movements for the humanoid; actions and behaviors of the humanoid based on a set of computer-encoded robotic movement and action primitives. The primitives are defined by motions/actions of articulated degrees of freedom that range in complexity from simple to complex, and which can be combined in any form in serial/parallel fashion. These motion-primitives are termed to be minimanipulations and each has a clear time-indexed command input-structure and output behavior/performance profile that is intended to achieve a certain function. Minimanipulations comprise a new way of creating a general programmable-by-example platform for humanoid robots. One or more minimanipulation electronic libraries provide a large suite of higher-level sensing-and-execution sequences that are common building blocks for complex tasks, such as cooking, taking care of the infirm, or other tasks performed by the next generation of humanoid robots.”
Regarding claim 12, the combination of Park and Oleynik teaches:
The electronic device according to claim 11, the acts further comprise: obtaining requirement information input by a related person (see at least [0034]: “The work order may be an order provided by a human. The work command may be provided through an input means such as a keyboard or a mouse, but is not limited thereto, and may be generated by recognizing a human voice or a human motion. The work command receiving unit 110 may include at least one of an input module for inputting a work command, a voice recognition module for recognizing a human voice, and a motion recognition module for recognizing a human motion. The work command received by the work command receiving unit 110 is provided to the work command inferring unit 120.”)
and determining the task based on the requirement information (see at least [0034]: “The work order may be an order provided by a human. The work command may be provided through an input means such as a keyboard or a mouse, but is not limited thereto, and may be generated by recognizing a human voice or a human motion. The work command receiving unit 110 may include at least one of an input module for inputting a work command, a voice recognition module for recognizing a human voice, and a motion recognition module for recognizing a human motion. The work command received by the work command receiving unit 110 is provided to the work command inferring unit 120.”)
Regarding claim 13, the combination of Park and Oleynik teaches:
The electronic device according to claim 11, the acts further comprise:
obtaining an execution process for completing the task; and presenting the object information and the execution process (see at least [0035]: “The work command inference unit 120 infers the work command and determines a work structure (Task Tree). The work order consists of at least one sub-task, and each sub-task consists of at least one or more interaction types. The work command inference unit 120 determines at least one or more sub-tasks constituting the work command, and determines at least one or more interaction types required to perform the determined sub-task.”)
Regarding claim 14, the combination of Park and Oleynik teaches:
The electronic device according to claim 11, wherein generating the plurality of sub-tasks to be executed for completing the task based on the object information comprises:
obtaining a plurality of predetermined action operation templates (see at least [0035]: “The work command inference unit 120 infers the work command and determines a work structure (Task Tree). The work order consists of at least one sub-task, and each sub-task consists of at least one or more interaction types. The work command inference unit 120 determines at least one or more sub-tasks constituting the work command, and determines at least one or more interaction types required to perform the determined sub-task.”)
and inputting the plurality of action operation templates, the object information, and the task into a predetermined sub-task generation model to obtain the plurality of sub-tasks to be executed for completing the task (see at least [0036]: “For example, "transfer object A to user 1"-The work command may consist of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command “transfer object A to user 1”, the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.”)
Regarding claim 15, the combination of Park and Oleynik teaches:
The electronic device according to claim 11, wherein controlling the gripping apparatus and the moving apparatus to collaborate to execute the plurality of sub-tasks to complete the task comprises:
for each sub-task of the plurality of sub-tasks, determining moving information of the moving apparatus and gripping information of the gripping apparatus based on the sub-task (see at least [0046]: “The robot motion command generator 140 may determine an order of robot tasks so that the robot performs a plurality of dynamically generated interaction types, and may generate a robot motion command. The robot motion command generation unit 140 may plan the motion of the robot based on the dynamically generated interaction type, and may plan a process in which each interaction type is performed as a whole. The robot motion command generation unit 140 may provide the generated robot motion command to the robot.”)
and executing the sub-task based on the moving information and the gripping information to complete the task (see at least [0048]: “According to the robot motion command, which is a combination of interaction types, the robot can perform the entire operation, and the object, human, and current information of the robot reflecting the result of the operation of the robot can be fed back to the database unit 130 and stored. And, it can be used to dynamically create an interaction type corresponding to the next work command.”)
Regarding claim 16, the combination of Park and Oleynik teaches:
The electronic device according to claim 15, wherein executing the sub-task based on the moving information and the gripping information comprises:
controlling the moving apparatus to move based on the moving information, causing the moving apparatus to move to a corresponding object of the sub-task (see at least [0046]: “The robot motion command generator 140 may determine an order of robot tasks so that the robot performs a plurality of dynamically generated interaction types, and may generate a robot motion command. The robot motion command generation unit 140 may plan the motion of the robot based on the dynamically generated interaction type, and may plan a process in which each interaction type is performed as a whole. The robot motion command generation unit 140 may provide the generated robot motion command to the robot.”)
and controlling the gripping apparatus to grip the object based on the gripping information to execute the sub-task (see at least [0048]: “According to the robot motion command, which is a combination of interaction types, the robot can perform the entire operation, and the object, human, and current information of the robot reflecting the result of the operation of the robot can be fed back to the database unit 130 and stored. And, it can be used to dynamically create an interaction type corresponding to the next work command.”)
Regarding claim 17, the combination of Park and Oleynik teaches:
The electronic device according to claim 16, wherein the moving information comprises:
a location of the corresponding object of the sub-task and an initial location of the moving apparatus (See at least [0044]: “As another example, even if the interaction type corresponding to the same MoveTo, the moving direction and speed may be different depending on the object and the object. In addition, when there is an obstacle that is expected to collide in the moving process, MovoTo must be created to avoid the obstacle. In addition, when the object A is handed to the user, the HandOver interaction type may be changed according to the location where the interaction with the human is performed. That is, interaction types such as MoveTo and HandOver must be dynamically generated to have different parameter values according to the work command and the surrounding environment. Accordingly, the work command inference unit 120 reads out information and an interaction type interface stored in the database unit 130, such as robots, objects, and humans, and provides a plurality of interactions constituting the currently provided work command. Types can be created dynamically.”)
and controlling the moving apparatus to move based on the moving information comprises: controlling the moving apparatus to move from the initial location to the location of the corresponding object of the sub-task (See at least [0044]: “As another example, even if the interaction type corresponding to the same MoveTo, the moving direction and speed may be different depending on the object and the object. In addition, when there is an obstacle that is expected to collide in the moving process, MovoTo must be created to avoid the obstacle. In addition, when the object A is handed to the user, the HandOver interaction type may be changed according to the location where the interaction with the human is performed. That is, interaction types such as MoveTo and HandOver must be dynamically generated to have different parameter values according to the work command and the surrounding environment. Accordingly, the work command inference unit 120 reads out information and an interaction type interface stored in the database unit 130, such as robots, objects, and humans, and provides a plurality of interactions constituting the currently provided work command. Types can be created dynamically.”)
Regarding claim 18, the combination of Park and Oleynik teaches:
The electronic device according to claim 11, wherein determining the object information required for completing the task based on the task comprises:
obtaining user feature information of a related person and determining the object information required for completing the task based on the user feature information and the task (see at least [0040]: “The robot information refers to information related to a robot performing a task, and includes at least information on the position of the robot. Here, the object is an object included in an environment in which the robot and the human belong, and means an object capable of interacting with the robot and/or human. The object information includes location information of all objects located in the environment and physical information such as size and weight. The human information refers to information on the robot and a human capable of interacting with an object, and includes at least human location information and a location where the interaction with the human is performed (eg, the location of a hand).”)
Regarding claim 19, the combination of Park and Oleynik teaches:
The electronic device according to claim 18, wherein determining the object information required for completing the task based on the user feature information and the task comprises:
determining an object preference of a user based on the user feature information and the task and determining the object information required for completing the task based on the object preference (see at least [0041]: “The work command inference unit 120 may determine the work structure of the work command by further considering robot information, object information, human information, and the like. That is, the work command inference unit 120 may additionally generate a sub task required to perform the work command by further considering robot information, object information, human information, and the like. As shown in FIG. 3, the work command provided from the work command receiving unit 110 is "transfer object A to user 1", or object B is selected in consideration of information on an object located under object B. You can create additional subtasks to move. That is, the work command inference unit 120 considers information about a robot, an object, a human, and the like, and performs a sub-task for performing a work command that transmits the object A to the user 1 "Lifting object B, putting down object B, and Lifting A, passing object A to User 1, and “GoToHome” can be used.”)
Regarding claim 20, Park discloses:
A non-transitory computer-readable storage medium having a computer program stored thereon, wherein, the computer program when executed by a processor implements acts comprising:
obtaining a task (see at least [0036]: “For example, "transfer object A to user 1"-The work command may consist of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command “transfer object A to user 1”, the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.”)
determining object information required for completing the task based on the task, the object information comprising at least one object (see at least [0036]: “For example, "transfer object A to user 1"-The work command may consist of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command “transfer object A to user 1”, the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.”)
generating a plurality of sub-tasks to be executed for completing the task based on the object information, each sub-task comprising operation action information for a corresponding object (see at least [0041]: “The work command inference unit 120 may determine the work structure of the work command by further considering robot information, object information, human information, and the like. That is, the work command inference unit 120 may additionally generate a sub task required to perform the work command by further considering robot information, object information, human information, and the like. As shown in FIG. 3, the work command provided from the work command receiving unit 110 is "transfer object A to user 1", or object B is selected in consideration of information on an object located under object B. You can create additional subtasks to move. That is, the work command inference unit 120 considers information about a robot, an object, a human, and the like, and performs a sub-task for performing a work command that transmits the object A to the user 1 "Lifting object B, putting down object B, and Lifting A, passing object A to User 1, and “GoToHome” can be used.”)
and controlling a gripping apparatus and a moving apparatus to collaborate to execute the plurality of sub-tasks to complete the task (see at least [0048]: “According to the robot motion command, which is a combination of interaction types, the robot can perform the entire operation, and the object, human, and current information of the robot reflecting the result of the operation of the robot can be fed back to the database unit 130 and stored. And, it can be used to dynamically create an interaction type corresponding to the next work command.”)
Park does not explicitly disclose, but Oleynik, in the analogous endeavor of robotics teaches:
wherein the at least one object comprises at least one material or at least one tool to be utilized for completing the task, wherein determining the object information comprises determining the at least one material or the at least one tool based on a predetermined correspondence relationship table, and wherein the predetermined corresponding relationship table comprises a plurality of tasks and corresponding object information (see figures 29 and 30 which depict relationships between work objects or tools and given tasks or manipulations to be made based on the given work object or tool. See further [0060]-[0061]: “FIG. 29 is a table illustrating a database library structure of minimanipulations objects for use in the standardized robotic kitchen in accordance with the present disclosure. FIG. 30 is a table illustrating a database library structure of standardized objects for use in the standardized robotic kitchen in accordance with the present disclosure.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Park with the object relationship chart (described as a minimanipulation library) within the Oleynik reference. This is because, as stated in Oleynik [0026]: “Embodiments of the present disclosure are directed to the technical features relating to the ability of being able to create complex robotic humanoid movements, actions, and interactions with tools and the instrumented environment by automatically building movements for the humanoid; actions and behaviors of the humanoid based on a set of computer-encoded robotic movement and action primitives. The primitives are defined by motions/actions of articulated degrees of freedom that range in complexity from simple to complex, and which can be combined in any form in serial/parallel fashion. These motion-primitives are termed to be minimanipulations and each has a clear time-indexed command input-structure and output behavior/performance profile that is intended to achieve a certain function. Minimanipulations comprise a new way of creating a general programmable-by-example platform for humanoid robots. One or more minimanipulation electronic libraries provide a large suite of higher-level sensing-and-execution sequences that are common building blocks for complex tasks, such as cooking, taking care of the infirm, or other tasks performed by the next generation of humanoid robots.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Heckmann et al. (EP 3591521 A1), which discloses:
“A system for assisting a user in fulfilling a task, the system comprises a human interface unit for communicating with the user, a task input unit configured to obtain unstructured knowledge source data on the task, and a processor. The processor interprets a user input obtained by the human interface unit. The processor further analyzes the obtained unstructured knowledge source data for generating an internal representation of the task and monitors a task progress in performing the task by interpreting at least one of the user input and image data. The processor generates a support signal based on the generated internal representation and the monitored task progress and outputs the generated support signal, wherein the support signal comprises information on manipulating at least one object or information how to perform the task.”
Fang et al. (US 11256539 B2), which discloses:
“A task processing method, apparatus, and system based on a distributed system. The method comprises: obtaining, by a task processing device, task information from a master control device by initiating a task request, wherein the task information corresponding to the task request is stored in the master control device; starting, by a main process of the task processing device, a corresponding task subprocess by triggering a proxy process for executing the task information after receiving the task information; and storing, by the task processing device, task execution information to a target location through the proxy process, wherein the task execution information comprises one or more of the following pieces of information: a task identifier of the task information, process information of the task subprocess, log information and a return code generated when the task subprocess executes the task information. The present invention solves the main process of the task processing device monitors, according to the task identifier, whether the task information is started and executed, and monitor, according to the return code, whether the task information is fully executed.”
Johnson et al. (US 20200087069 A1), which discloses:
“In an embodiment, a method for handling an order includes determining a plurality of ingredients based on an order, received from a user over a network, for a location having a plurality of robots. The method further includes planning at least one trajectory for at least one robot based on the plurality of ingredients and utensils available at the location, and proximity of each ingredient and utensil to the at least one robot. Each trajectory can be configured to move one of the plurality of ingredients into a container associated with the order. In an embodiment, the method includes executing the at least one trajectory by the at least one robot to fulfill the order. In an embodiment, the method includes moving the container to a pickup area.”
Zheng et al. (US 20220184808 A1), which discloses:
“A motion trajectory planning method for a robotic manipulator having a visual inspection system, includes: in response to a command instruction, obtaining environmental data collected by the visual inspection system; determining an initial DS model motion trajectory of the robotic manipulator according to the command instruction, the environmental data, and a preset teaching motion DS model library, wherein the teaching motion DS model library includes at least one DS model motion trajectory generated based on human teaching activities; and at least based on a result of determining whether there is an obstacle, whose pose is on the initial DS model motion trajectory, in a first object included in the environmental data, correcting the initial DS model motion trajectory to obtain a desired motion trajectory of the robotic manipulator.”
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ELIZABETH NELESKI whose telephone number is (571)272-6064. The examiner can normally be reached 10 - 6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THOMAS WORDEN can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.R.N./Examiner, Art Unit 3658
/THOMAS E WORDEN/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights