DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The amendment filed 11/10/2025 has been entered. Claims 1, 3-4, and 11-12 are amended. Claim 2 is cancelled. Claims 1, 3-8, and 10-15 remain pending in the application. Applicant’s amendments to the claims have overcome each and every 112(b) rejection set forth in the Non-Final Office Action mailed 9/4/2025.
Applicant’s arguments, see pages 13-18, with respect to the previously cited prior art not teaching the amended subject matter has been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Kupcsik (US 20200398427 A1), Bhat (US 20210331316 A1), Wen (US 20190091859 A1), and Seno (US 20190275675 A1).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3-4, and 10-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kupcsik (US 20200398427 A1) in view of Bhat (US 20210331316 A1), Wen (US 20190091859 A1), and Seno (US 20190275675 A1).
Regarding Claim 1,
Kupcsik teaches
A movement planning device comprising: a processor configured to: (“a device for planning a manipulation task of an agent, particularly a robot.” See at least [0026]; “causing the computer to perform the following steps:” See at least claim 14)
acquire task information including information on a start state and a target state of a task given to a robot device; (“A final goal configuration G is provided in step S2 which can be translated into the final state of the end effector 22. … Similarly, the initial configuration of the end-effector 22 can be imposed as the initial observation.” See at least [0063]; “the problem instance P.sub.p needs to be re-constructed whenever a new initial state or goal specification is given.” See at least [0074])
generate an abstract action sequence including one or more abstract actions arranged in an order of execution so as to reach the target state from the start state based on the task information by using a symbolic planner; (“determine a concatenated sequence of manipulation skills selected from the number of learned manipulation skills based on their symbolic abstraction so that a given goal specification indicating a complex manipulation task is satisfied.” See at least [0026]; “Once the domain and problem files are specified, a PDDL (Planning Domain Definition Language.) planner has to find a sequence of actions to fulfill the given goal specification, starting from the initial state.” See at least [0050])
generate a movement sequence including one or more physical actions for performing the abstract actions included in the abstract action sequence in the order of execution by using a motion planner; (“Given this sequence a*.sub.D, each skill within a*.sub.D, is reproduced as the end-effector trajectory level, so to maximize the probability of satisfying the given goal G.” See at least [0076]; “Generate the robot end-effector trajectory that optimally tracks ŝ*.sub.T.sub.D, namely, to reproduce all skills in a*.sub.D. Given the states sequence ŝ*.sub.T.sub.D.” See at least [0081])
output a movement group which includes one or more movement sequences generated using the motion planner (“instruct execution of the sequence of manipulation skills.” See at least [0029]; “a linear quadratic regulator (LQR) can be applied to generate the necessary control commands to reproduce the optimal state sequence ŝ*.sub.T.sub.D. A step-wise reference is obtained from {circumflex over (μ)}.sub.s.sub.t ∀s.sub.t∈ŝ*.sub.T.sub.D, which is tracked by the LQR.sup.1 using associated tracking precision matrices.” See at least [0081])
the processor is further configured to generate the abstract action sequence so that the cost estimated by the cost estimation model is optimized, by using the symbolic planner (“Following, it is referred to the planning and sequencing of trained and abstracted skills. The PDDL definition P has been constructed, which can be directly fed into any compatible PDDL planner. Different optimization techniques can be enforced during the planning, e.g., minimizing the total length of the plan or total cost.” See at least [0075])
Kupcsik does not explicitly teach, but Bhat teaches
determine whether the generated movement sequence is physically executable in a real environment by the robot device; (“prior to executing the sequence of operations, the computing device 102 may perform a collision check analysis for the feasibility of successfully completing the sequence of operations. For example, the computing device 102 may analyze the sequence of operations to determine whether particular workpieces are not accessible (e.g., because other workpieces or equipment block the robotic arm 114 from accessing the workpieces 108). As another example, the computing device 102 may analyze the sequence of operations to determine whether performing particular operations (e.g. moving workpieces 108) will cause the workpieces 108 to collide with other equipment or workpieces.” See at least [0047])
output a movement group which includes one or more movement sequences generated using the motion planner and in which all of the movement sequences that are included are determined to be physically executable (“responsive to determining that the sequence of the operations is feasible: executing the sequence of the operations.” See at least Claim 12, wherein executing the sequence includes outputting the movement group.)
wherein, in a case where the abstract action sequence includes an abstract action that is inexecutable in the real environment and a movement sequence generated for the abstract action is determined to be physically inexecutable, the processor is configured to discard an abstract movement sequence that is after the abstract action, and generate a new abstract action sequence after the abstract action by using the symbolic planner, wherein the new abstract action sequence is partially different from the abstract movement sequence, (“responsive to determining that the sequence of the operations is not feasible: regenerating a new sequence of operations based on the one or more preconditions and the one or more effects of the plurality of motion primitives, the target state and the initial state of the plurality of workpieces.” See at least [0018], wherein regenerating a new sequence involves discarding a previously generated sequence.; “the computing device 102 may analyze the sequence of operations to determine whether performing particular operations (e.g. moving workpieces 108) will cause the workpieces 108 to collide with other equipment or workpieces. If the computing device 102 determines that a collision is likely, or that the sequence of operations is otherwise not feasible, a new sequence of operations may be generated (e.g., by repeating block 428).” See at least [0047], wherein regenerating an entire new sequence includes regenerating the sequence after the particular operation causing the collision.; “the sequence of operations may be identified at least in part using a graph traversal algorithm, such as the A* search algorithm or the like. In particular, such a search may be performed on a database 118 of motion primitives 120 to identify the sequence of operations.” See at least [0046], wherein at least the graph traversal algorithm is a symbolic planner. Alternatively, the symbolic planner of Kupcsik would be used to regenerate the sequence when the prior art is viewed in combination.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik to further include the teachings of Bhat with a reasonable expectation of success to improve safety and efficiency by automatically ensuring that the robotic arm is not impeded in its completion of the task and to “enable a robotic arm to accurately and efficiently construct a series of actions to accomplish new tasks and/or to respond to new environmental variables.” (See at least [0030])
Bhat also does not explicitly teach, but Wen teaches
wherein the symbolic planner includes a cost estimation model trained by machine learning to estimate a cost of an abstract action, (“Methods and systems are disclosed for integration of computer aided design (CAD) models and human expert demonstrations to learn cost function parameters, which are fed to a Deep Inverse Reinforcement Learning (DIRL) algorithm to automatically generate robot control policies.” See at least [0012], wherein the cost function is a cost estimation model.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik and Bhat to further include the teachings of Wen with a reasonable expectation of success to improve programming efficiency, quality, and adaptability. (“Advantages of the proposed system and method are as follows. Programming of robots can be achieved in reduced time, cost and engineering efforts compared with conventional methods. By including contribution from both CAD-based design models and expert demonstration, double safety guarantees can be achieved. Development of higher-quality cost functions translates to robots learning manipulative tasks having higher generalization capability.” See at least [0014])
Wen also does not explicitly teach, but Seno teaches
wherein the cost of the abstract action estimated by the cost estimation model includes a movement time and a drive amount of the robot device, wherein the movement time of the robot device is a period of time required to execute a movement sequence of the robot device, and the drive amount of the robot device in executing the movement sequence is defined by a physical quantity associated with a mechanical driving of the robot device. (“when the second evaluation function “minimization of the moving distance cost of the arm tip” is selected, the trajectory is planned such that the total moving distances of the robot arm tips are minimized. Furthermore, for example, when the third evaluation function “minimization of operating time cost” is selected, the trajectory is planned such that the operating time of the robot is shortest. … The user may simultaneously select a plurality of evaluation functions. Thus, in this case, the trajectory that minimizes the weighted sum of the selected plural evaluation functions is planned.” See at least [0037-0038]) It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik, Bhat, and Wen to further include the teachings of Seno with a reasonable expectation of success for improved efficiency by minimizing moving distance and operating time of the robot. (See at least [0037-0038])
Regarding Claim 3,
Kupcsik does not explicitly teach, but Wen teaches
wherein the processor is further configured to: acquire a plurality of learning data sets each constituted by a combination of a training sample indicating an abstract action for training and a correct answer label indicating a true value of a cost of the abstract action for training; and perform machine learning of the cost estimation model by using the plurality of learning data sets obtained, wherein the machine learning is configured by training the cost estimation model so that an estimated value of a cost for the abstract action for training indicated by the training sample conforms to a true value indicated by the correct answer label for each learning data set. (“Candidate training data inputs related to robotic manipulations, such as trajectories, may be generated by CAD-based simulation engine 111 using parametrized cost functions. Linear cost functions may be used, which define a cost by a linear combination of feature vectors from manual feature engineering or automatically learned from machine learning algorithms. Neural networks approximate nonlinear functions, and are more representative when compared to a linear cost. Cost functions usually trade-off multiple objectives (e.g., between the accuracy and the resource needed to achieve the required accuracy).” See at least [0016]; “as the demonstrator guides the robot, trajectories along a grid space may be recorded in space and time domains, and various parameters related to the task to be learned may be derived from the recorded trajectories. … Human expert demonstration implicitly defines the costs to be optimized, which can be inferred from the trajectories.” See at least [0019]; “At 206, the DIRL engine 115 receives the learned cost functions as evaluation criteria, and executes a DIRL algorithm to select different robot control policies 207.” See at least [0028]; Examiner Interpretation: The implicitly defined costs by the expert demonstrations are equivalent to the correct answer labels indicating a true value of a cost.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik to further include the teachings of Wen with a reasonable expectation of success to improve programming efficiency, quality, and adaptability. (“Advantages of the proposed system and method are as follows. Programming of robots can be achieved in reduced time, cost and engineering efforts compared with conventional methods. By including contribution from both CAD-based design models and expert demonstration, double safety guarantees can be achieved. Development of higher-quality cost functions translates to robots learning manipulative tasks having higher generalization capability.” See at least [0014])
Regarding Claim 4,
Kupcsik does not explicitly teach, but Wen teaches
wherein the correct answer label is configured to indicate a true value of a cost (“Human expert demonstration implicitly defines the costs to be optimized, which can be inferred from the trajectories.” See at least [0019]; Examiner Interpretation: The implicitly defined costs by the expert demonstrations are equivalent to the correct answer labels indicating a true value of a cost.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik to further include the teachings of Wen with a reasonable expectation of success to improve programming efficiency, quality, and adaptability. (“Advantages of the proposed system and method are as follows. Programming of robots can be achieved in reduced time, cost and engineering efforts compared with conventional methods. By including contribution from both CAD-based design models and expert demonstration, double safety guarantees can be achieved. Development of higher-quality cost functions translates to robots learning manipulative tasks having higher generalization capability.” See at least [0014])
Wen also does not explicitly teach, but Seno teaches
calculated in accordance with the period of time required to execute the movement sequence generated by the motion planner for the abstract action for training, and the drive amount of the robot device in executing the movement sequence. (“when the second evaluation function “minimization of the moving distance cost of the arm tip” is selected, the trajectory is planned such that the total moving distances of the robot arm tips are minimized. Furthermore, for example, when the third evaluation function “minimization of operating time cost” is selected, the trajectory is planned such that the operating time of the robot is shortest. … The user may simultaneously select a plurality of evaluation functions. Thus, in this case, the trajectory that minimizes the weighted sum of the selected plural evaluation functions is planned.” See at least [0037-0038]) It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik and Wen to further include the teachings of Seno with a reasonable expectation of success for improved efficiency by minimizing moving distance and operating time of the robot. (See at least [0037-0038])
Regarding Claim 10,
Kupcsik further teaches
wherein the robot device includes one or more robot hands, (“The robot arm 2 is a multi-DoF robotic arm with several links 21 and an end-effector 22.” See at least [0034] and fig. 1 (provided below) wherein the end-effector 22 is a robot hand.)
PNG
media_image1.png
372
454
media_image1.png
Greyscale
and the task is assembling work for a product constituted by one or more parts. (“For example, the skill “insert the peg in the cylinder” involves the objects “peg” and “cylinder.”” See at least [0044]; “In general, the goal specification G represents the desired configuration of the arm and the objects, assumed to be feasible. As an example, one specification could be “within(peg,cylinder)ΛonTop(cylinder, box)”, i.e., “the peg should be inside the cylinder and the cylinder should be on top of the box”.” See at least [0046])
Regarding Claim 11,
Kupcsik teaches
A movement planning method comprising: causing a computer to execute steps as follows, including: (“a method for planning an object manipulation” See at least [0005]; “A computer-implemented method for planning a manipulation task of a robot.” See a least claim 1.)
acquiring task information including information on a start state and a target state of a task given to a robot device, (“A final goal configuration G is provided in step S2 which can be translated into the final state of the end effector 22. … Similarly, the initial configuration of the end-effector 22 can be imposed as the initial observation.” See at least [0063]; “the problem instance P.sub.p needs to be re-constructed whenever a new initial state or goal specification is given.” See at least [0074])
generating an abstract action sequence including one or more abstract actions arranged in an order of execution so as to reach the target state from the start state based on the task information by using a symbolic planner, (“determine a concatenated sequence of manipulation skills selected from the number of learned manipulation skills based on their symbolic abstraction so that a given goal specification indicating a complex manipulation task is satisfied.” See at least [0026]; “Once the domain and problem files are specified, a PDDL (Planning Domain Definition Language.) planner has to find a sequence of actions to fulfill the given goal specification, starting from the initial state.” See at least [0050])
generating a movement sequence including one or more physical actions for performing the abstract actions included in the abstract action sequence in the order of execution by using a motion planner, (“Given this sequence a*.sub.D, each skill within a*.sub.D, is reproduced as the end-effector trajectory level, so to maximize the probability of satisfying the given goal G.” See at least [0076]; “Generate the robot end-effector trajectory that optimally tracks ŝ*.sub.T.sub.D, namely, to reproduce all skills in a*.sub.D. Given the states sequence ŝ*.sub.T.sub.D.” See at least [0081])
outputting a movement group which includes one or more movement sequences generated using the motion planner (“instruct execution of the sequence of manipulation skills.” See at least [0029]; “a linear quadratic regulator (LQR) can be applied to generate the necessary control commands to reproduce the optimal state sequence ŝ*.sub.T.sub.D. A step-wise reference is obtained from {circumflex over (μ)}.sub.s.sub.t ∀s.sub.t∈ŝ*.sub.T.sub.D, which is tracked by the LQR.sup.1 using associated tracking precision matrices.” See at least [0081])
and generating the abstract action sequence so that the cost estimated by the cost estimation model is optimized, by using the symbolic planner (“Following, it is referred to the planning and sequencing of trained and abstracted skills. The PDDL definition P has been constructed, which can be directly fed into any compatible PDDL planner. Different optimization techniques can be enforced during the planning, e.g., minimizing the total length of the plan or total cost.” See at least [0075])
Kupcsik does not explicitly teach, but Bhat teaches
determining whether the generated movement sequence is physically executable in a real environment by the robot device, (“prior to executing the sequence of operations, the computing device 102 may perform a collision check analysis for the feasibility of successfully completing the sequence of operations. For example, the computing device 102 may analyze the sequence of operations to determine whether particular workpieces are not accessible (e.g., because other workpieces or equipment block the robotic arm 114 from accessing the workpieces 108). As another example, the computing device 102 may analyze the sequence of operations to determine whether performing particular operations (e.g. moving workpieces 108) will cause the workpieces 108 to collide with other equipment or workpieces.” See at least [0047])
and outputting a movement group which includes one or more movement sequences generated using the motion planner and in which all of the movement sequences that are included are determined to be physically executable, (“responsive to determining that the sequence of the operations is feasible: executing the sequence of the operations.” See at least Claim 12, wherein executing the sequence includes outputting the movement group.)
wherein in the determining, in a case where the abstract action sequence includes an abstract action that is inexecutable in the real environment and a movement sequence generated for the abstract action is determined to be physically inexecutable, the computer discards an abstract movement sequence that is after the abstract action, and returns to the generating of the abstract action sequence to generate a new abstract action sequence after the abstract action by using the symbolic planner, wherein the new abstract action sequence is partially different from the abstract movement sequence. (“responsive to determining that the sequence of the operations is not feasible: regenerating a new sequence of operations based on the one or more preconditions and the one or more effects of the plurality of motion primitives, the target state and the initial state of the plurality of workpieces.” See at least [0018], wherein regenerating a new sequence involves discarding a previously generated sequence.; “the computing device 102 may analyze the sequence of operations to determine whether performing particular operations (e.g. moving workpieces 108) will cause the workpieces 108 to collide with other equipment or workpieces. If the computing device 102 determines that a collision is likely, or that the sequence of operations is otherwise not feasible, a new sequence of operations may be generated (e.g., by repeating block 428).” See at least [0047], wherein regenerating an entire new sequence includes regenerating the sequence after the particular operation causing the collision.; “the sequence of operations may be identified at least in part using a graph traversal algorithm, such as the A* search algorithm or the like. In particular, such a search may be performed on a database 118 of motion primitives 120 to identify the sequence of operations.” See at least [0046], wherein at least the graph traversal algorithm is a symbolic planner. Alternatively, the symbolic planner of Kupcsik would be used to regenerate the sequence when the prior art is viewed in combination.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik to further include the teachings of Bhat with a reasonable expectation of success to improve safety and efficiency by automatically ensuring that the robotic arm is not impeded in its completion of the task and to “enable a robotic arm to accurately and efficiently construct a series of actions to accomplish new tasks and/or to respond to new environmental variables.” (See at least [0030])
Bhat also does not explicitly teach, but Wen teaches
wherein the symbolic planner includes a cost estimation model trained by machine learning to estimate a cost of an abstract action, (“Methods and systems are disclosed for integration of computer aided design (CAD) models and human expert demonstrations to learn cost function parameters, which are fed to a Deep Inverse Reinforcement Learning (DIRL) algorithm to automatically generate robot control policies.” See at least [0012], wherein the cost function is a cost estimation model.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik and Bhat to further include the teachings of Wen with a reasonable expectation of success to improve programming efficiency, quality, and adaptability. (“Advantages of the proposed system and method are as follows. Programming of robots can be achieved in reduced time, cost and engineering efforts compared with conventional methods. By including contribution from both CAD-based design models and expert demonstration, double safety guarantees can be achieved. Development of higher-quality cost functions translates to robots learning manipulative tasks having higher generalization capability.” See at least [0014])
Wen also does not explicitly teach, but Seno teaches
wherein the cost of the abstract action estimated by the cost estimation model includes a movement time and a drive amount of the robot device, wherein the movement time of the robot device is a period of time required to execute a movement sequence of the robot device, and the drive amount of the robot device in executing the movement sequence is defined by a physical quantity associated with a mechanical driving of the robot device. (“when the second evaluation function “minimization of the moving distance cost of the arm tip” is selected, the trajectory is planned such that the total moving distances of the robot arm tips are minimized. Furthermore, for example, when the third evaluation function “minimization of operating time cost” is selected, the trajectory is planned such that the operating time of the robot is shortest. … The user may simultaneously select a plurality of evaluation functions. Thus, in this case, the trajectory that minimizes the weighted sum of the selected plural evaluation functions is planned.” See at least [0037-0038]) It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik, Bhat, and Wen to further include the teachings of Seno with a reasonable expectation of success for improved efficiency by minimizing moving distance and operating time of the robot. (See at least [0037-0038])
Regarding Claim 12,
Kupcsik teaches
A non-transitory computer readable medium, storing a movement planning program causing a computer to execute steps as follows, (“A non-transitory machine-readable storage medium on which is stored a computer program including a routine of set instructions for planning a manipulation task of a robot, the computer program, when executed by a computer, causing the computer to perform the following steps.” See at least claim 14)
including acquiring task information including information on a start state and a target state of a task given to a robot device, (“A final goal configuration G is provided in step S2 which can be translated into the final state of the end effector 22. … Similarly, the initial configuration of the end-effector 22 can be imposed as the initial observation.” See at least [0063]; “the problem instance P.sub.p needs to be re-constructed whenever a new initial state or goal specification is given.” See at least [0074])
generating an abstract action sequence including one or more abstract actions arranged in an order of execution so as to reach the target state from the start state based on the task information by using a symbolic planner, (“determine a concatenated sequence of manipulation skills selected from the number of learned manipulation skills based on their symbolic abstraction so that a given goal specification indicating a complex manipulation task is satisfied.” See at least [0026]; “Once the domain and problem files are specified, a PDDL (Planning Domain Definition Language.) planner has to find a sequence of actions to fulfill the given goal specification, starting from the initial state.” See at least [0050])
generating a movement sequence including one or more physical actions for performing the abstract actions included in the abstract action sequence in the order of execution by using a motion planner, (“Given this sequence a*.sub.D, each skill within a*.sub.D, is reproduced as the end-effector trajectory level, so to maximize the probability of satisfying the given goal G.” See at least [0076]; “Generate the robot end-effector trajectory that optimally tracks ŝ*.sub.T.sub.D, namely, to reproduce all skills in a*.sub.D. Given the states sequence ŝ*.sub.T.sub.D.” See at least [0081])
outputting a movement group which includes one or more movement sequences generated using the motion planner (“instruct execution of the sequence of manipulation skills.” See at least [0029]; “a linear quadratic regulator (LQR) can be applied to generate the necessary control commands to reproduce the optimal state sequence ŝ*.sub.T.sub.D. A step-wise reference is obtained from {circumflex over (μ)}.sub.s.sub.t ∀s.sub.t∈ŝ*.sub.T.sub.D, which is tracked by the LQR.sup.1 using associated tracking precision matrices.” See at least [0081])
and generating the abstract action sequence so that the cost estimated by the cost estimation model is optimized, by using the symbolic planner (“Following, it is referred to the planning and sequencing of trained and abstracted skills. The PDDL definition P has been constructed, which can be directly fed into any compatible PDDL planner. Different optimization techniques can be enforced during the planning, e.g., minimizing the total length of the plan or total cost.” See at least [0075])
Kupcsik does not explicitly teach, but Bhat teaches
determining whether the generated movement sequence is physically executable in a real environment by the robot device, (“prior to executing the sequence of operations, the computing device 102 may perform a collision check analysis for the feasibility of successfully completing the sequence of operations. For example, the computing device 102 may analyze the sequence of operations to determine whether particular workpieces are not accessible (e.g., because other workpieces or equipment block the robotic arm 114 from accessing the workpieces 108). As another example, the computing device 102 may analyze the sequence of operations to determine whether performing particular operations (e.g. moving workpieces 108) will cause the workpieces 108 to collide with other equipment or workpieces.” See at least [0047])
and outputting a movement group which includes one or more movement sequences generated using the motion planner and in which all of the movement sequences that are included are determined to be physically executable, (“responsive to determining that the sequence of the operations is feasible: executing the sequence of the operations.” See at least Claim 12, wherein executing the sequence includes outputting the movement group.)
wherein in the determining, in a case where the abstract action sequence includes an abstract action that is inexecutable in the real environment and a movement sequence generated for the abstract action is determined to be physically inexecutable, the computer discards an abstract movement sequence that is after the abstract action, and returns to the generating of the abstract action sequence to generate a new abstract action sequence after the abstract action by using the symbolic planner, wherein the new abstract action sequence is partially different from the abstract movement sequence. (“responsive to determining that the sequence of the operations is not feasible: regenerating a new sequence of operations based on the one or more preconditions and the one or more effects of the plurality of motion primitives, the target state and the initial state of the plurality of workpieces.” See at least [0018], wherein regenerating a new sequence involves discarding a previously generated sequence.; “the computing device 102 may analyze the sequence of operations to determine whether performing particular operations (e.g. moving workpieces 108) will cause the workpieces 108 to collide with other equipment or workpieces. If the computing device 102 determines that a collision is likely, or that the sequence of operations is otherwise not feasible, a new sequence of operations may be generated (e.g., by repeating block 428).” See at least [0047], wherein regenerating an entire new sequence includes regenerating the sequence after the particular operation causing the collision.; “the sequence of operations may be identified at least in part using a graph traversal algorithm, such as the A* search algorithm or the like. In particular, such a search may be performed on a database 118 of motion primitives 120 to identify the sequence of operations.” See at least [0046], wherein at least the graph traversal algorithm is a symbolic planner. Alternatively, the symbolic planner of Kupcsik would be used to regenerate the sequence when the prior art is viewed in combination.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik to further include the teachings of Bhat with a reasonable expectation of success to improve safety and efficiency by automatically ensuring that the robotic arm is not impeded in its completion of the task and to “enable a robotic arm to accurately and efficiently construct a series of actions to accomplish new tasks and/or to respond to new environmental variables.” (See at least [0030])
Bhat also does not explicitly teach, but Wen teaches
wherein the symbolic planner includes a cost estimation model trained by machine learning to estimate a cost of an abstract action, (“Methods and systems are disclosed for integration of computer aided design (CAD) models and human expert demonstrations to learn cost function parameters, which are fed to a Deep Inverse Reinforcement Learning (DIRL) algorithm to automatically generate robot control policies.” See at least [0012], wherein the cost function is a cost estimation model.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik and Bhat to further include the teachings of Wen with a reasonable expectation of success to improve programming efficiency, quality, and adaptability. (“Advantages of the proposed system and method are as follows. Programming of robots can be achieved in reduced time, cost and engineering efforts compared with conventional methods. By including contribution from both CAD-based design models and expert demonstration, double safety guarantees can be achieved. Development of higher-quality cost functions translates to robots learning manipulative tasks having higher generalization capability.” See at least [0014])
Wen also does not explicitly teach, but Seno teaches
wherein the cost of the abstract action estimated by the cost estimation model includes a movement time and a drive amount of the robot device, wherein the movement time of the robot device is a period of time required to execute a movement sequence of the robot device, and the drive amount of the robot device in executing the movement sequence is defined by a physical quantity associated with a mechanical driving of the robot device. (“when the second evaluation function “minimization of the moving distance cost of the arm tip” is selected, the trajectory is planned such that the total moving distances of the robot arm tips are minimized. Furthermore, for example, when the third evaluation function “minimization of operating time cost” is selected, the trajectory is planned such that the operating time of the robot is shortest. … The user may simultaneously select a plurality of evaluation functions. Thus, in this case, the trajectory that minimizes the weighted sum of the selected plural evaluation functions is planned.” See at least [0037-0038]) It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of Kupcsik, Bhat, and Wen to further include the teachings of Seno with a reasonable expectation of success for improved efficiency by minimizing moving distance and operating time of the robot. (See at least [0037-0038])
Claim(s) 5 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kupcsik (US 20200398427 A1) in view of Bhat (US 20210331316 A1), Wen (US 20190091859 A1), Seno (US 20190275675 A1), and Murray (US 20200398428 A1).
Regarding Claims 5 and 13,
Kupcsik does not explicitly teach, but Murray teaches
wherein the correct answer label is configured to indicate a true value of a cost calculated in accordance with a probability that the movement sequence generated by the motion planner for the abstract action for training is determined to be physically inexecutable. (“For nodes in the planning graph 300 where there is a probability that direct transition between the nodes will cause a collision with an obstacle, the motion planner (e.g., cost setter 254, FIG. 2) assigns a cost value or weight to the edges of the planning graph 300 transitioning between those nodes (e.g., edges 310a, 310b, 310c, 310d, 310e, 310f, 310g, 310h) indicating the probability of a collision with the obstacle.” See at least [0165], wherein probability of a collision is equivalent to a probability that the movements are physically inexecutable.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik to further include the teachings of Murray with a reasonable expectation of success to improve safety because the “algorithms described herein may, in at least some implementations, guarantee collision-free robot coordination for multiple robots in tight, shared workspaces. Collision-free motion may be guaranteed for all parts (e.g., robotic appendages, end-of-arm tools, end effectors) of the robots, even when operating at high velocity.” (See at least [0009])
Claim(s) 6-7 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kupcsik (US 20200398427 A1) in view of Bhat (US 20210331316 A1), Wen (US 20190091859 A1), Seno (US 20190275675 A1), and Otsuka (US 20190314983 A1).
Regarding Claims 6 and 14,
Kupcsik does not explicitly teach, but Otsuka teaches
wherein the correct answer label is configured to indicate a true value of a cost calculated in accordance with a user's feedback for the abstract action for training. (“The learning section 154 learns an action model for deciding an action of the autonomous mobile object 10 on the basis of environment information of an action environment, and an evaluation value indicating a cost when the autonomous mobile object 10 takes an action in the action environment.” See at least [0123]; “Specifically, the generation section 155 generates a UI screen associated with an evaluation value for each position on an environment map showing the action range of the autonomous mobile object 10. The action range of the autonomous mobile object 10 is a range within which the autonomous mobile object 10 can take an action. The generated UI image is displayed, for example, by the user terminal 20, and receives a user operation such as changing an evaluation value. The decision section 151 decides an action of the autonomous mobile object 10 in the action environment on the basis of the evaluation value input according to a user operation on a UI image.” See at least [0127], wherein a user changing the evaluation value is equivalent to user feedback and the evaluation value indicates the cost for the action.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik to further include the teachings of Otsuka with a reasonable expectation of success to implement user feedback to adjust the robot control because it is desirable for the robot actions reflect a user’s requests (see at least [0126]).
Regarding Claim 7,
Kupcsik does not explicitly teach, but Otsuka teaches
wherein the processor is further configured to output a list of abstract actions included in an abstract action sequence generated using the symbolic planner to the user and to receive the user's feedback for the output list of the abstract actions, wherein the processor is further configured to acquire the learning data set from a result of the user's feedback for the list of the abstract actions. (“The learning section 154 learns an action model for deciding an action of the autonomous mobile object 10 on the basis of environment information of an action environment, and an evaluation value indicating a cost when the autonomous mobile object 10 takes an action in the action environment.” See at least [0123]; “Specifically, the generation section 155 generates a UI screen associated with an evaluation value for each position on an environment map showing the action range of the autonomous mobile object 10. The action range of the autonomous mobile object 10 is a range within which the autonomous mobile object 10 can take an action. The generated UI image is displayed, for example, by the user terminal 20, and receives a user operation such as changing an evaluation value. The decision section 151 decides an action of the autonomous mobile object 10 in the action environment on the basis of the evaluation value input according to a user operation on a UI image. … A UI screen 50 illustrated in FIG. 15 shows that information indicating an evaluation value actually measured at each position in a floor plan of a user's house in which the autonomous mobile object 10 is installed is superimposed and displayed on the position.” See at least [0127-0128], wherein displaying each position is equivalent to outputting a list of the different actions because the robot is controlled to travel to any one of the positions.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik to further include the teachings of Otsuka with a reasonable expectation of success to implement user feedback to adjust the robot control because it is desirable for the robot actions reflect a user’s requests (see at least [0126]).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kupcsik (US 20200398427 A1) in view of Bhat (US 20210331316 A1), Wen (US 20190091859 A1), Seno (US 20190275675 A1), and Rodriguez Garcia (US 20210146532 A1).
Regarding Claim 8,
Kupcsik does not explicitly teach, but Bhat teaches
(“the sequence of operations may be identified at least in part using a graph traversal algorithm, such as the A* search algorithm or the like.” See at least [0046])
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik to further include the teachings of Bhat with a reasonable expectation of success to improve safety and efficiency by automatically ensuring that the robotic arm is not impeded in its completion of the task and to “enable a robotic arm to accurately and efficiently construct a series of actions to accomplish new tasks and/or to respond to new environmental variables.” (See at least [0030])
Bhat does not explicitly teach, but Rodriguez Garcia teaches
wherein a state space of the task is represented by a graph including edges corresponding to abstract actions and nodes corresponding to abstract attributes as targets to be changed by execution of the abstract actions (“A search for a sequence of manipulation primitives may be formulated as a graph search problem. Nodes of a manipulation graph may represent possible object stable placements and edges of the graph may represent manipulation primitive actions transforming the object from one stable placement to another. An algorithm may be used to search for the shortest path within the constructed graph achieving the desired pose to pose reconfiguration. Examples of appropriate algorithms include but are not limited to Dijkstra's algorithm and A* algorithm.” See at least [0029])
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik and Bhat to further include the teachings of Rodriguez Garcia to improve simplicity and efficiency for planning a complex manipulation. (See at least [0028])
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kupcsik (US 20200398427 A1) in view of Bhat (US 20210331316 A1), Wen (US 20190091859 A1), Murray (US 20200398428 A1), and Otsuka (US 20190314983 A1).
Regarding Claim 15,
Kupcsik does not explicitly teach, but Otsuka teaches
wherein the correct answer label is configured to indicate a true value of a cost calculated in accordance with a user's feedback for the abstract action for training. (“The learning section 154 learns an action model for deciding an action of the autonomous mobile object 10 on the basis of environment information of an action environment, and an evaluation value indicating a cost when the autonomous mobile object 10 takes an action in the action environment.” See at least [0123]; “Specifically, the generation section 155 generates a UI screen associated with an evaluation value for each position on an environment map showing the action range of the autonomous mobile object 10. The action range of the autonomous mobile object 10 is a range within which the autonomous mobile object 10 can take an action. The generated UI image is displayed, for example, by the user terminal 20, and receives a user operation such as changing an evaluation value. The decision section 151 decides an action of the autonomous mobile object 10 in the action environment on the basis of the evaluation value input according to a user operation on a UI image.” See at least [0127], wherein a user changing the evaluation value is equivalent to user feedback and the evaluation value indicates the cost for the action.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to modify the teachings of modified Kupcsik to further include the teachings of Otsuka with a reasonable expectation of success to implement user feedback to adjust the robot control because it is desirable for the robot actions reflect a user’s requests (see at least [0126]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Karston G Evans whose telephone number is (571)272-8480. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on (571)270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.G.E./Examiner, Art Unit 3657
/ABBY LIN/Supervisory Patent Examiner, Art Unit 3657