DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Pursuant to communications filed on 31 March 2025, this is a First Action Non-Final Rejection on the Merits. Claims 1-20 are currently pending in the instant application.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01 July 2025 (2) are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) have been considered by the Examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8, 11-17 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, a computer-implemented method for training a machine learning model to control a robot, the method comprising:
performing, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot to perform a different assembly task; and
performing, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks.
Step 1: Statutory Category – Yes.
The claim(s) recite(s) a method (i.e. process), therefore the claim(s) fall within one of the four statutory categories. MPEP 2106.03.
Step 2A, Prong One evaluation: Judicial Exception – Yes.
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under the broadest reasonable interpretation, the claim covers performance using mental processes.
The claim recites the limitation of “performing, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot to perform a different assembly task” in the context of this claim is an abstract idea, wherein a human mentally observes and/or evaluates “one or more first training operations”. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind.
The claim additionally recites the limitation of “performing, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks” in the context of this claim is an abstract idea, wherein a human mentally observes and/or evaluates “one or more second training operations”. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind.
As provided above, under the Step 2A, Prong One evaluation, independent claim 1 recites a series of method steps that do not require any structural elements for performing/executing said method. Therefore, claim 1 recites a judicial exception of an abstract idea, specifically, a mental process, because (1) the collecting of the data used for the data analysis may be considered insignificant pre-solutionary activity, and (2) the human mind may be considered analogous to generic computing structure. Additionally, in support of this analysis under Step 2A, Prong One, MPEP 2106.04(a)(2) III. A. clearly explains that claims that contain limitations that can practically be performed in the human mind include observations, evaluations, judgements, and opinions. This section of the MPEP also mentions Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016), wherein the courts found that the claims recited a mental process because they merely included collecting information, analyzing it, and displaying certain results of the collection and analysis, when the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind. These findings from Electric Power Group are thus very relevant/analogous to the findings of the Examiner with regards to the instant application’s independent claim 1 being analyzed under 35 USC 101, Step 2A, Prong One. Examiner notes wherein the method is “for training a machine learning model to control a robot” (i.e. intended use/application) as provided in the preamble, and there is not a positively recited step for training a machine learning model and/or control step for controlling the robot to perform one or a plurality of assembly tasks, provided with the currently provided method steps, thus the method, under the broadest reasonable interpretation, is not positively/explicitly used for training a machine learning model, which is subsequently utilized for controlling a robot, so it fails to satisfy the requirements under this section. Examiner additionally notes wherein providing a positively recited control step that controls the robot to perform a plurality of assembly tasks (as similarly provided in dependent claim 9), would likely overcome the 35 USC 101 rejection provided herein.
Step 2A, Prong Two evaluation: Practical Application – No.
Claim 1, is evaluated whether as a whole it integrates the recited judicial exception into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception.
Regarding the claimed limitation(s)/element(s) of “one or more first trained machine learning models” and “a second trained machine learning model”, the Examiner submits that these limitations are simply computing elements that are recited at a high level of generality to which the abstract ideas are applied. These generic computing elements merely automate the abstract idea(s) presented above, without adding significantly more to distinguish themselves, such as by having unique structural components that incorporate features that cannot be done in the human mind. Regarding the claimed “first trained machine learning model” and “second trained machine learning model”, as it is stated in the claim and the specification, are generic computing element(s) that, as stated in paragraph 0036, “Specialist critic models 121 are machine learning models, such as neural networks, which can be trained to evaluate actions…”. Thus for the additional element(s) of claim 1 analyzed individually, and/or taken as a whole, there is insufficient reasoning as to why the additional elements turn the abstract ideas into practical applications, since the additional elements turn the abstract ideas into practical applications, since the additional elements merely recite automating the abstract ideas. Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Examiner notes wherein the method is “for training a machine learning model to control a robot” (i.e. intended use/application) as provided in the preamble, and further the “to generate one or more first trained machine learning models…to control a robot to perform a different assembly task” and “to generate a second trained machine learning model…to control the robot to perform a plurality of assembly tasks” (i.e. intended use(s)/application(s)) as provided in the above limitation(s) are not definitively required within the claimed arrangement (i.e., they may be considered external to the claimed arrangement), thus the method, under the broadest reasonable interpretation, does not positively recite “training a machine learning model” and/or “controlling a robot to perform one or more assembly tasks”, so it fails to satisfy the requirements under this section. Therefore the claim is ineligible.
Step 2B, evaluation: Inventive Concept – No.
Claim 1, is evaluated as to whether the claims as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
With regards to Step 2B of the 101 analysis, claim 1 does not recite any additional elements that amount to significantly more than the judicial exception for the same reasons as described above in Step 2A Prong Two. Specifically, the claimed “one or more first trained machine learning models” and “a second trained machine learning model” as defined in the specification, only recite applying generic computing elements and therefore do not recite significantly more than the judicial exception. Generally, applying an exception using generic computing element(s) or receiving and interpreting data cannot provide an inventive concept. As noted above, Examiner additionally notes wherein the method is “for training a machine learning model to control a robot” (i.e. intended use/application) as provided in the preamble, and further the “to generate one or more first trained machine learning models…to control a robot to perform a different assembly task” and “to generate a second trained machine learning model…to control the robot to perform a plurality of assembly tasks” (i.e. intended use(s)/application(s)) as provided in the above limitation(s) are not definitively required within the claimed arrangement (i.e., they may be considered external to the claimed arrangement), thus the method, under the broadest reasonable interpretation, does not positively recite “training a machine learning model” and/or “controlling a robot to perform one or more assembly tasks”, so it fails to satisfy the requirements under this section. Thus, since independent claim 1 is: (a) directed toward an abstract idea, (b) does not recite additional elements that integrate the judicial exception into a practical application, and (c) does not recite additional elements that amount to significantly more than the judicial exception, it is clear that independent claim 1 is directed towards non-statutory subject matter.
Regarding claims 2-8, these claims do not recite any further limitations that cause the claim(s) to be directed towards statutory subject matter. The claims merely recite an abstract idea. Each of the further limitations expound upon the abstract idea and do not recite additional elements that are not well understood, routine or conventional. Therefore claims 2-8 are similarly rejected as being directed towards non-statutory subject matter.
Regarding claim 11, One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
performing, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot to perform a different assembly task; and
performing, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks.
Step 1: Statutory Category – Yes.
The claim(s) recite(s) a method (i.e. process), therefore the claim(s) fall within one of the four statutory categories. MPEP 2106.03.
Step 2A, Prong One evaluation: Judicial Exception – Yes.
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under the broadest reasonable interpretation, the claim covers performance using mental processes.
The claim recites the limitation of “performing, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot to perform a different assembly task” in the context of this claim is an abstract idea, wherein a human mentally observes and/or evaluates “one or more first training operations”. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind.
The claim additionally recites the limitation of “performing, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks” in the context of this claim is an abstract idea, wherein a human mentally observes and/or evaluates “one or more second training operations”. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind.
As provided above, under the Step 2A, Prong One evaluation, independent claim 11 recites a series of method steps that merely require one or more non-transitory computer readable media storing instructions and one or more processors, which merely describes generic computing components that executes data collection and analysis. Therefore, claim 11 recites a judicial exception of an abstract idea, specifically, a mental process, because (1) the collecting of the data used for the data analysis may be considered insignificant pre-solutionary activity, and (2) the human mind may be considered analogous to generic computing structure. Additionally, in support of this analysis under Step 2A, Prong One, MPEP 2106.04(a)(2) III. A. clearly explains that claims that contain limitations that can practically be performed in the human mind include observations, evaluations, judgements, and opinions. This section of the MPEP also mentions Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016), wherein the courts found that the claims recited a mental process because they merely included collecting information, analyzing it, and displaying certain results of the collection and analysis, when the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind. These findings from Electric Power Group are thus very relevant/analogous to the findings of the Examiner with regards to the instant application’s independent claim 11 being analyzed under 35 USC 101, Step 2A, Prong One. Examiner notes wherein there is not a positively recited step for training a machine learning model and/or control step for controlling the robot to perform one or a plurality of assembly tasks, provided with the currently provided method steps, thus the method, under the broadest reasonable interpretation, is not positively/explicitly used for training a machine learning model, which is subsequently utilized for controlling a robot, so it fails to satisfy the requirements under this section. Examiner additionally notes wherein providing a positively recited control step that controls the robot to perform a plurality of assembly tasks (as similarly provided in dependent claim 18), would likely overcome the 35 USC 101 rejection provided herein.
Step 2A, Prong Two evaluation: Practical Application – No.
Claim 11, is evaluated whether as a whole it integrates the recited judicial exception into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception.
Regarding the claimed limitation(s)/element(s) of “one or more non-transitory computer readable media”, “one or more processors”, “one or more first trained machine learning models” and “a second trained machine learning model”, the Examiner submits that these limitations are simply computing elements that are recited at a high level of generality to which the abstract ideas are applied. These generic computing elements merely automate the abstract idea(s) presented above, without adding significantly more to distinguish themselves, such as by having unique structural components that incorporate features that cannot be done in the human mind. Regarding the claimed “one or more non-transitory computer readable media” and “one or more processors, as it is stated in the claim and the specification, are generic computing element(s) that, as stated in at least paragraph 0030-0032, wherein “System memory 114 of machine learning server 110 stores content, such as software applications and data, for use by processor(s) 112 and the GPU(s) and/or other processing units. System memory 114 can be any type of memory capable of storing data and software applications, such as a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash ROM), or any suitable combination of the foregoing”. Regarding the claimed “first trained machine learning model” and “second trained machine learning model”, as it is stated in the claim and the specification, are generic computing element(s) that, as stated in paragraph 0036, “Specialist critic models 121 are machine learning models, such as neural networks, which can be trained to evaluate actions…”. Thus for the additional element(s) of claim 11 analyzed individually, and/or taken as a whole, there is insufficient reasoning as to why the additional elements turn the abstract ideas into practical applications, since the additional elements turn the abstract ideas into practical applications, since the additional elements merely recite automating the abstract ideas. Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Examiner notes wherein the method is “for training a machine learning model to control a robot” (i.e. intended use/application) as provided in the preamble, and further the “to generate one or more first trained machine learning models…to control a robot to perform a different assembly task” and “to generate a second trained machine learning model…to control the robot to perform a plurality of assembly tasks” (i.e. intended use(s)/application(s)) as provided in the above limitation(s) are not definitively required within the claimed arrangement (i.e., they may be considered external to the claimed arrangement), thus the method, under the broadest reasonable interpretation, does not positively recite “training a machine learning model” and/or “controlling a robot to perform one or more assembly tasks”, so it fails to satisfy the requirements under this section. Therefore the claim is ineligible.
Step 2B, evaluation: Inventive Concept – No.
Claim 11, is evaluated as to whether the claims as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
With regards to Step 2B of the 101 analysis, claim 11 does not recite any additional elements that amount to significantly more than the judicial exception for the same reasons as described above in Step 2A Prong Two. Specifically, the claimed “one or more non-transitory computer readable media”, “one or more processors”, “one or more first trained machine learning models” and “a second trained machine learning model” as defined in the specification, only recite applying generic computing elements and therefore do not recite significantly more than the judicial exception. Generally, applying an exception using generic computing element(s) or receiving and interpreting data cannot provide an inventive concept. As noted above, Examiner additionally notes wherein the method provides the intended use/application “to generate one or more first trained machine learning models…to control a robot to perform a different assembly task” and “to generate a second trained machine learning model…to control the robot to perform a plurality of assembly tasks” (i.e. intended use(s)/application(s)) as provided in the above limitation(s) are not definitively required within the claimed arrangement (i.e., they may be considered external to the claimed arrangement), thus the method, under the broadest reasonable interpretation, does not positively recite “training a machine learning model” and/or “controlling a robot to perform one or more assembly tasks”, so it fails to satisfy the requirements under this section. Thus, since independent claim 11 is: (a) directed toward an abstract idea, (b) does not recite additional elements that integrate the judicial exception into a practical application, and (c) does not recite additional elements that amount to significantly more than the judicial exception, it is clear that independent claim 11 is directed towards non-statutory subject matter.
Regarding claims 12-17, these claims do not recite any further limitations that cause the claim(s) to be directed towards statutory subject matter. The claims merely recite an abstract idea. Each of the further limitations expound upon the abstract idea and do not recite additional elements that are not well understood, routine or conventional. Therefore claims 12-17 are similarly rejected as being directed towards non-statutory subject matter.
Regarding claim 20, a system comprising:
one or more memories storing instructions, and
one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to:
perform, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot to perform a different assembly task, and
perform, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks.
Step 1: Statutory Category – Yes.
The claim(s) recite(s) a system, therefore the claim(s) fall within one of the four statutory categories. MPEP 2106.03.
Step 2A, Prong One evaluation: Judicial Exception – Yes.
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under the broadest reasonable interpretation, the claim covers performance using mental processes.
The claim recites the limitation of “perform, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot to perform a different assembly task” in the context of this claim is an abstract idea, wherein a human mentally observes and/or evaluates “one or more first training operations”. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind.
The claim additionally recites the limitation of “perform, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks” in the context of this claim is an abstract idea, wherein a human mentally observes and/or evaluates “one or more second training operations”. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind. Humans have the ability to obtain, recognize and interpret data from multiple sources including other humans and machines, and therefore Examiner submits that this action can be done within the human mind.
As provided above, under the Step 2A, Prong One evaluation, independent claim 20 recites a system comprising one or more memories storing instructions, and one or more processors, which merely describes generic computing components that executes data collection and analysis. Therefore, claim 20 recites a judicial exception of an abstract idea, specifically, a mental process, because (1) the collecting of the data used for the data analysis may be considered insignificant pre-solutionary activity, and (2) the human mind may be considered analogous to generic computing structure. Additionally, in support of this analysis under Step 2A, Prong One, MPEP 2106.04(a)(2) III. A. clearly explains that claims that contain limitations that can practically be performed in the human mind include observations, evaluations, judgements, and opinions. This section of the MPEP also mentions Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016), wherein the courts found that the claims recited a mental process because they merely included collecting information, analyzing it, and displaying certain results of the collection and analysis, when the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind. These findings from Electric Power Group are thus very relevant/analogous to the findings of the Examiner with regards to the instant application’s independent claim 20 being analyzed under 35 USC 101, Step 2A, Prong One. Examiner notes wherein the system fails to include a positively recited limitation for training a machine learning model and/or controlling the robot to perform one or a plurality of assembly tasks, provided with the currently provided limitations, thus the system, under the broadest reasonable interpretation, is not positively/explicitly configured to train a machine learning model, and further configured to control a robot, so it fails to satisfy the requirements under this section. Examiner additionally notes wherein providing a positively recited control function that controls the robot to perform a plurality of assembly tasks would likely overcome the 35 USC 101 rejection provided herein.
Step 2A, Prong Two evaluation: Practical Application – No.
Claim 20, is evaluated whether as a whole it integrates the recited judicial exception into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception.
Regarding the claimed limitation(s)/element(s) of “one or more memories storing instructions”, “one or more processors”, “one or more first trained machine learning models” and “a second trained machine learning model”, the Examiner submits that these limitations are simply computing elements that are recited at a high level of generality to which the abstract ideas are applied. These generic computing elements merely automate the abstract idea(s) presented above, without adding significantly more to distinguish themselves, such as by having unique structural components that incorporate features that cannot be done in the human mind. Regarding the claimed “one or more memories storing instructions” and “one or more processors, as it is stated in the claim and the specification, are generic computing element(s) that, as stated in at least paragraph 0030-0032, wherein “System memory 114 of machine learning server 110 stores content, such as software applications and data, for use by processor(s) 112 and the GPU(s) and/or other processing units. System memory 114 can be any type of memory capable of storing data and software applications, such as a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash ROM), or any suitable combination of the foregoing”. Regarding the claimed “first trained machine learning model” and “second trained machine learning model”, as it is stated in the claim and the specification, are generic computing element(s) that, as stated in paragraph 0036, “Specialist critic models 121 are machine learning models, such as neural networks, which can be trained to evaluate actions…”. Thus for the additional element(s) of claim 20 analyzed individually, and/or taken as a whole, there is insufficient reasoning as to why the additional elements turn the abstract ideas into practical applications, since the additional elements turn the abstract ideas into practical applications, since the additional elements merely recite automating the abstract ideas. Examiner additionally notes wherein the system with a processor configured to perform the abstract idea above, and further provides the intended use/application “to generate one or more first trained machine learning models…to control a robot to perform a different assembly task” and “to generate a second trained machine learning model…to control the robot to perform a plurality of assembly tasks” (i.e. intended use(s)/application(s)) as provided in the above limitation(s) are not definitively required within the claimed arrangement (i.e., they may be considered external to the claimed arrangement). Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore the claim is ineligible.
Step 2B, evaluation: Inventive Concept – No.
Claim 20, is evaluated as to whether the claims as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
With regards to Step 2B of the 101 analysis, claim 20 does not recite any additional elements that amount to significantly more than the judicial exception for the same reasons as described above in Step 2A Prong Two. Specifically, the claimed “one or more memories storing instructions”, “one or more processors”, “one or more first trained machine learning models” and “a second trained machine learning model” as defined in the specification, only recite applying generic computing elements and therefore do not recite significantly more than the judicial exception. Generally, applying an exception using generic computing element(s) or receiving and interpreting data cannot provide an inventive concept. As noted above, Examiner additionally notes wherein the system with a processor configured to perform the abstract idea above, and further provides the intended use/application “to generate one or more first trained machine learning models…to control a robot to perform a different assembly task” and “to generate a second trained machine learning model…to control the robot to perform a plurality of assembly tasks” (i.e. intended use(s)/application(s)) as provided in the above limitation(s) are not definitively required within the claimed arrangement (i.e., they may be considered external to the claimed arrangement), thus the system, under the broadest reasonable interpretation, fails to include a positively recited limitation for training a machine learning model and/or controlling the robot to perform one or a plurality of assembly tasks, so it fails to satisfy the requirements under this section. Thus, since independent claim 20 is: (a) directed toward an abstract idea, (b) does not recite additional elements that integrate the judicial exception into a practical application, and (c) does not recite additional elements that amount to significantly more than the judicial exception, it is clear that independent claim 20 is directed towards non-statutory subject matter.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Gombolay et al (US 2023/0086122 A1, hereinafter Gombolay).
Regarding claim 1, Gombolay teaches a computer-implemented method for training a machine learning model to control a robot (Figure 1, robotic system 104), the method comprising:
performing, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot to perform a different assembly task (Figures 1 & 4; at least as in paragraphs 0040-0043, wherein “the human-robot collaborative flexible manufacturing system 100 includes a user-guided or assisted training system (shown as “Training System” 102), a robotic system 104 (shown as 104′), and source workpiece 106 (shown as 106′) that allows a human expert (also referred to as the user) to demonstrate and teach various steps of a manufacturing or assembly process to a robotic system” and further as in at least paragraphs 0080-0083, wherein “a process 400 to utilize human guided-AI training in a flexible manufacturing or assembly task. The human guided-AI training for the manufacturing process shown in FIG. 4 includes inputting guided-sequence data 402. The guided sequence data 402 may include a human-guided demonstration where the human kinesthetically guides the robotic system (104) to perform a variety of tasks”); and
performing, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks (Figures 1 & 4; at least as in paragraphs 0040-0043, wherein “The model 107 of the source workpiece 106′ can be used in combination with dynamic movement primitives 109 (existing DMP or newly generated DMP) and new part instructions 111 to direct the robotic system 104′, or another robotic system 104”, to perform the same action and/or task on a new workpiece 106″, in same workpiece 112 or a new workspace 112′, in a subsequent manufacturing and/or assembly operation 113 (shown as 113′ and 113″ respectively)” and further as in at least paragraphs 0080-0083, wherein “the training system 102 can generate DMPs through user-guided input. In one embodiment, the training system 102 maintains a library or catalog of recorded DMPs. The library or catalog allows the robotic system 104 to perform various tasks without the need for human-guided re-training. The library or catalog can be employed to combine multiple DMPs into more complex tasks or macros. In one example, a first DMP may move the robotic arm 115 from a first position non-adjacent to a workpiece 106″ to a second position adjacent to the workpiece. A second DMP may then execute a task by moving the robotic arm, e.g., screwing, fastening, or welding, at the workpiece surface. The user may combine two or more DMPs as a macro to command the robotic arm 115 to execute a task at that section of the workpiece. Other DMPs may also be combined to enable the robotic arm to move across the workpiece and perform different tasks at different locations.”).
Regarding claim 2, Gombolay further teaches wherein performing one or more training operations to generate the one or more first trained machine learning models comprises:
processing one or more first observations using an untrained machine learning model to generate an action (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
generating, based on the action and using a simulator, simulated data and one or more second observations (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
processing the one or more second observations and the action using an untrained evaluation model to generate one or more evaluations (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
calculating, based on the simulated data and the demonstration data, a reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
updating one or more parameters of the untrained machine learning model and one or more parameters of the untrained evaluation model based on the one or more evaluations and the reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 3, Gombolay further teaches wherein the simulated data comprises at least one of one or more joint angles of the robot or one or more poses of a part included in the one or more parts (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 4, Gombolay further teaches wherein calculating the reward comprises:
calculating, based on the simulated data, a reinforcement learning reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
calculating, based on the demonstration data and the simulated data, an imitation learning reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
calculating, based on the reinforcement learning reward and the imitation learning reward, the reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 5, Gombolay further teaches wherein calculating the reward comprises using at least one of a dynamic time warping or a signature transform to measure a similarity between a simulated trajectory included in the simulated data and a demonstration trajectory included in the demonstration data (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 6, Gombolay further teaches wherein performing one or more training operations to generate the second trained machine learning model comprises:
processing one or more first observations using a first trained machine learning model included in the one or more first trained machine learning models to generate one or more first robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
generating, based on the one or more first robot actions and using a simulator, additional demonstration data (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
generating, based on the additional demonstration data, one or more second observations (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
processing the one or more second observations and the one or more geometries using an untrained machine learning model to generate one or more second robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
calculating, based on the additional demonstration data and the one or more second robot actions, a first loss (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
updating one or more parameters of the untrained machine learning model based on the first loss to generate a trained machine learning model (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 7, Gombolay further teaches wherein performing one or more training operations to generate the second trained machine learning model further comprises:
processing the one or more second observations and the one or more geometries using the trained machine learning model to generate one or more third robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
generating, based on the one or more third robot actions and using the simulator, one or more third observations and data aggregation (DAgger) data (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
processing the one or more third observations using the trained first machine learning model to generate one or more fourth robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
calculating, based on the DAgger data and the one or more fourth robot actions, a second loss (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
updating one or more parameters of the trained machine learning model based on the second loss (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 8, Gombolay further teaches wherein performing one or more training operations to generate the second trained machine learning model further comprises using a sampling-based curriculum to re-train the second trained machine learning model (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 9, Gombolay teaches the method further comprising: processing, based on one or more sensor inputs, sensor data to generate processed sensor data (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and processing the processed sensor data and the one or more geometries using the second trained machine learning model to generate one or more robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and controlling the robot to move based on the one or more robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 10, Gombolay further teaches wherein processing the sensor data comprises performing, based on the sensor data, at least one of one or more pose estimation operations, one or more segmentation operations, or one or more model-based estimation operations (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 11, Gombolay teaches one or more non-transitory computer readable media storing instructions that, when executed by one or more processors (at least as in paragraph 0022, wherein “a system is disclosed including a processor, a memory having instructions stored thereon…”), cause the one or more processors to perform the steps of:
performing, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot (Figure 1, robotic system 104) to perform a different assembly task (Figures 1 & 4; at least as in paragraphs 0040-0043, wherein “the human-robot collaborative flexible manufacturing system 100 includes a user-guided or assisted training system (shown as “Training System” 102), a robotic system 104 (shown as 104′), and source workpiece 106 (shown as 106′) that allows a human expert (also referred to as the user) to demonstrate and teach various steps of a manufacturing or assembly process to a robotic system” and further as in at least paragraphs 0080-0083, wherein “a process 400 to utilize human guided-AI training in a flexible manufacturing or assembly task. The human guided-AI training for the manufacturing process shown in FIG. 4 includes inputting guided-sequence data 402. The guided sequence data 402 may include a human-guided demonstration where the human kinesthetically guides the robotic system (104) to perform a variety of tasks”); and
performing, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks (Figures 1 & 4; at least as in paragraphs 0040-0043, wherein “The model 107 of the source workpiece 106′ can be used in combination with dynamic movement primitives 109 (existing DMP or newly generated DMP) and new part instructions 111 to direct the robotic system 104′, or another robotic system 104”, to perform the same action and/or task on a new workpiece 106″, in same workpiece 112 or a new workspace 112′, in a subsequent manufacturing and/or assembly operation 113 (shown as 113′ and 113″ respectively)” and further as in at least paragraphs 0080-0083, wherein “the training system 102 can generate DMPs through user-guided input. In one embodiment, the training system 102 maintains a library or catalog of recorded DMPs. The library or catalog allows the robotic system 104 to perform various tasks without the need for human-guided re-training. The library or catalog can be employed to combine multiple DMPs into more complex tasks or macros. In one example, a first DMP may move the robotic arm 115 from a first position non-adjacent to a workpiece 106″ to a second position adjacent to the workpiece. A second DMP may then execute a task by moving the robotic arm, e.g., screwing, fastening, or welding, at the workpiece surface. The user may combine two or more DMPs as a macro to command the robotic arm 115 to execute a task at that section of the workpiece. Other DMPs may also be combined to enable the robotic arm to move across the workpiece and perform different tasks at different locations.”).
Regarding claim 12, Gombolay further teaches wherein performing one or more training operations to generate the one or more first trained machine learning models comprises:
processing one or more first observations using an untrained machine learning model to generate an action (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
generating, based on the action and using a simulator, simulated data and one or more second observations (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
processing the one or more second observations and the action using an untrained evaluation model to generate one or more evaluations (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
calculating, based on the simulated data and the demonstration data, a reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
updating one or more parameters of the untrained machine learning model and one or more parameters of the untrained evaluation model based on the one or more evaluations and the reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 13, Gombolay further teaches wherein calculating the reward comprises:
calculating, based on the simulated data, a reinforcement learning reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
calculating, based on the demonstration data and the simulated data, an imitation learning reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
calculating, based on the reinforcement learning reward and the imitation learning reward, the reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 14, Gombolay further teaches wherein calculating the reward comprises:
calculating a simulation error reward using a simulation-aware policy update (SAPU) that measures a maximum interpenetration distance at a timestep (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
in response to determining that the simulation error reward is less than a threshold, applying a weight to the simulation error reward in inverse proportion to the maximum interpenetration distance (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
in response to determining that the simulation reward is not less than the threshold, not updating the simulation error reward (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 15, Gombolay further teaches wherein performing one or more training operations to generate the second trained machine learning model comprises:
processing one or more first observations using a first trained machine learning model included in the one or more first trained machine learning models to generate one or more first robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
generating, based on the one or more first robot actions and using a simulator, additional demonstration data (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
generating, based on the additional demonstration data, one or more second observations (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
processing the one or more second observations and the one or more geometries using an untrained machine learning model to generate one or more second robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
calculating, based on the additional demonstration data and the one or more second robot actions, a first loss (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
updating one or more parameters of the untrained machine learning model based on the first loss to generate a trained machine learning model (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 16, Gombolay further teaches wherein performing one or more training operations to generate the second trained machine learning model further comprises:
processing the one or more second observations and the one or more geometries using the trained machine learning model to generate one or more third robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
generating, based on the one or more third robot actions and using the simulator, one or more third observations and data aggregation (DAgger) data (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
processing the one or more third observations using the trained first machine learning model to generate one or more fourth robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119);
calculating, based on the DAgger data and the one or more fourth robot actions, a second loss (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
updating one or more parameters of the trained machine learning model based on the second loss (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 17, Gombolay further teaches wherein the demonstration data comprises a plurality of reversed disassembly trajectories associated with the one or more assembly tasks (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 18, Gombolay further teaches wherein the instructions, when executed by the one or more processors, further cause the one or more processors to perform the steps of:
processing, based on one or more sensor inputs, sensor data to generate processed sensor data (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
processing the processed sensor data and the one or more geometries using the second trained machine learning model to generate one or more robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119); and
controlling the robot to move based on the one or more robot actions (Figures 1-6; at least as in paragraphs 0067-0069, 0072-0073, 0079-0083, 0100-0101 and 0114-0119)
Regarding claim 19, Gombolay further teaches wherein processing the processed sensor data and the one or more geometries using the trained machine learning model comprises generating a latent vector, based on the one or more geometries, using a trained autoencoder (Figures 1-6; at least as in paragraphs 0067-0069, 0071-0073, 0079-0083, 0100-0101 and 0114-0119).
Regarding claim 20, Gombolay teaches a system (Figure 1, system 100; at least as in paragraphs 0040-0043) comprising:
one or more memories storing instructions (at least as in paragraph 0022, wherein “a system is disclosed including a processor, a memory having instructions stored thereon…”), and
one or more processors that are coupled to the one or more memories (at least as in paragraph 0022, wherein “a system is disclosed including a processor, a memory having instructions stored thereon…”) and, when executing the instructions, are configured to:
perform, based on demonstration data associated with one or more assembly tasks, one or more first training operations to generate one or more first trained machine learning models, wherein each first trained machine learning included in the one or more first trained machine learning models is trained to control a robot (Figure 1, robotic system 104) to perform a different assembly task (Figures 1 & 4; at least as in paragraphs 0040-0043, wherein “the human-robot collaborative flexible manufacturing system 100 includes a user-guided or assisted training system (shown as “Training System” 102), a robotic system 104 (shown as 104′), and source workpiece 106 (shown as 106′) that allows a human expert (also referred to as the user) to demonstrate and teach various steps of a manufacturing or assembly process to a robotic system” and further as in at least paragraphs 0080-0083, wherein “a process 400 to utilize human guided-AI training in a flexible manufacturing or assembly task. The human guided-AI training for the manufacturing process shown in FIG. 4 includes inputting guided-sequence data 402. The guided sequence data 402 may include a human-guided demonstration where the human kinesthetically guides the robotic system (104) to perform a variety of tasks”), and
perform, based on the one or more first trained machine learning models and one or more geometries associated with one or more parts, one or more second training operations to generate a second trained machine learning model, wherein the second trained machine learning model is trained to control the robot to perform a plurality of assembly tasks (Figures 1 & 4; at least as in paragraphs 0040-0043, wherein “The model 107 of the source workpiece 106′ can be used in combination with dynamic movement primitives 109 (existing DMP or newly generated DMP) and new part instructions 111 to direct the robotic system 104′, or another robotic system 104”, to perform the same action and/or task on a new workpiece 106″, in same workpiece 112 or a new workspace 112′, in a subsequent manufacturing and/or assembly operation 113 (shown as 113′ and 113″ respectively)” and further as in at least paragraphs 0080-0083, wherein “the training system 102 can generate DMPs through user-guided input. In one embodiment, the training system 102 maintains a library or catalog of recorded DMPs. The library or catalog allows the robotic system 104 to perform various tasks without the need for human-guided re-training. The library or catalog can be employed to combine multiple DMPs into more complex tasks or macros. In one example, a first DMP may move the robotic arm 115 from a first position non-adjacent to a workpiece 106″ to a second position adjacent to the workpiece. A second DMP may then execute a task by moving the robotic arm, e.g., screwing, fastening, or welding, at the workpiece surface. The user may combine two or more DMPs as a macro to command the robotic arm 115 to execute a task at that section of the workpiece. Other DMPs may also be combined to enable the robotic arm to move across the workpiece and perform different tasks at different locations.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See attached PTO-892 – Notice of References Cited form. Examiner additionally notes the following prior art references, in the same field of endeavor as the instant invention, and also appears to read on several of the currently provided claim limitations above;
US 2023/0040623 A1, issued to Le et al, which is directed towards a deep reinforcement learning apparatus and corresponding system for a robotic pick and place system that utilizes simulations with machine learning techniques.
US 2025/0326116 A1, issued to Corcodel et al, which is directed towards a system and method for controlling a robotic manipulator according to a task through machine learning techniques.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN L SAMPLE whose telephone number is (571)270-5925. The examiner can normally be reached Monday-Friday 7:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached at (571)270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN L SAMPLE/Primary Examiner, Art Unit 3657
