DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment filed on Feb. 26th, 2026. The amendments are linked to the original application filed on Oct. 4th, 2021.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on Feb. 26th, 2026 has been entered.
Response to Amendment
The examiner thanks the applicant for the remarks, edits and arguments.
Regarding Claim Rejections – 35 U.S.C. 101
The applicant argues the amended independent claims do not cite abstract ideas and recites an amended limitation. The applicant believes that this limitations does not disclose a mental concept and describes “a specific technical implementation in a manufacturing environment where automated quality assessment is performed on physical products during manufacturing operations.”. The MPEP 2106.04(a)(2)(III) defines examples of abstract ideas which can be performed in a human mind, “accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions.”. Using the broadest reasonable interpretation of the claims, in light of the specification, this limitation recites a process or an evaluation of data to produce a binary result of the evaluation. The examiner believes it is reasonable for a human to be able to evaluate data, in a set of data, and provide a judgement or result of each of the sections of data. This limitation does not disclose a special unique machine to execute these process and it is reasonably assumed that this process is carried out on a generic computer. According to MPEP 2106.04(a)(2)(III)(C), “In evaluating whether a claim that requires a computer recites a mental process, examiners should carefully consider the broadest reasonable interpretation of the claim in light of the specification. For instance, examiners should review the specification to determine if the claimed invention is described as a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process.”. Taking this into consideration the examiner believes that the limitation consists of an abstract idea of evaluating sets of data. Next the examiner would like to note that the process is executed on a generic computing system. Using the MPEP definitions, the claim limitation, “conducting a quality evaluation on each of the plurality of subtasks, wherein the quality evaluation comprises a binary quality check indicating pass or fail of the product following completion of a subtask” would recite an abstract idea which is to be performed on a generic computer. Therefore, the examiner believes this limitation is a mental concept.
Next, The Examiner has reviewed this limitation and finds the applicants argument persuasive for this limitation. However, the examiner would like to note that the claimed amendments still recite other limitations that would be considered abstract ideas and mathematical concepts. The given amended limitation discloses a process that cannot be completed in the human mind and would be further evaluated as an additional element in steps 2A, Prong 2 and Step 2B.
Next, the applicant argues the limitation, ““wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training" as recited by amended claim 1 describes a specific distributed computing architecture that solves technical problems in manufacturing environments. This is not generic computer implementation but a particular arrangement of edge computing systems that enables local video processing while maintaining centralized learning coordination across multiple manufacturing workcells.” The examiner would like to note the claim limitations are evaluated using the Alice/Mayo test given their broadest reasonable interpretations of the claim limitations. Using the BRI the examiner believes that this limitation recites the use of generic systems, “edge systems”, which can be any form of computing system from a smart watch to a high-end computing systems able to process video data and perform the given functions. The examiner does not believe that this limitation discloses complex or unique computing systems and believes that the concepts of distributed computing and centralized learning to be well understood, routine and/or conventual machine learning and general computing concepts.
Next, the applicant argues, “Contrary to the Examiner's characterization, the claims do not merely apply abstract ideas using generic computers. Instead, they solve the specific technical problem of training robots to perform manufacturing tasks by observing human workers, evaluating task quality in real manufacturing environments, and distributing learned sequences across multiple manufacturing workcells. The "applying the learned subtask sequences to control robot operations" limitation as recited by amended claim 1 directly controls physical manufacturing equipment, which is a concrete application that goes well beyond abstract ideas.” The examiner would like to note that this limitation does not disclose an abstract idea. Further, in the previous office action this limitation was not interpreted to be an abstract idea. The examiner believes that this claim limitation does not recite the process of controlling robotic systems. The limitation recites that it applies the learned information to a control robot operations. As it is interpreted, this limitation appears to merely apply the learned concept to another system. Therefore, the examiner believes this limitation is evaluated under Step 2A, Prong 2 and would not integrate a judicial exception into a practical application.
Next the applicant argues, “Claims 2-17 depend from amended claim 1 and add additional technical details such as robot vision systems, manufacturing system integration, and specific video processing techniques. These dependent claims are allowable for the same reasons as amended claim 1, as they further specify the technical implementation of the robot training system in manufacturing environments.” The examiner would like to note that the applicant has not provided any other evidence to support that the dependent claims provide, “additional technical details such as robot vision systems, manufacturing system integration, and specific video processing techniques”.
Finally, after each submitted amendments the Alice/Mayo test is performed on the claims as a whole. The examiner believes that the claims still recite mental concepts of observations, evaluations, judgments and opinions. The newly added claims 18-22 have also been evaluated with the claims as a whole and the examiner believes that these do not integrate the judicial exceptions into a practical application. Further the examiner believes that a person of ordinary skill would not be able to evaluate the claims and identify the inventive concepts in light of the specification. Therefore, the examiner believes that the current rejection under 35 U.S.C. 101 is upheld and the current amended claims recite patent ineligible subject matter.
Regarding Claim Rejections – 35 U.S.C. 103
The applicant argues that the current amended claims overcome the combination of prior arts provided. For example, the applicant believes that Finn et al (“Finn”) and Duan et al (“Duan”) do not disclose, “a binary quality check indicating pass or fail of the product". The applicant believes that Finn teaches a loss function and Duan discloses, “task completion success rather than product quality evaluation in manufacturing. Neither reference teaches conducting quality evaluation on manufacturing products with binary pass/fail determinations following subtask completion as specifically recited by the amended claims.”
Next, the applicant argues that Finn and Duan, do not disclose or teach binary quality control measures for manufacturing products. The applicant believes that the amendments made further distinguish the claimed invention from the proposed arts. The examiner would like to note that the claims themselves do not disclose a process of evaluating products as a result of a manufacturing process. The independent claim states, “receiving information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” using the broadest reasonable interpretation this claim discloses receiving information related to performing product manufacturing. This does not disclose that the information received would also contain the end result or product but instead disclose the video process on how the manufacturing process was performed. The art proposed discloses a process of evaluating the received information, the video clips or sequences. Therefore, the examiner believes that Duan does disclose a process for evaluating subtasks in a sequence of events. Finally, Duan discloses the use of training parameters which would determine if the task was executed correctly or incorrectly and the system receives further training.
Next, the applicant argues that the primary arts do not disclose a process of reinforcement learning with reward values. The applicant believes the proposed arts disclose a process that does not use rewards to train models and further argues that the proposed arts do not disclose a product quality evaluation to perform quality checks of products. The examiner would again like to point to the first limitation of the independent claims. The claim recites a process of receiving information about performing product manufacturing and does not disclose the information about the end result of the manufacturing processing process. A person of ordinary skill in the art would not recognize that this claim discloses a process of evaluating an end result or product. Therefore, the examiner has taken the interpretation that the final quality evaluation is an evaluation the demonstration. The examiner has found that Finn does disclose a process that is able to evaluate the subtasks within the set of tasks from demonstrations of a human actor. Further, the examiner would also like to note that the process MAML, “model-agnostic meta-learning”, which is the core element of Finn, is “an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient de scent and applicable to a variety of different learning problems, including classification, regression, and reinforcement learning.” (Finn et al, “Model-Agnostic Meta-Learning for Fast Adaptation of Deep Network”, Abstract, 2017, arXiv:1703.03400v3 [cs.LG]). The examiner has found new art to teach the added claim limitations. This new art Liu et al (“Liu”) also discloses a process of using reinforcement learning which utilizes rewards. The examiner believes these articles are related because of their use of intimation learning to train robotic systems from demonstrations. The examiner believes that the new proposed art cures the deficiencies of Finn and Duan and teaches the independent claims.
Next, the applicant argues that Finn and Duan fail to disclose a distributed system in multiple environments. The examiner has evaluated this argument and has found the applicant persuasive. Finn discloses a system that is able to train a singular robot or system. Next, Duan also discloses a training process for robotics in general and also fails to disclose a distributed system. However, the examiner has found new subject matter, Liu, that is able to teach this element. Liu discloses a federated learning system which contains a multitude of edge systems connected to cloud systems. These systems are able to distribute the learned information from one edge system to another. The examiner believes one of ordinary skill in the art would be able to discover these documents while researching the general subject of imitation learning, Duan and Finn, with implementing concepts related to federated systems and modify one or the other to produce the claimed system.
Next, the applicant argues, the proposed art Barajas would also not teach the missing deficiencies of Finn and Duan in the independent claims. The applicant argues that Barajas does not teach a distributed system and does not teach any binary quality checks. The examiner would like to point to the previous paragraph in regards to the binary quality check and distributed system. The examiner does not rely on Barajas to cure these noted deficiencies and the examiner believes that Finn, Duan and Liu disclose the independent claims.
Next, the applicant argues that Fox discloses a system that uses change point detection to identify motions in sequences of motion data. The applicant argues that this process is different that task identification process claimed in claims 5 and 13. The examiner would like to highlight the interpretation of this claims, “further comprising recognizing each of the plurality of subtasks based on change point detection to the human actions as determined from feature extraction, wherein detected change points from the change point detection are utilized to separate the each of the plurality of subtasks by time period.” (Emphasis added). Using the broadest reasonable interpretation of this claim the examiner believes this recites a process of recognizing subsets of tasks based on change point detections from extracted features, the changes detected are used to separate each of the tasks by a time period. The examiner has disclosed that Fox teaches these elements, “Introduction page 2; "To discover the dynamic behaviors shared between multiple time series, we propose a feature-based model. The entire collection of time series can be described by a globally shared set of possible behaviors. Individually, however, each time series will only exhibit a subset of these behaviors. The goal of joint analysis is to discover which behaviors are shared among the time series and which are unique.” (Emphasis added). The examiner believes that a person of ordinary skill in the art would be able to use the information in Fox to disclose or combine with other prior arts, to disclose a process of identifying motions or human task from a set of task sing feature data.
Next, The applicant argues that Huang does not disclose the amended elements of the independent claims. The applicant believes that the Huang does not disclose a process of distributed learning and the use of binary quality evaluations. The examiner would like to point to the previous response in regards to the independent claims. The examiner does not rely on Huang to cure the deficiencies of Finn and Duan in the amended independent claims. Instead, the examiner relies on the combination of Finn, Duan and Liu.
Next, the applicant argues that the amended claims overcome any combination of the provided arts because these arts fail to disclose the process of a binary final quality check, reinforcement learning using rewards all within a distributed system containing multiple edge systems and connected core learning centers. The examiner would like to point to the previous stated responses on the interpretation of these concepts in the claims and the newly proposed combination of Finn, Duan and Liu. The examiner believes that combining or modifying the concepts of Finn, Duan and Liu would lead one of ordinary skill in the art to disclose the claimed invention.
Next, the applicant argues that the one of ordinary skill in the art would not have the motivation to combine the prior arts that are listed to disclose the current amended claims. The applicant believes that the concepts disclosed in the proposed arts do not recognize the problems disclosed in the claims. Further the applicant believes that arts provided are unrelated and one of the art would not have motivation to combine the stated arts outside of hindsight. Finally, the applicant argues that there is no motivation to combine these articles contained within the articles or elsewhere. Therefore, the combination of these arts is invalid.
The examiner would first like to address the combination of Finn and Duan in the independent claims. Finn states testing of their method and comparison the results to different methods, “We evaluate our method on one-shot imitation in three experimental domains. In each setting, we compare our proposed method to a subset of the following methods:” (Finn, Experiments, pp. 6). Further, Finn discloses a LSTM method used to test against, “recurrent neural network which ingests the provided demonstration and the current observation, and outputs the current action, as proposed by Duan et al. [5].” (Emphasis added) (Finn, Experiments, pp. 6). This article cited is the article Duan, the same article used in this rejection. At the time of publishing, Finn has recognized the concepts of Duan and was able to compare the results of Duan and Finn to show an improvement of their system. This teaches that a person of ordinary skill in the art would be able to recognize the elements disclosed in Finn and Duan would have motivation to use concepts, teachings or modify concepts of either to disclose the concepts of claimed invention.
Next, the examiner would like to address the combination of Finn/Duan and other prior arts. Barajas teaches a system that teaches a robotic system to perform a task after a demonstration. The system in Barajas will use senor data and the ability to segment data into subsets and evaluate these subsets. One of ordinary skill in the art who was researching training automated control of robotic systems through demonstration would be able to review the concepts in Barajas. Then, with further teaching on concepts such as imitation learning and optimizing demonstrations for systems using as one-shot learning. These concepts are all related to teaching robotic systems based on one or many demonstrations. One would have the motivation to use teachings such as Finn and/or Duan to modify concepts of robotic movements and improve the field of teaching by demonstration.
Next, the examiner would like to address combination of Finn/Duan and Fox. Both Finn and Duan use demonstrations to train systems. Further, both disclose systems to identify robotic movements or tasks within a sequence of tasks. The core concept of Fox is to use different methods to evaluate and segment motion data from a sequence of motion data. Fox discloses process that is able to evaluate motion data over time and segment different motions into their own subtask or action. One of ordinary skill in the art that was researching concepts of imitation or one-shot learning from visual demonstrations would have the motivation to review documents on segmenting human motions in time data. One could be motivated to review the teachings of Fox and modify the segmentation of the visual actions, or tasks, process disclosed in Finn and/or Duan.
Next, the examiner would like to address combination of Finn/Duan and Huang. Huang teaches a system that is able to evaluate video demonstrations and learn the actions of the task. Huang discloses the use of imitation learning and attempts to also perform a task using a limited number of demonstrations. Further, Huang states, “We demonstrate that NTG is able to outperform both methods with unstructured representation [8], and methods with a hand designed hierarchical structure [41] on a diverse set of tasks, including simulated environment with photo-realistic rendering and a real-world dataset.” (Emphasis added) (Huang, Conclusion, pp. 8). Citation 8 in Huang is the same Duan article used as prior art in this rejection. One of ordinary skill in the art who was researching or designing a robotic system using reinforcement learning and imitation learning would be aware of the one-shot learning system disclosed in Duan. One would be motivated to use the teaching of Duan as a baseline to improve the concepts disclosed using Huang.
Next, the applicant argues that the prior arts of Finn/Duan fail to disclose the limitation of the newly add claim 18. The applicant believes that Finn does not disclose the same change point detection system as claimed in claim 18. The examiner has reviewed this argument and finds it persuasive. The art Finn does not disclose the process of using change point detection as stated in claim 18. The examiner would like to note that further search and consideration of the newly added claims was performed and the examiner has found art that is able to teach these concepts. The examiner believes that the combination of Duan/Finn and Roitberg et al (“Roitberg”) discloses the parts of the newly proposed claim 18.
Finally, the applicant argues that the newly added claims, 18-22, provide technical support to the independent claims and that the prior arts provided fail to disclose the newly added elements and claims. The Applicant also argues that the newly added claims recite technical processes and comply with 35 U.S.C. 101. The examiner would like to note that after each submitted amendment, the examiner is required to reapply the Alice/Mayo test on the claims and conduct a complete search to ensure the claims comply with 35 U.S.C. 101/103. As stated above, the Alice/Mao test has been applied to the amended claims for the reasons stated in the response above, and the 101 rejection below, the examiner believes the claims fail to comply with 35 U.S.C. 101, therefore the rejection under 35 U.S.C. 101 has been upheld. Next, after reviewing the claim language, amendments, remarks from the applicant, and a complete search the examiner believes that new arts has been discovered which, when used in combination of the previously provided arts, would lead one of ordinary skill in the art to disclose the claimed invention. For the reasons stated above, and in the 103 rejection section below, the examiner believes the rejection under 35 USC 103 is upheld.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 8-14, and 16-22 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Claim 1
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 1 recites, "A method, comprising:" therefore it is directed to the statutory category of a process.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“conducting a quality evaluation on each of the plurality of subtasks, wherein the quality evaluation comprises a product a binary quality check indicating pass or fail of the product following completion of a subtask;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a set of tasks in a video steam and would be able, with using a computer as a tool, perform a quality check and evaluation on the video data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determining one or more subtask sequences from the plurality of subtasks;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to, using a computer as a tool, evaluate a stream of video data containing human motions and segment that data into induvial and different subtasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“evaluating each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a set of tasks in a video steam and would be able, with using a computer as a tool, perform a quality check and evaluation on the video data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“wherein the evaluating the each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences comprises: constructing a function configured to predict a final quality evaluation of the product for the one or more subtask sequences from the quality evaluations of the plurality of subtasks associated with the each of the one or more subtask sequences,” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to generate and use a mathematical function to make a prediction. This claim discloses a math operation and therefore is ineligible.
“wherein the function is constructed using reinforcement learning with rewards based on prediction accuracy of the final quality evaluation compared to actual quality checks from a manufacturing system;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human can use a mathematic concepts such as reinforcement learning to develop an evaluation and prediction method. This claim discloses a math operation and therefore is ineligible.
“utilizing a validation set to evaluate the final quality evaluation for the each of the one or more subtask sequences and generate a prediction reward;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to validate information using observations and evaluation of a validation set. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“modifying the function based on the evaluation of the final quality evaluation for the each of the one or more subtask sequences based on the prediction reward;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a mathematical function and modify it to produce the desired result. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“iteratively repeating the constructing, utilizing, and modifying to finalize the function through reinforced learning;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to iterate through a set of tasks and perform the steps above. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “receiving information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
“outputting ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “receiving information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
“outputting ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 2
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the information associated with the human actions to train the associated robot in the edge system comprises video clips, each of video chips associated with a subtask from the plurality of subtasks;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the outputting the ones of the one or more subtask sequences to train the associated robot comprises providing ones of the video clips associated with ones of the subtasks associated with the each of the one or more subtask sequences.” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the information associated with the human actions to train the associated robot in the edge system comprises video clips, each of video chips associated with a subtask from the plurality of subtasks;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the outputting the ones of the one or more subtask sequences to train the associated robot comprises providing ones of the video clips associated with ones of the subtasks associated with the each of the one or more subtask sequences.” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 3
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the robot comprises robot vision configured to record video from which the video clips are generated;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein a manufacturing system is configured to provide a task involving the plurality of subtasks to the edge system for execution and to provide a quality evaluation of the task for the evaluation of the each of the one or more subtask sequences.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the robot comprises robot vision configured to record video from which the video clips are generated;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein a manufacturing system is configured to provide a task involving the plurality of subtasks to the edge system for execution and to provide a quality evaluation of the task for the evaluation of the each of the one or more subtask sequences.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 4
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the video clips comprises the human actions of the plurality of subtasks.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the video clips comprises the human actions of the plurality of subtasks.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 5
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“further comprising recognizing each of the plurality of subtasks based on change point detection to the human actions as determined from feature extraction, wherein detected change points from the change point detection are utilized to separate the each of the plurality of subtasks by tine period.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to, while using a computer as a tool, divide a video stream into different subsections based on observations made of the stream and time stamps. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 6
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the video clips are recorded by a camera that is separate from the robot.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the video clips are recorded by a camera that is separate from the robot.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 7 (Cancelled)
Claim 8
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“where the using the evaluation of the each of the one or more subtask sequences to train the associated robot comprises: selecting ones of the one or more subtask sequences based on the outputted evaluation and frequency of the each of the one or more subtask sequences;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to review a segment of video data and select a subsection of the video based on an evaluation of that data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“extracting video frames corresponding to each of the selected ones of the one or more subtask sequences;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a stream of data and select frames within that stream which correspond to specific actions or tasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)( c).
“segmenting actions from the extracted video frames;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a stream of data and select frames within that stream which correspond to specific actions or tasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determining trajectory and trajectory parameters for the associated robot from the segmented actions; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to mathematical concepts and produce a function which is able to perform specified actions such as determining trajectories and motions of robotic systems. This claim discloses a math operation and therefore is ineligible.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “executing reinforcement learning on the associated robot based on the trajectory, the trajectory parameters, and the segmented actions to learn the selected ones of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “executing reinforcement learning on the associated robot based on the trajectory, the trajectory parameters, and the segmented actions to learn the selected ones of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 9
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 9 recites, "A non-transitory computer readable medium, storing instructions for executing a process comprising:" therefore it is directed to the statutory category of a machine.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“conducting a quality evaluation on each of the plurality of subtasks, wherein the quality evaluation comprises a binary quality check indicating pass or fail of the product following completion of a subtask;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a set of tasks in a video steam and would be able, with using a computer as a tool, perform a quality check and evaluation on the video data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determining one or more subtask sequences from the plurality of subtasks;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to, using a computer as a tool, evaluate a stream of video data containing human motions and segment that data into induvial and different subtasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“evaluating each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a set of tasks in a video steam and would be able, with using a computer as a tool, perform a quality check and evaluation on the video data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“wherein the evaluating the each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences comprises: constructing a function configured to predict a final quality evaluation of the product for the one or more subtask sequences from the quality evaluations of the plurality of subtasks associated with the each of the one or more subtask sequences,” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to generate and use a mathematical function to make a prediction. This claim discloses a math operation and therefore is ineligible.
“wherein the function is constructed using reinforcement learning with rewards based on prediction accuracy of the final quality evaluation compared to actual quality checks from a manufacturing system;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human can use a mathematic concepts such as reinforcement learning to develop an evaluation and prediction method. This claim discloses a math operation and therefore is ineligible.
“utilizing a validation set to evaluate the final quality evaluation for the each of the one or more subtask sequences and generate a prediction reward;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to validate information using observations and evaluation of a validation set. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“modifying the function based on the evaluation of the final quality evaluation for the each of the one or more subtask sequences based on the prediction reward;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a mathematical function and modify it to produce the desired result. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“iteratively repeating the constructing, utilizing, and modifying to finalize the function through reinforced learning;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to iterate through a set of tasks and perform the steps above. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “receiving information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
“outputting ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “receiving information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
“outputting ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 10
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the information associated with the human actions to train the associated robot in the edge system comprises video clips, each of the video clips associated with a subtask from the plurality of subtasks;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the outputting the ones of the one or more subtask sequences to train the associated robot comprises providing ones of the video clips associated with ones of the subtasks associated with the each of the one or more subtask sequences.” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the information associated with the human actions to train the associated robot in the edge system comprises video clips, each of the video clips associated with a subtask from the plurality of subtasks;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the outputting the ones of the one or more subtask sequences to train the associated robot comprises providing ones of the video clips associated with ones of the subtasks associated with the each of the one or more subtask sequences.” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 11
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the robot comprises robot vision configured to record video from which the video clips are generated;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein a manufacturing system is configured to provide a task involving the plurality of subtasks to the edge system for execution and to provide a quality evaluation of the task for the evaluation of the each of the one or more subtask sequences.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the robot comprises robot vision configured to record video from which the video clips are generated;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein a manufacturing system is configured to provide a task involving the plurality of subtasks to the edge system for execution and to provide a quality evaluation of the task for the evaluation of the each of the one or more subtask sequences.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 12
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the video clips comprises the human actions of the plurality of subtasks.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the video clips comprises the human actions of the plurality of subtasks.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 13
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“the instructions further comprising recognizing each of the plurality of subtasks based on change point detection to the human actions as determined from feature extraction, wherein detected change points from the change point detection are utilized to separate the each of the plurality of subtasks by time period.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to, while using a computer as a tool, divide a video stream into different subsections based on observations made of the stream and time stamps. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 14
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the video clips are recorded by a camera that is separate from the robot.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the video clips are recorded by a camera that is separate from the robot.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 15 (Cancelled)
Claim 16
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“wherein the using the evaluation of the each of the one or more subtask sequences to train the associated robot comprises: selecting ones of the one or more subtask sequences based on the outputted evaluation and frequency of the each of the one or more subtask sequences;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to review a segment of video data and select a subsection of the video based on an evaluation of that data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“extracting video frames corresponding to each of the selected ones of the one or more subtask sequences;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a stream of data and select frames within that stream which correspond to specific actions or tasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)( c).
“segmenting actions from the extracted video frames;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a stream of data and select frames within that stream which correspond to specific actions or tasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determining trajectory and trajectory parameters for the associated robot from the segmented actions;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to mathematical concepts and produce a function which is able to perform specified actions such as determining trajectories and motions of robotic systems. This claim discloses a math operation and therefore is ineligible.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “executing reinforcement learning on the associated robot based on the trajectory, the trajectory parameters, and the segmented actions to learn the selected ones of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “executing reinforcement learning on the associated robot based on the trajectory, the trajectory parameters, and the segmented actions to learn the selected ones of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 17
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 1 recites, "An apparatus, comprising: a processor, configured to:" therefore it is directed to the statutory category of a machine.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“conduct a quality evaluation on each of the plurality of subtasks, wherein the quality evaluation comprises a binary quality check indicating pass or fail of the product following completion of a subtask;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a set of tasks in a video steam and would be able, with using a computer as a tool, perform a quality check and evaluation on the video data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determine one or more subtask sequences from the plurality of subtasks;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to, using a computer as a tool, evaluate a stream of video data containing human motions and segment that data into induvial and different subtasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“evaluate each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a set of tasks in a video steam and would be able, with using a computer as a tool, perform a quality check and evaluation on the video data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“wherein the evaluate the each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences comprises: constructing a function configured to predict a final quality evaluation of the product for the one or more subtask sequences from the quality evaluations of the plurality of subtasks associated with the each of the one or more subtask sequences,” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to generate and use a mathematical function to make a prediction. This claim discloses a math operation and therefore is ineligible.
“wherein the function is constructed using reinforcement learning with rewards based on prediction accuracy of the final quality evaluation compared to actual quality checks from a manufacturing system;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human can use a mathematic concepts such as reinforcement learning to develop an evaluation and prediction method. This claim discloses a math operation and therefore is ineligible.
“utilizing a validation set to evaluate the final quality evaluation for the each of the one or more subtask sequences and generate a prediction reward;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to validate information using observations and evaluation of a validation set. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“modifying the function based on the evaluation of the final quality evaluation for the each of the one or more subtask sequences based on the prediction reward;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a mathematical function and modify it to produce the desired result. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“iteratively repeating the constructing, utilizing, and modifying to finalize the function through reinforced learning;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to iterate through a set of tasks and perform the steps above. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “receive information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
“output ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “receive information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
“output ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“applying the learned subtask sequences to control robot operations,” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 18
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“performing change point detection on the recorded video to identify time periods based on significant changes in the human actions;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to observe data on a computer and determine different change points in tasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“identifying each of the plurality of subtasks for individual time periods identified by the change point detection;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to observe data and identify task at given timeframes in a video steam. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “recording video of the human actions performing a task using robot vision;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“generating video clips for each identified subtask based on a corresponding time period; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“extracting feature vectors from each video clip using a convolutional neural network to extract spatio-temporal features.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “recording video of the human actions performing a task using robot vision;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“generating video clips for each identified subtask based on a corresponding time period; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“extracting feature vectors from each video clip using a convolutional neural network to extract spatio-temporal features.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 19
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“initializing a probability distribution for each subtask over a set of features;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to observe a set of tasks and initialize a probably of these tasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“sampling a feature vector for each subtask according to the probability distribution;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human can identify a task in a set of tasks. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“clustering the feature vector and applying a threshold to learn a binary quality checker function for each subtask;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to manipulate, organize, and evaluate data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“comparing the predicted final quality evaluation with an actual quality check from a product quality check system to generate the prediction reward; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to compare the results of a task with a given evaluation metric. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “using the binary quality checker functions to predict the final quality evaluation;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“updating the probability distribution for each subtask based on the prediction reward;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein a probability distribution database stores, for each subtask, a task identifier, a subtask identifier, features being used, and a probability estimation indicative of which feature will be more useful for subtask selection.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “using the binary quality checker functions to predict the final quality evaluation;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“updating the probability distribution for each subtask based on the prediction reward;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein a probability distribution database stores, for each subtask, a task identifier, a subtask identifier, features being used, and a probability estimation indicative of which feature will be more useful for subtask selection.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 20
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“for each subtask sequence, calculating a correctness metric based on a difference between the number of times observed and the number of times correct, normalized by a total number of observations;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to calculate values with given mathematical concepts and formulas. This claim discloses a math operation and therefore is ineligible.
“selecting a subtask sequence having a minimum correctness metric among subtask sequences that have been observed more than a threshold number of times; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to select a task in a ordered set and based on a given value. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “receiving multiple subtask sequences from multiple observations of a task, each subtask sequence having been observed a number of times and having been correct a number of times based on the quality evaluation;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
“outputting the selected subtask sequence to train the associated robot.” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “receiving multiple subtask sequences from multiple observations of a task, each subtask sequence having been observed a number of times and having been correct a number of times based on the quality evaluation;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
“outputting the selected subtask sequence to train the associated robot.” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 21
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“segmenting actions from the extracted video frames and assigning unique identifiers associated with the learned subtask sequences;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a sequence of video data and segment that data based on actions performed in the video stream. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“generating trajectory information from the segmented actions, the trajectory information comprising a sequence of waypoints and end-effector poses for robot manipulation;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and alter robotic trajectory vales based on the evaluated data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “extracting video frames from video clips corresponding to the learned subtask sequences;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“training a model using reinforcement learning based on the trajectory information and the end-effector poses;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“testing the trained model in a simulation environment; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“deploying the tested model to control physical robot operations in a manufacturing workcell.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “extracting video frames from video clips corresponding to the learned subtask sequences;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“training a model using reinforcement learning based on the trajectory information and the end-effector poses;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“testing the trained model in a simulation environment; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“deploying the tested model to control physical robot operations in a manufacturing workcell.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 22
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “the edge learning system is one of a plurality of edge learning systems, each edge learning system associated with a respective robot in a respective manufacturing workcell;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“the core learning system is connected to the plurality of edge learning systems via network connections;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“each edge learning system is configured to record human actions in its respective workcell, divide tasks into subtasks, generate subtask videos, and transmit feature vectors and metadata to the core learning system;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“the core learning system is configured to receive subtask information from the plurality of edge learning systems, perform the evaluating of subtask sequences using data aggregated from the plurality of edge learning systems, and distribute evaluated subtask sequences back to each of the plurality of edge learning systems; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“the method further comprises enabling a robot in a workcell without a human worker present to learn tasks using the evaluated subtask sequences received from the core learning system that were derived from observations in other workcells.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “the edge learning system is one of a plurality of edge learning systems, each edge learning system associated with a respective robot in a respective manufacturing workcell;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“the core learning system is connected to the plurality of edge learning systems via network connections;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“each edge learning system is configured to record human actions in its respective workcell, divide tasks into subtasks, generate subtask videos, and transmit feature vectors and metadata to the core learning system;” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“the core learning system is configured to receive subtask information from the plurality of edge learning systems, perform the evaluating of subtask sequences using data aggregated from the plurality of edge learning systems, and distribute evaluated subtask sequences back to each of the plurality of edge learning systems; and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“the method further comprises enabling a robot in a workcell without a human worker present to learn tasks using the evaluated subtask sequences received from the core learning system that were derived from observations in other workcells.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4, 9, 10, 12, 17, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Finn et al., (Finn et al., "One-Shot Visual Imitation Learning via Meta-Learning", 2017, hereinafter "Finn") in view of Duan et al., (Duan et al., "One-Shot Imitation Learning", 2017, hereinafter "Duan") and Liu et al., (Liu et al, “Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems With Heterogeneous Sensor Data”, 2020, hereinafter “Liu”).
Regarding claim 1, Finn discloses, "A method comprising:" (Algorithm 1, pp. 3; This algorithm discloses the method performed by the system. The method will take a set of tasks and evaluate them. After training and evaluating a robotic system is able to perform the tasks shown.)
“receiving information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” (Problem Statement, pp. 3; “In our meta-learning scenario, we consider a distribution over tasks p(T). In the one-shot learning setting, the policy is trained to learn a new task
T
i
drawn from p(T) from only one demonstration generated by
T
i
.” This will receive a set of tasks, which contain different subtasks for the system to learn. These subtasks, as disclosed in the Experiments section of the article can include, pushing, pulling and object location within a field of view. Each of these task can be performed in manufacturing settings.) and (Introduction, pp. 1-2; “The primary contribution of this paper is to demonstrate an approach for one-shot imitation learning from raw pixels. We evaluate our approach on two simulated planar reaching domains, on simulated pushing tasks, and on visual placing tasks on a real robot (See Figure 1). Our approach is able to learn visuomotor policies that can adapt to new task variants using only one visual demonstration, including settings where only a raw video of the demonstration is available without access to the controls applied by the demonstrator. By employing a parameter-efficient meta-learning method, our approach requires a relatively modest number of demonstrations for meta-learning and scales to raw pixel inputs. As a result, our method can successfully be applied to real robotic systems.” This article discloses a method which is able to train a real robotic system to perform given tasks. This method uses visual imitation of actions to learn, this means that the actions performed are shown to the system, these actions, as listed in the experiments section, includes pushing, pulling and object detection. All of these actions are considering human actions.)
“evaluating each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences; and” (Problem Statement, pp. 3; “During meta-training, a task
T
i
is sampled from p(T), the policy is trained using one demonstration from an expert
π
i
⋆
on
T
i
, and then tested on a new demonstration from
π
i
⋆
to determine its training and test error according to the loss
L
.” This method is able to evaluate a task, within a set of tasks. This able to test an output and determine a loss which can be used to refine the system. The is performed on each of the tasks in the distribution p(T) and this is seen in algorithm 1, line 6.)
“outputting ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences (Meta-Imitation Learning with MAML, pp. 4; “Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1. The result of meta-training is a policy that can be adapted to new tasks using a single demonstration. Thus, at meta-test time, a new task T is sampled, one demonstration for that task is provided, and the model is updated to acquire a policy for that task. During meta-test time, a new task might involve new goals or manipulating new, previously unseen objects.” This article discloses a process to train a robot agent. It will intake a set of actions and evaluate them. The end result will be a robotic system able to perform a human ask it was shown. This system will perform the actions in algorithm 1 and then output the updated model parameters for the system.)
“wherein the evaluating the each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences comprises: constructing a function configured to predict a final quality evaluation of the product for the one or more subtask sequences from the quality evaluations of the plurality of subtasks associated with the each of the one or more subtask sequences,” (Algorithm 1, pp. 3; This algorithm discloses the learning process in the article. At line 6 the task is evaluated using equation 2. This equation will determine the trajectories for the different components of the robot. This uses a loss function on the parameters. This teaches the evaluation of each subtask in a set of tasks, seen at line 4 is a for-loop. It teaches the use of a function to evaluate the different tasks and states of the machine and it uses a loss function to evaluate the accuracy or quality of the action.)
“utilizing a validation set to evaluate the final quality evaluation for the each of the one or more subtask sequences and generate a prediction reward;” (Meta-Imitation Learning with MAML, pp. 4; Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1.” In this model each of the tasks are performed two times. The first demonstration a parameter is determined for each task with equation 2, then the second demonstration is used as a comparison. This means each of the tasks in the distribution of task will act as its own training-validation set.)
“modifying the function based on the evaluation of the final quality evaluation for the each of the one or more subtask sequences based on the prediction reward;” (Algorithm 1, pp. 3; This shows the method used in the article. At line 6-8 the model will evaluate a task and then update or modify the parameters of the model. This section of pseudo code is in a for-loop from lines 4-9, which will iterate through each task in the distribution of tasks in p(T).)
“iteratively repeating the constructing, utilizing, and modifying to finalize the function through reinforced learning;” (Algorithm 1, pp. 3; This algorithm shows the learning process proposed in the article. This will take a set of task and parameters and will iterate through each task,
T
i
, in the distribution of p(T) in a for-loop at line 4. Inside this for-loop is an evaluation of the task at line 6 and updating/modifying the parameters in line 7.)
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” (Meta-Imitation Learning with MAML, pp. 4; “For meta-learning, we assume a dataset of demonstrations with at least two demonstrations per task. This data is only used during meta-training; meta-test time assumes only one demonstration for each new task. During meta-training, each meta-optimization step entails the following: A batch of tasks is sampled and two demonstrations are sampled per task. Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1.” As seen in algorithm 1, this will evaluate each of the tasks and update the models’ parameters. This uses gradient decent to evaluate the parameters of the demonstration. The system will use this gradient to update the parameters after the two demonstrations.)
“applying the learned subtask sequences to control robot operations,” (Introduction, pp. 1-2; “The primary contribution of this paper is to demonstrate an approach for one-shot imitation learning from raw pixels. We evaluate our approach on two simulated planar reaching domains, on simulated pushing tasks, and on visual placing tasks on a real robot (See Figure 1). Our approach is able to learn visuomotor policies that can adapt to new task variants using only one visual demonstration, including settings where only a raw video of the demonstration is available without access to the controls applied by the demonstrator. By employing a parameter-efficient meta-learning method, our approach requires a relatively modest number of demonstrations for meta-learning and scales to raw pixel inputs. As a result, our method can successfully be applied to real robotic systems.” The primary goal of the proposed method is to teach a robotic system by demonstration and robotic imitation. This will evaluate sets of tasks and iterate through each subtask to update model parameters to train a robotic system.)
Finn fails to explicitly disclose the remaining elements of this claim. However, Duan discloses, “conducting a quality evaluation on each of the plurality of subtasks, wherein the quality evaluation comprises a product a binary quality check indicating pass or fail of the product following completion of a subtask;” (Problem Formalization, pp. 3-4; “A demonstration
d
~
D
(
t
)
is a sequence of observations and actions
:
d
=
[
o
1
,
a
1
,
o
2
,
a
2
,
…
,
o
T
,
a
T
]
.
. We assume that the distribution of tasks
T
is given, and that we can obtain successful demonstrations for each task. We assume that there is some scalar-valued evaluation function
R
t
(
d
)
(e.g. a binary value indicating success) for each task, although this is not required during training. The objective is to maximize the expected performance of the policy, where the expectation is taken over tasks
r
∈
T
, and demonstrations
d
∈
D
(
t
)
.” This article discloses a training method that trains a system after a demonstration. This method would assume the tasks that are performed are successful and would not require further evaluation. However, methods can be used to evaluate the accuracy of a given sub task. This article compares its training methods with methods such as DAggar which is able to evaluate learned tasks based on thresholds.)
“determining one or more subtask sequences from the plurality of subtasks;” (Problem Formalization, pp. 3; “We denote a distribution of tasks by
T
, an individual task by
t
~
T
, and a distribution of demonstrations for the task t by
D
(
t
)
.” This article discloses a method which uses demonstrations to teach a robotic system. This will intake a distribution of tasks which contains many subtasks.)
Finn and Duan fail to explicitly disclose the remaining elements of this claim. However, Liu discloses, “wherein the function is constructed using reinforcement learning with rewards based on prediction accuracy of the final quality evaluation compared to actual quality checks from a manufacturing system;” (Knowledge Acquiring by Imitation Learning, pp. 3510-3511; “Local robots acquire knowledge through imitation learning in FIL. Imitation learning is commonly posed either as behavioral cloning [14] or as inverse reinforcement learning [15], both of which require demonstrations. Imitation learning has empowered recent advances in learning robotic manipulation tasks by addressing shortcomings of reinforcement learning such as exploration [16] and reward specification [17]. The knowledge acquiring approach used in FIL of local robots belongs to behavioral cloning, which focuses on learning the expert’s policy using supervised learning. The way behavioral cloning works is quite simple. Given demonstrations of robots, we will divide these into state-action pairs. We treat these pairs as i.i.d. examples and finally, we apply supervised learning.” Liu discloses a system that uses reinforcement learning with rewards. This model is able to perform evaluations on tasks and receive updates from a central server.)
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” (Framework of FIL, pp. 3511; “The framework of FIL is performed in Cloud-Robot- Environment setup. There are local robots, cloud servers, communication services and computing device. Local robots learn skills through imitation learning and the cloud server fuses knowledge. We develop a federated learning algorithm to fuse private models into the shared model in the cloud. With the shared model, the cloud server is capable of generating guide models corresponding to requests of local robots. After that, the local robots perform transfer learning based on the guide model. Finally, the final policy will be quickly obtained.” This article discloses a federated learning system that contains edge robots able to evaluate images and process them. Edge models can update parameters by sending parameters to the cloud server aggregator model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Finn, Duan and Liu. Finn teaches a method that uses reinforcement learning and limited demonstrations to teach a robotic system a task. Duan teaches a method called one-shot imitation learning and the goal of the system is teaching a robotic system using limited number of demonstrations. Liu teaches a federated system that is able to use local edge robots who can be trained to complete tasks and use a global server to distribute learned parameters from one edge system to another. One of ordinary skill would have motivation to combine different reinforcement learning methods which are able to teach robots a task after a limited number of demonstrations with a larger system able to execute and connect multiple robotic systems to distribute knowledge to different edge robots, “The performance of controllers based on these policies in key challenging tasks are presented in Fig. 10. The results are summarized in Table II. From the results, we can see that the imitation learning models obtained in cloud robotic system perform significantly better in accuracy, compared with general models that trained by traditional imitation learning without shared knowledge. FIL improves the training process of imitation learning with the help of shared knowledge. There is a pre-trained model from the cloud for transfer in local imitation learning. So, there is no need for local robots to learn from scratch. We present the comparison of train process in Fig. 10. From the figure, we can see that the transferred policies have lower error starting point and the error value.” (Liu, Evaluation for the Knowledge-Transfer Ability of FIL, pp. 3515).
Regarding claim 2, Duan discloses, wherein the information associated with the human actions to train the associated robot in the edge system comprises video clips, each of video chips associated with a subtask from the plurality of subtasks;” (Problem Formalization, p. 3; “We denote a distribution of tasks by
T
, an individual task by
t
~
T
, and a distribution of demonstrations for the task t by
D
(
t
)
. A policy is symbolized by
π
θ
(
a
|
o
,
d
)
, where a is an action, o is an observation, d is a demonstration, and θ are the parameters of the policy. A demonstration
d
~
D
(
t
)
is a sequence of observations and actions:
d
=
[
o
1
,
a
1
,
o
2
,
a
2
,
…
,
o
T
,
a
T
]
. We assume that the distribution of tasks T is given, and that we can obtain successful demonstrations for each task.” This model contains a distribution of demonstrations and tasks. The set of demonstrations contains individual sets of observations and actions. The demonstrations will be used to train the system a corresponding task.)
“wherein the outputting the ones of the one or more subtask sequences to train the associated robot comprises providing ones of the video clips associated with ones of the subtasks associated with the each of the one or more subtask sequences.” (Algorithm, pp. 4; “We start by collecting a set of demonstrations for each task, where we add noise to the actions in order to have wider coverage in the trajectory space. In each training iteration, we sample a list of tasks (with replacement). For each sampled task, we sample a demonstration as well as a small batch of observation-action pairs. The policy is trained to regress against the desired actions when conditioned on the current observation and the demonstration, by minimizing an
l
2
or cross-entropy loss based on whether actions are continuous or discrete.” This system will use the distribution of tasks and will collect a distribution of demonstrations of the tasks. Each of these individual demonstrations within the distribution will represent an observations and actions. This demonstration is then used to train the model the demonstrated task.)
Regarding claim 4, Finn discloses, “wherein the video clips comprises the human actions of the plurality of subtasks.” (Figure 3, pp. 6; “Example tasks from the policy’s perspective. In the top row, each pair of images shows the start and final scenes of the demonstration. The bottom row shows the corresponding scenes of the learned policy roll-out. Left: Given one demonstration of reaching a target of a particular color, the policy must learn to reach for the same color in a new setting. Center: The robot pushes the target object to the goal after seeing a demonstration of pushing the same object toward the goal in a different scene. Right: We provide a demonstration of placing an object on the target, then the robot must place the object on the same target in a new setting.” This model is given recordings of objects and tasks. The actions used in the experiment represent the human actions of object identification and object manipulation.)
Regarding claim 9, Finn discloses, “A non-transitory computer readable medium, storing instructions for executing a process comprising:” (Introduction, pp. 1-2; "The primary contribution of this paper is to demonstrate an approach for one-shot imitation learning from raw pixels. We evaluate our approach on two simulated planar reaching domains, on simulated pushing tasks, and on visual placing tasks on a real robot (See Figure 1). Our approach is able to learn visuomotor policies that can adapt to new task variants using only one visual demonstration, including settings where only a raw video of the demonstration is available without access to the controls applied by the demonstrator. By employing a parameter-efficient meta-learning method, our approach requires a relatively modest number of demonstrations for meta-learning and scales to raw pixel inputs. As a result, our method can successfully be applied to real robotic systems." This system uses a generic computer to train a robotic system. It is known in the art that a generic computer contains computer instructions to perform actions on that computing system. This article, as shown in figure 1, was designed for use on a robotic system. This system would also contain processor coupled to memory to perform the methods listed in the article.)
“receiving information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system;” (Problem Statement, pp. 3; “In our meta-learning scenario, we consider a distribution over tasks p(T). In the one-shot learning setting, the policy is trained to learn a new task
T
i
drawn from p(T) from only one demonstration generated by
T
i
.” This will receive a set of tasks, which contain different subtasks for the system to learn. These subtasks, as disclosed in the Experiments section of the article can include, pushing, pulling and object location within a field of view. Each of these task can be performed in manufacturing settings.) and (Introduction, pp. 1-2; “The primary contribution of this paper is to demonstrate an approach for one-shot imitation learning from raw pixels. We evaluate our approach on two simulated planar reaching domains, on simulated pushing tasks, and on visual placing tasks on a real robot (See Figure 1). Our approach is able to learn visuomotor policies that can adapt to new task variants using only one visual demonstration, including settings where only a raw video of the demonstration is available without access to the controls applied by the demonstrator. By employing a parameter-efficient meta-learning method, our approach requires a relatively modest number of demonstrations for meta-learning and scales to raw pixel inputs. As a result, our method can successfully be applied to real robotic systems.” This article discloses a method which is able to train a real robotic system to perform given tasks. This method uses visual imitation of actions to learn, this means that the actions performed are shown to the system, these actions, as listed in the experiments section, includes pushing, pulling and object detection. All of these actions are considering human actions.)
“evaluating each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences; and” (Problem Statement, pp. 3; “During meta-training, a task
T
i
is sampled from p(T), the policy is trained using one demonstration from an expert
π
i
⋆
on
T
i
, and then tested on a new demonstration from
π
i
⋆
to determine its training and test error according to the loss
L
.” This method is able to evaluate a task, within a set of tasks. This able to test an output and determine a loss which can be used to refine the system. The is performed on each of the tasks in the distribution p(T) and this is seen in algorithm 1, line 6.)
“outputting ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences,” (Meta-Imitation Learning with MAML, pp. 4; “Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1. The result of meta-training is a policy that can be adapted to new tasks using a single demonstration. Thus, at meta-test time, a new task T is sampled, one demonstration for that task is provided, and the model is updated to acquire a policy for that task. During meta-test time, a new task might involve new goals or manipulating new, previously unseen objects.” This article discloses a process to train a robot agent. It will intake a set of actions and evaluate them. The end result will be a robotic system able to perform a human ask it was shown. This system will perform the actions in algorithm 1 and then output the updated model parameters for the system.)
“wherein the evaluating the each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences comprises: constructing a function configured to predict a final quality evaluation of the product for the one or more subtask sequences from the quality evaluations of the plurality of subtasks associated with the each of the one or more subtask sequences,” (Algorithm 1, pp. 3; This algorithm discloses the learning process in the article. At line 6 the task is evaluated using equation 2. This equation will determine the trajectories for the different components of the robot. This uses a loss function on the parameters. This teaches the evaluation of each subtask in a set of tasks, seen at line 4 is a for-loop. It teaches the use of a function to evaluate the different tasks and states of the machine and it uses a loss function to evaluate the accuracy or quality of the action.)
“utilizing a validation set to evaluate the final quality evaluation for the each of the one or more subtask sequences and generate a prediction reward;” (Meta-Imitation Learning with MAML, pp. 4; Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1.” In this model each of the tasks are performed two times. The first demonstration a parameter is determined for each task with equation 2, then the second demonstration is used as a comparison. This means each of the tasks in the distribution of task will act as its own training-validation set.)
“modifying the function based on the evaluation of the final quality evaluation for the each of the one or more subtask sequences based on the prediction reward;” (Algorithm 1, pp. 3; This shows the method used in the article. At line 6-8 the model will evaluate a task and then update or modify the parameters of the model. This section of pseudo code is in a for-loop from lines 4-9, which will iterate through each task in the distribution of tasks in p(T).)
“iteratively repeating the constructing, utilizing, and modifying to finalize the function through reinforced learning;” (Algorithm 1, pp. 3; This algorithm shows the learning process proposed in the article. This will take a set of task and parameters and will iterate through each task,
T
i
, in the distribution of p(T) in a for-loop at line 4. Inside this for-loop is an evaluation of the task at line 6 and updating/modifying the parameters in line 7.)
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” (Meta-Imitation Learning with MAML, pp. 4; “For meta-learning, we assume a dataset of demonstrations with at least two demonstrations per task. This data is only used during meta-training; meta-test time assumes only one demonstration for each new task. During meta-training, each meta-optimization step entails the following: A batch of tasks is sampled and two demonstrations are sampled per task. Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1.” As seen in algorithm 1, this will evaluate each of the tasks and update the models’ parameters. This uses gradient decent to evaluate the parameters of the demonstration. The system will use this gradient to update the parameters after the two demonstrations.)
“applying the learned subtask sequences to control robot operations,” (Introduction, pp. 1-2; “The primary contribution of this paper is to demonstrate an approach for one-shot imitation learning from raw pixels. We evaluate our approach on two simulated planar reaching domains, on simulated pushing tasks, and on visual placing tasks on a real robot (See Figure 1). Our approach is able to learn visuomotor policies that can adapt to new task variants using only one visual demonstration, including settings where only a raw video of the demonstration is available without access to the controls applied by the demonstrator. By employing a parameter-efficient meta-learning method, our approach requires a relatively modest number of demonstrations for meta-learning and scales to raw pixel inputs. As a result, our method can successfully be applied to real robotic systems.” The primary goal of the proposed method is to teach a robotic system by demonstration and robotic imitation. This will evaluate sets of tasks and iterate through each subtask to update model parameters to train a robotic system.)
Finn fails to explicitly disclose the remaining elements of this claim. However, Duan discloses, “conducting a quality evaluation on each of the plurality of subtasks, wherein the quality evaluation comprises a binary quality check indicating pass or fail of the product following completion of a subtask;” (Problem Formalization, pp. 3-4; “A demonstration
d
~
D
(
t
)
is a sequence of observations and actions
:
d
=
[
o
1
,
a
1
,
o
2
,
a
2
,
…
,
o
T
,
a
T
]
.
. We assume that the distribution of tasks
T
is given, and that we can obtain successful demonstrations for each task. We assume that there is some scalar-valued evaluation function
R
t
(
d
)
(e.g. a binary value indicating success) for each task, although this is not required during training. The objective is to maximize the expected performance of the policy, where the expectation is taken over tasks
r
∈
T
, and demonstrations
d
∈
D
(
t
)
.” This article discloses a training method that trains a system after a demonstration. This method would assume the tasks that are performed are successful and would not require further evaluation. However, methods can be used to evaluate the accuracy of a given sub task. This article compares its training methods with methods such as DAggar which is able to evaluate learned tasks based on thresholds.)
“determining one or more subtask sequences from the plurality of subtasks;” (Problem Formalization, pp. 3; “We denote a distribution of tasks by
T
, an individual task by
t
~
T
, and a distribution of demonstrations for the task t by
D
(
t
)
.” This article discloses a method which uses demonstrations to teach a robotic system. This will intake a distribution of tasks which contains many subtasks.)
Finn and Duan fail to explicitly disclose the remaining elements of this claim. However, Liu discloses, “wherein the function is constructed using reinforcement learning with rewards based on prediction accuracy of the final quality evaluation compared to actual quality checks from a manufacturing system;” (Knowledge Acquiring by Imitation Learning, pp. 3510-3511; “Local robots acquire knowledge through imitation learning in FIL. Imitation learning is commonly posed either as behavioral cloning [14] or as inverse reinforcement learning [15], both of which require demonstrations. Imitation learning has empowered recent advances in learning robotic manipulation tasks by addressing shortcomings of reinforcement learning such as exploration [16] and reward specification [17]. The knowledge acquiring approach used in FIL of local robots belongs to behavioral cloning, which focuses on learning the expert’s policy using supervised learning. The way behavioral cloning works is quite simple. Given demonstrations of robots, we will divide these into state-action pairs. We treat these pairs as i.i.d. examples and finally, we apply supervised learning.” Liu discloses a system that uses reinforcement learning with rewards. This model is able to perform evaluations on tasks and receive updates from a central server.)
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” (Framework of FIL, pp. 3511; “The framework of FIL is performed in Cloud-Robot- Environment setup. There are local robots, cloud servers, communication services and computing device. Local robots learn skills through imitation learning and the cloud server fuses knowledge. We develop a federated learning algorithm to fuse private models into the shared model in the cloud. With the shared model, the cloud server is capable of generating guide models corresponding to requests of local robots. After that, the local robots perform transfer learning based on the guide model. Finally, the final policy will be quickly obtained.” This article discloses a federated learning system that contains edge robots able to evaluate images and process them. Edge models can update parameters by sending parameters to the cloud server aggregator model.)
Regarding claim 10, Duan discloses, “wherein the information associated with the human actions to train the associated robot in the edge system comprises video clips, each of the video clips associated with a subtask from the plurality of subtasks;” (Problem Formalization, p. 3; “We denote a distribution of tasks by
T
, an individual task by
t
~
T
, and a distribution of demonstrations for the task t by
D
(
t
)
. A policy is symbolized by
π
θ
(
a
|
o
,
d
)
, where a is an action, o is an observation, d is a demonstration, and θ are the parameters of the policy. A demonstration
d
~
D
(
t
)
is a sequence of observations and actions:
d
=
[
o
1
,
a
1
,
o
2
,
a
2
,
…
,
o
T
,
a
T
]
. We assume that the distribution of tasks T is given, and that we can obtain successful demonstrations for each task.” This model contains a distribution of demonstrations and tasks. The set of demonstrations contains individual sets of observations and actions. The demonstrations will be used to train the system a corresponding task.)
“wherein the outputting the ones of the one or more subtask sequences to train the associated robot comprises providing ones of the video clips associated with ones of the subtasks associated with the each of the one or more subtask sequences.” (Algorithm, pp. 4; “We start by collecting a set of demonstrations for each task, where we add noise to the actions in order to have wider coverage in the trajectory space. In each training iteration, we sample a list of tasks (with replacement). For each sampled task, we sample a demonstration as well as a small batch of observation-action pairs. The policy is trained to regress against the desired actions when conditioned on the current observation and the demonstration, by minimizing an
l
2
or cross-entropy loss based on whether actions are continuous or discrete.” This system will use the distribution of tasks and will collect a distribution of demonstrations of the tasks. Each of these individual demonstrations within the distribution will represent an observations and actions. This demonstration is then used to train the model the demonstrated task.)
Regarding claim 12, Finn discloses, “wherein the video clips comprises the human actions of the plurality of subtasks.” (Figure 3, pp. 6; “Example tasks from the policy’s perspective. In the top row, each pair of images shows the start and final scenes of the demonstration. The bottom row shows the corresponding scenes of the learned policy roll-out. Left: Given one demonstration of reaching a target of a particular color, the policy must learn to reach for the same color in a new setting. Center: The robot pushes the target object to the goal after seeing a demonstration of pushing the same object toward the goal in a different scene. Right: We provide a demonstration of placing an object on the target, then the robot must place the object on the same target in a new setting.” This model is given recordings of objects and tasks. The actions used in the experiment represent the human actions of object identification and object manipulation.)
Regarding claim 17, Finn discloses, “An apparatus, comprising: a processor, configured to:” (Real-World Placing, pp. 8; "With this goal in mind, we designed a robotic placing experiment using a 7-DoF PR2 robot arm and RGB camera, where the goal is to place a held item into a target container, such as a cup, plate, or bowl, while ignoring two distractors. We collected roughly 1300 demonstrations for meta-training using a diverse range of objects, and evaluated one-shot learning using held-out, unseen objects (see Figure 5). The policy is provided a single visual demonstration of placing the held item onto the target, but with varied positions of the target and distractors, as illustrated in Figure 3. Demonstrations were collected using human teleoperation through a motion controller and virtual reality headset [34], and each demo included the camera video, the sequence of end-effector poses, and the sequence of actions - the end-effector linear and angular velocities." This is one of the experiments performed using this system. This system comprises of a generic computer, robotic arm and a camera to execute demonstrated actions. It is known in the art that a generic computer contains one or more processors connected to memory to execute instructions stored in that memory.)
“receive information associated with a plurality of subtasks for performing product manufacturing, the received information associated with human actions to train an associated robot in an edge system” (Problem Statement, pp. 3; “In our meta-learning scenario, we consider a distribution over tasks p(T). In the one-shot learning setting, the policy is trained to learn a new task
T
i
drawn from p(T) from only one demonstration generated by
T
i
.” This will receive a set of tasks, which contain different subtasks for the system to learn. These subtasks, as disclosed in the Experiments section of the article can include, pushing, pulling and object location within a field of view. Each of these task can be performed in manufacturing settings.) and (Introduction, pp. 1-2; “The primary contribution of this paper is to demonstrate an approach for one-shot imitation learning from raw pixels. We evaluate our approach on two simulated planar reaching domains, on simulated pushing tasks, and on visual placing tasks on a real robot (See Figure 1). Our approach is able to learn visuomotor policies that can adapt to new task variants using only one visual demonstration, including settings where only a raw video of the demonstration is available without access to the controls applied by the demonstrator. By employing a parameter-efficient meta-learning method, our approach requires a relatively modest number of demonstrations for meta-learning and scales to raw pixel inputs. As a result, our method can successfully be applied to real robotic systems.” This article discloses a method which is able to train a real robotic system to perform given tasks. This method uses visual imitation of actions to learn, this means that the actions performed are shown to the system, these actions, as listed in the experiments section, includes pushing, pulling and object detection. All of these actions are considering human actions.)
“evaluate each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences; and” (Problem Statement, pp. 3; “During meta-training, a task
T
i
is sampled from p(T), the policy is trained using one demonstration from an expert
π
i
⋆
on
T
i
, and then tested on a new demonstration from
π
i
⋆
to determine its training and test error according to the loss
L
.” This method is able to evaluate a task, within a set of tasks. This able to test an output and determine a loss which can be used to refine the system. The is performed on each of the tasks in the distribution p(T) and this is seen in algorithm 1, line 6.)
“output ones of the one or more subtask sequences to train the associated robot based on the evaluation of the each of the one or more subtask sequences,” (Meta-Imitation Learning with MAML, pp. 4; “Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1. The result of meta-training is a policy that can be adapted to new tasks using a single demonstration. Thus, at meta-test time, a new task T is sampled, one demonstration for that task is provided, and the model is updated to acquire a policy for that task. During meta-test time, a new task might involve new goals or manipulating new, previously unseen objects.” This article discloses a process to train a robot agent. It will intake a set of actions and evaluate them. The end result will be a robotic system able to perform a human ask it was shown. This system will perform the actions in algorithm 1 and then output the updated model parameters for the system.)
“wherein the evaluate the each of the one or more subtask sequences based on the quality evaluation of the each of the plurality of subtasks associated with the each of the one or more subtask sequences comprises: constructing a function configured to predict a final quality evaluation of the product for the one or more subtask sequences from the quality evaluations of the plurality of subtasks associated with the each of the one or more subtask sequences,” (Algorithm 1, pp. 3; This algorithm discloses the learning process in the article. At line 6 the task is evaluated using equation 2. This equation will determine the trajectories for the different components of the robot. This uses a loss function on the parameters. This teaches the evaluation of each subtask in a set of tasks, seen at line 4 is a for-loop. It teaches the use of a function to evaluate the different tasks and states of the machine and it uses a loss function to evaluate the accuracy or quality of the action.)
“utilizing a validation set to evaluate the final quality evaluation for the each of the one or more subtask sequences and generate a prediction reward;” (Meta-Imitation Learning with MAML, pp. 4; Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1.” In this model each of the tasks are performed two times. The first demonstration a parameter is determined for each task with equation 2, then the second demonstration is used as a comparison. This means each of the tasks in the distribution of task will act as its own training-validation set.)
“modifying the function based on the evaluation of the final quality evaluation for the each of the one or more subtask sequences based on the prediction reward;” (Algorithm 1, pp. 3; This shows the method used in the article. At line 6-8 the model will evaluate a task and then update or modify the parameters of the model. This section of pseudo code is in a for-loop from lines 4-9, which will iterate through each task in the distribution of tasks in p(T).)
“iteratively repeating the constructing, utilizing, and modifying to finalize the function through reinforced learning;” (Algorithm 1, pp. 3; This algorithm shows the learning process proposed in the article. This will take a set of task and parameters and will iterate through each task,
T
i
, in the distribution of p(T) in a for-loop at line 4. Inside this for-loop is an evaluation of the task at line 6 and updating/modifying the parameters in line 7.)
“executing the finalized function to evaluate the each of the one or more subtask sequences; and” (Meta-Imitation Learning with MAML, pp. 4; “For meta-learning, we assume a dataset of demonstrations with at least two demonstrations per task. This data is only used during meta-training; meta-test time assumes only one demonstration for each new task. During meta-training, each meta-optimization step entails the following: A batch of tasks is sampled and two demonstrations are sampled per task. Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1.” As seen in algorithm 1, this will evaluate each of the tasks and update the models’ parameters. This uses gradient decent to evaluate the parameters of the demonstration. The system will use this gradient to update the parameters after the two demonstrations.)
“applying the learned subtask sequences to control robot operations,” (Introduction, pp. 1-2; “The primary contribution of this paper is to demonstrate an approach for one-shot imitation learning from raw pixels. We evaluate our approach on two simulated planar reaching domains, on simulated pushing tasks, and on visual placing tasks on a real robot (See Figure 1). Our approach is able to learn visuomotor policies that can adapt to new task variants using only one visual demonstration, including settings where only a raw video of the demonstration is available without access to the controls applied by the demonstrator. By employing a parameter-efficient meta-learning method, our approach requires a relatively modest number of demonstrations for meta-learning and scales to raw pixel inputs. As a result, our method can successfully be applied to real robotic systems.” The primary goal of the proposed method is to teach a robotic system by demonstration and robotic imitation. This will evaluate sets of tasks and iterate through each subtask to update model parameters to train a robotic system.)
Finn fails to explicitly disclose the remaining elements of this claim. However, Duan discloses, “conduct a quality evaluation on each of the plurality of subtasks, wherein the quality evaluation comprises a binary quality check indicating pass or fail of the product following completion of a subtask;” (Problem Formalization, pp. 3-4; “A demonstration
d
~
D
(
t
)
is a sequence of observations and actions
:
d
=
[
o
1
,
a
1
,
o
2
,
a
2
,
…
,
o
T
,
a
T
]
.
. We assume that the distribution of tasks
T
is given, and that we can obtain successful demonstrations for each task. We assume that there is some scalar-valued evaluation function
R
t
(
d
)
(e.g. a binary value indicating success) for each task, although this is not required during training. The objective is to maximize the expected performance of the policy, where the expectation is taken over tasks
r
∈
T
, and demonstrations
d
∈
D
(
t
)
.” This article discloses a training method that trains a system after a demonstration. This method would assume the tasks that are performed are successful and would not require further evaluation. However, methods can be used to evaluate the accuracy of a given sub task. This article compares its training methods with methods such as DAggar which is able to evaluate learned tasks based on thresholds.)
“determine one or more subtask sequences from the plurality of subtasks;” (Problem Formalization, pp. 3; “We denote a distribution of tasks by
T
, an individual task by
t
~
T
, and a distribution of demonstrations for the task t by
D
(
t
)
.” This article discloses a method which uses demonstrations to teach a robotic system. This will intake a distribution of tasks which contains many subtasks.)
Finn and Duan fail to explicitly disclose the remaining elements of this claim. However, Liu discloses, wherein the function is constructed using reinforcement learning with rewards based on prediction accuracy of the final quality evaluation compared to actual quality checks from a manufacturing system;” (Knowledge Acquiring by Imitation Learning, pp. 3510-3511; “Local robots acquire knowledge through imitation learning in FIL. Imitation learning is commonly posed either as behavioral cloning [14] or as inverse reinforcement learning [15], both of which require demonstrations. Imitation learning has empowered recent advances in learning robotic manipulation tasks by addressing shortcomings of reinforcement learning such as exploration [16] and reward specification [17]. The knowledge acquiring approach used in FIL of local robots belongs to behavioral cloning, which focuses on learning the experts policy using supervised learning. The way behavioral cloning works is quite simple. Given demonstrations of robots, we will divide these into state-action pairs. We treat these pairs as i.i.d. examples and finally, we apply supervised learning.” Liu discloses a system that uses reinforcement learning with rewards. This model is able to perform evaluations on tasks and receive updates from a central server.)
“wherein the edge system comprises an edge learning system configured to process video data locally to identify subtasks and extract feature vectors, and wherein a core learning system receives the feature vectors from multiple edge learning systems and distributes evaluated subtask sequences back to the edge learning systems for robot training.” (Framework of FIL, pp. 3511; “The framework of FIL is performed in Cloud-Robot- Environment setup. There are local robots, cloud servers, communication services and computing device. Local robots learn skills through imitation learning and the cloud server fuses knowledge. We develop a federated learning algorithm to fuse private models into the shared model in the cloud. With the shared model, the cloud server is capable of generating guide models corresponding to requests of local robots. After that, the local robots perform transfer learning based on the guide model. Finally, the final policy will be quickly obtained.” This article discloses a federated learning system that contains edge robots able to evaluate images and process them. Edge models can update parameters by sending parameters to the cloud server aggregator model.)
Regarding claim 22, Liu discloses, “the edge learning system is one of a plurality of edge learning systems, each edge learning system associated with a respective robot in a respective manufacturing workcell;” (Framework of FIL, pp. 9511; “The framework of FIL is performed in Cloud-Robot- Environment setup. There are local robots, cloud servers, communication services and computing device. Local robots learn skills through imitation learning and the cloud server fuses knowledge.” Liu discloses a federated learning environment which contains edge devices and a central cloud server. This system allows for multiple edge devices to learn tasks learned from other edge devices within the network.)
“the core learning system is connected to the plurality of edge learning systems via network connections;” (Framework of FIL, pp. 3511; “Local robots learn skills through imitation learning and the cloud server fuses knowledge. We develop a federated learning algorithm to fuse private models into the shared model in the cloud.” This article discloses a federated environment. This includes a central cloud server which is interpreted to be the core learning system.)
“each edge learning system is configured to record human actions in its respective workcell, divide tasks into subtasks, generate subtask videos, and transmit feature vectors and metadata to the core learning system;” (Framework of FIL, 3511; “Based on the self-driving task, we typically collected three types of data. The three agents use three different types of dataset separately.” The edge devices in the system are able to collect data of different forms such as video data.) and (Framework of FIL, 3511; “Outputs of the three private models are with same types. The inputs to each model are different: RGB images, depth images, and semantic segmentation images. Then the parameters of all three models will be uploaded to the cloud and fused there. Henceforth, the cloud will be capable of generating guide models for different types of input.” This system is able to evaluate data on the edge devices and process models on the cloud server. The edge devices will be able to send model parameters to the cloud server for model aggregation.)
“the core learning system is configured to receive subtask information from the plurality of edge learning systems, perform the evaluating of subtask sequences using data aggregated from the plurality of edge learning systems, and distribute evaluated subtask sequences back to each of the plurality of edge learning systems; and” (Framework of FIL, pp, 3511; “Then the parameters of all three models will be uploaded to the cloud and fused there. Henceforth, the cloud will be capable of generating guide models for different types of input. When a local robot requests a service, the cloud will provide a guide model in correspondence with the type of sensor data.” The parameters form the edge systems are sent to the cloud server for evaluation and aggregation. This model is able to take the segments of data from the different edge models and provide feedback and model updates based on this evaluation.)
“the method further comprises enabling a robot in a workcell without a human worker present to learn tasks using the evaluated subtask sequences received from the core learning system that were derived from observations in other workcells.” (Framework of FIL, pp. 3511; “When a local robot requests a service, the cloud will provide a guide model in correspondence with the type of sensor data. FIL can be performed either online or offline. As presented in Algorithm 1, the whole framework can be summarized as following steps: Step 1: Imitation learning performed by local robots; Step 2: Parameters (of private models) Transmitting; Step 3: Fusing (in the cloud) knowledge; Step 4: Responding to the local requests and generating guide models for them. Noted that step 1 and step 2 are simultaneous while FIL performs online. Labels of the cloud data will be updated simultaneously.” The system in this article discloses that the edge systems are able to video input video data and learn information from them. Other edge systems in the network can learn other task but can also benefit from the global model updates based on the data from other edge systems.)
Claims 3, 6, 11, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Finn, Duan and Liu in view of Barajas et al., (Barajas et al, "VISUAL DEBUGGING OF ROBOTICTASKS", 2014, US 2015/0239127 A1, hereinafter "Barajas").
Regarding claim 3, Barajas discloses, “wherein the robot comprises robot vision configured to record video from which the video clips are generated;” (DETAILED DESCRIPTION, pp. 2, [0022]; "Additionally, an environmental sensor 25 Such as a vision system may be positioned with respect to the robot 12 and configured to film, video tape, image, and/or otherwise record anything in its field of view (arrow V), e.g., the behavior of the robot 12 in its operating environment or work space, as environmental information (arrow 26), which is transmitted to the controller 50", This robot uses visual sensors to assess the environment and the robot. This system takes in visual input and is able to return this input as video clips back to the system.)
“wherein a manufacturing system is configured to provide a task involving the plurality of subtasks to the edge system for execution and to provide a quality evaluation of the task for the evaluation of the each of the one or more subtask sequences.” (Detailed Description, pp. 2, [0017]; "With reference to the drawings, wherein like reference numbers refer to the same or similar components throughout the several views, an example robotic system 10 is shown schematically in FIG. 1 having a robot 12. In an example embodiment, the robot 12 may be a manufacturing robot operable for conducting material handling or assembly tasks, e.g., a 6-axis robot of the type known in the art, or an autonomous dexterous robot having at least 6 degrees of freedom.", This system is designed to execute tasks within a manufacturing setting. This robot can work on an assembly line and perform manufacturing tasks.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Finn, Duan, Liu, and Barajas. Finn teaches a method that uses reinforcement learning and limited demonstrations to teach a robotic system a task. Duan teaches a method called one-shot imitation learning and the goal of the system is teaching a robotic system using limited number of demonstrations. Liu teaches a federated system that is able to use local edge robots who can be trained to complete tasks and use a global server to distribute learned parameters from one edge system to another. Barajas teaches a system which is able to interpret and interact with its surroundings. One of ordinary skill would have motivation to combine different reinforcement learning methods which are able to teach robots a task after a limited number of demonstrations with a larger system able to execute and connect multiple robotic systems to distribute knowledge to different edge robots with a system that is able to teach a robotic system to interact with its environment and learn actions within, "Use of the robotic system 10 described hereinabove with reference to FIGS. 1-4 is intended to provide an approach for visual debugging of robotic tasks via depiction of details of how a robot is acting on an object in its work space. Depiction of the objections in a virtual space in con junction with markers showing planned trajectories of the robot 12 allows a user to see if incorrect goals are planned. A user is thus able to see the intentions of the robot 12 of FIG. 1 before the robot 12 actually takes action. Such an approach may be particularly useful in a robotic system such as the robotic system 10 of FIG. 1 in which the robot 12 is not programmed with specific coordinates and control points, but rather moves to objects as reference points and takes autonomous actions." (Barajas, DETAILED DESCRIPTION, pp. 6).
Regarding claim 6, Barajas discloses, “wherein the video clips are recorded by a camera that is separate from the robot.” (Detailed Description, pp. 2, [0022]; "Additionally, an environmental sensor 25 Such as a vision system may be positioned with respect to the robot 12 and configured to film, video tape, image, and/or otherwise record anything in its field of view (arrow V), e.g., the behavior of the robot 12 in its operating environment or work space, as environmental information (arrow 26), which is transmitted to the controller 50.", This robot uses visual sensors to assess the environment. This video recording device is separate from the actual robot as seen in Figure 1. The Robot is described as #12 and the video recording device is described as #25.)
Regarding claim 11, Barajas discloses, “wherein the robot comprises robot vision configured to record video from which the video clips are generated;” (DETAILED DESCRIPTION, pp. 2, [0022]; "Additionally, an environmental sensor 25 Such as a vision system may be positioned with respect to the robot 12 and configured to film, video tape, image, and/or otherwise record anything in its field of view (arrow V), e.g., the behavior of the robot 12 in its operating environment or work space, as environmental information (arrow 26), which is transmitted to the controller 50", This robot uses visual sensors to assess the environment and the robot. This system takes in visual input and is able to return this input as video clips back to the system.)
“wherein a manufacturing system is configured to provide a task involving the plurality of subtasks to the edge system for execution and to provide a quality evaluation of the task for the evaluation of the each of the one or more subtask sequences.” (Detailed Description, pp. 2, [0017]; "With reference to the drawings, wherein like reference numbers refer to the same or similar components throughout the several views, an example robotic system 10 is shown schematically in FIG. 1 having a robot 12. In an example embodiment, the robot 12 may be a manufacturing robot operable for conducting material handling or assembly tasks, e.g., a 6-axis robot of the type known in the art, or an autonomous dexterous robot having at least 6 degrees of freedom.", This system is designed to execute tasks within a manufacturing setting. This robot can work on an assembly line and perform manufacturing tasks.)
Regarding claim 14, Barajas discloses, “wherein the video clips are recorded by a camera that is separate from the robot.” (Detailed Description, pp. 2, [0022]; "Additionally, an environmental sensor 25 Such as a vision system may be positioned with respect to the robot 12 and configured to film, video tape, image, and/or otherwise record anything in its field of view (arrow V), e.g., the behavior of the robot 12 in its operating environment or work space, as environmental information (arrow 26), which is transmitted to the controller 50.", This robot uses visual sensors to assess the environment. This video recording device is separate from the actual robot as seen in Figure 1. The Robot is described as #12 and the video recording device is described as #25.)
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Finn, Duan, and Liu in view of Fox et al., (Fox et al., "JOINT MODELING OF MULTIPLE TIME SERIES VIA THE BETA PROCESS WITH APPLICATION TO MOTION CAPTURE SEGMENTATION", 2014, hereinafter "Fox").
Regarding claim 5, Fox discloses, further comprising recognizing each of the plurality of subtasks based on change point detection to the human actions as determined from feature extraction, wherein detected change points from the change point detection are utilized to separate the each of the plurality of subtasks by tine period.” (Introduction page 2; "To discover the dynamic behaviors shared between multiple time series, we propose a feature-based model. The entire collection of time series can be described by a globally shared set of possible behaviors. Individually, however, each time series will only exhibit a subset of these behaviors. The goal of joint analysis is to discover which behaviors are shared among the time series and which are unique.", This article discusses how a range of different motions will be captured across a timeframe. The different motions within that time frame will be divided into their own segments and different motions will be registered with matching time frames.; and Page 6, §3.2; the section discloses using feature extraction techniques to extract feature vectors that are then used for change point detection; and Figure 1)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Finn, Duan, Liu, and Fox. Finn teaches a method that uses reinforcement learning and limited demonstrations to teach a robotic system a task. Duan teaches a method called one-shot imitation learning and the goal of the system is teaching a robotic system using limited number of demonstrations. Liu teaches a federated system that is able to use local edge robots who can be trained to complete tasks and use a global server to distribute learned parameters from one edge system to another. Fox teaches a system that is able to take in a stream of motion capture data containing many poses and actions and is able to segment and identify those actions and poses. One of ordinary skill would have motivation to combine different reinforcement learning methods which are able to teach robots a task after a limited number of demonstrations with a larger system able to execute and connect multiple robotic systems to distribute knowledge to different edge robots with a system that is able to take visual data, segment each of the individual poses and actions and identify those actions, "The BP-AR-HM M's accuracy is due to better recovery of the sparse behavior sharing exhibited in the data. This is shown in Figure 6, where we compare estimated binary feature matrices for all methods. In contrast to the sequence specific variability modeled by the BP-AR-HMM, both the GMM and HMM assume that each sequence uses all possible behaviors, which results in the strong vertical bands of white in almost all columns. Overall, the BP-AR-HMM produces superior results due to its flexible feature sharing and allowance for unique behaviors." (Fox, Comparison to alternate time series models, pp. 30)
Regarding claim 13, Fox discloses, “the instructions further comprising recognizing each of the plurality of subtasks based on change point detection to the human actions as determined from feature extraction, wherein detected change points from the change point detection are utilized to separate the each of the plurality of subtasks by time period.” (Introduction page 2; "To discover the dynamic behaviors shared between multiple time series, we propose a feature-based model. The entire collection of time series can be described by a globally shared set of possible behaviors. Individually, however, each time series will only exhibit a subset of these behaviors. The goal of joint analysis is to discover which behaviors are shared among the time series and which are unique.", This article discusses how a range of different motions will be captured across a timeframe. The different motions within that time frame will be divided into their own segments and different motions will be registered with matching time frames.; and Page 6, §3.2; the section discloses using feature extraction techniques to extract feature vectors that are then used for change point detection; and Figure 1)
Claims 8, 16 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Finn, Duan and Liu in view of Huang et al., (Huang et al., "Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration", 2019, hereinafter "Huang").
Regarding claim 8, Finn discloses, “where the using the evaluation of the each of the one or more subtask sequences to train the associated robot comprises:” (Algorithm 1, pp. 3; This algorithm will iterate through a distribution of tasks. It will evaluate each task and train the model based on that evaluation and execution of the task.)
“determining trajectory and trajectory parameters for the associated robot from the segmented actions; and” (Meta-Imitation Learning with MAML, pp. 3-4; “In this section, we describe how we can extend the model-agnostic meta-learning algorithm (MAML) to the imitation learning setting. The model’s input,
o
t
, is the agent’s observation at time t, e.g. an image, whereas the output at is the action taken at time t, e.g. torques applied to the robot’s joints. We will denote a demonstration trajectory as
T
:
=
{
o
1
,
a
1
,
…
o
T
,
a
T
}
and use a mean squared error loss as a function of policy parameters ϕ as follows: [see equation (2)] We will primarily consider the one-shot case, where only a single demonstration
T
(
j
)
is used for the gradient update. However, we can also use multiple demonstrations to resolve ambiguity.” As seen in algorithm 1, equation 2 is used to determine the trajectory for a given task. This is used to train the models parameters on a specified task.)
“executing reinforcement learning on the associated robot based on the trajectory, the trajectory parameters, and the segmented actions to learn the selected ones of the one or more subtask sequences; and” (Algorithm 1, pp. 3; Algorithm teaches the training method used in the article. This will take a set of tasks and evaluates them to train a robotic system using meta-learning. This process uses machine states and rewards to train a model using reinforcement learning. This process is performed on a robotic system so it can imitate different actions.)
“applying the learned subtask sequences to control robot operations.” (Algorithm 1, pp. 3; After the training process is completed, the algorithm will return the trained parameters to the robotic system. This is seen in line 12 of the algorithm.)
Finn, Duan, and Liu fail to explicitly disclose the remaining elements of this claim. However, Huang discloses, “selecting ones of the one or more subtask sequences based on the outputted evaluation and frequency of the each of the one or more subtask sequences;” (Demo Interpreter, pp. 4; “Given the demonstration
d
=
[
o
1
,
…
,
o
T
]
, our goal is to output
A
=
[
a
1
,
…
,
a
K
]
, the sequence of the actions executed in the demonstration as the initial edges in the CTG as shown in Figure 4. We adapt a seq2seq model from machine translation literature [28] as our demo interpreter. We do not use a frame-based classifier, as we do not need accurate per-frame action classification. What is critical here is that the sequence of actions A provides reasonable initial action order constraints (edges) to our conjugate task graph. We do assume the training demonstrations in
T
s
e
e
n
come with the action sequence A as supervision for our demo interpreter. We only require this “flat” supervision for
T
s
e
e
n
, as opposed to the strong hierarchical supervision used in the previous work [41].” The set of demonstrations are used to generate a set of corresponding actions. These demonstrations are segmented and are used to train the model different task sequences. These sequences have set beginning and ending being their changepoints.)
“extracting video frames corresponding to each of the selected ones of the one or more subtask sequences;” (Learning NTG Networks, column 2, page 5; "For the localizer, we use the video frames as input and the corresponding action labels from the demonstrations as targets. For the edge classifier, we collect all pairs of source-target nodes connected by transitions, and use the action label from the demonstration as the target.", This section teaches how the localizer uses the selected frames of the actions of the sequences to further train the system. The actions are records and the different actions are stored as Nodes and Edge Nodes with corresponding video frames.)
“segmenting actions from the extracted video frames;” (Neural Task Graph Execution, column 1 and 2, page 4; "We propose the NTG execution engine that interacts with the environment by executing the task graph. The execution engine executes a task graph in two steps: (i) Node Localization: The execution engine first localizes the current node in the graph based on the visual observation. (ii) Edge Classification: For a given node, there can be multiple outgoing edges for transitions to different actions. The edge classifier checks the (latent) preconditions of each possible next action and picks the most fitting one. These two steps enable the execution engine to use the generated Conjugate Task Graph as a reactive policy which completes the task given observations.", This teaches how the different actions within the video clip are segmented into nodes. Each of these nodes are imported to a graph for further evaluation. The nodes are separated into node and edge nodes which are the different tasks contained throughout the sequence.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Finn, Duan, Liu, and Huang. Finn teaches a method that uses reinforcement learning and limited demonstrations to teach a robotic system a task. Duan teaches a method called one-shot imitation learning and the goal of the system is teaching a robotic system using limited number of demonstrations. Liu teaches a federated system that is able to use local edge robots who can be trained to complete tasks and use a global server to distribute learned parameters from one edge system to another. Huang teaches a method that is able to recognize actions and task in and environment from demonstrations to generate a policy in that environment. One of ordinary skill would have motivation to combine different reinforcement learning methods which are able to teach robots a task after a limited number of demonstrations with a larger system able to execute and connect multiple robotic systems to distribute knowledge to different edge robots with a system that is able to record and evaluate its environment and produce actions based on demonstrations in an environment, "The results are shown in Figure 11. We compare to the no graph variant of our model and also the lower bound of a uniform policy. Unsurprisingly, the uniform policy performs the worst without capturing anything from the demonstration. The no-graph variant is able to capture some parts of the expert policy and better capture the expert demonstration. However, the policy generated by full NTG model substantially improves the NLL and is the most consistent with the expert demonstration." (Huang, Evaluating Real-world Surgical Data, pp. 8)
Regarding claim 16, Finn discloses, wherein the using the evaluation of the each of the one or more subtask sequences to train the associated robot comprises:” (Algorithm 1, pp. 3; This algorithm will iterate through a distribution of tasks. It will evaluate each task and train the model based on that evaluation and execution of the task.)
“determining trajectory and trajectory parameters for the associated robot from the segmented actions;” (Meta-Imitation Learning with MAML, pp. 3-4; “In this section, we describe how we can extend the model-agnostic meta-learning algorithm (MAML) to the imitation learning setting. The model’s input,
o
t
, is the agent’s observation at time t, e.g. an image, whereas the output at is the action taken at time t, e.g. torques applied to the robot’s joints. We will denote a demonstration trajectory as
T
:
=
{
o
1
,
a
1
,
…
o
T
,
a
T
}
and use a mean squared error loss as a function of policy parameters ϕ as follows: [see equation (2)] We will primarily consider the one-shot case, where only a single demonstration
T
(
j
)
is used for the gradient update. However, we can also use multiple demonstrations to resolve ambiguity.” As seen in algorithm 1, equation 2 is used to determine the trajectory for a given task. This is used to train the models parameters on a specified task.)
“executing reinforcement learning on the associated robot based on the trajectory, the trajectory parameters, and the segmented actions to learn the selected ones of the one or more subtask sequences; and” (Algorithm 1, pp. 3; Algorithm teaches the training method used in the article. This will take a set of tasks and evaluates them to train a robotic system using meta-learning. This process uses machine states and rewards to train a model using reinforcement learning. This process is performed on a robotic system so it can imitate different actions.)
“applying the learned subtask sequences to control robot operations.” (Algorithm 1, pp. 3; After the training process is completed, the algorithm will return the trained parameters to the robotic system. This is seen in line 12 of the algorithm.)
Finn, Duan, and Liu fail to explicitly disclose the remaining elements of this claim. However, Huang discloses, “selecting ones of the one or more subtask sequences based on the outputted evaluation and frequency of the each of the one or more subtask sequences;” (Demo Interpreter, pp. 4; “Given the demonstration
d
=
[
o
1
,
…
,
o
T
]
, our goal is to output
A
=
[
a
1
,
…
,
a
K
]
, the sequence of the actions executed in the demonstration as the initial edges in the CTG as shown in Figure 4. We adapt a seq2seq model from machine translation literature [28] as our demo interpreter. We do not use a frame-based classifier, as we do not need accurate per-frame action classification. What is critical here is that the sequence of actions A provides reasonable initial action order constraints (edges) to our conjugate task graph. We do assume the training demonstrations in
T
s
e
e
n
come with the action sequence A as supervision for our demo interpreter. We only require this “flat” supervision for
T
s
e
e
n
, as opposed to the strong hierarchical supervision used in the previous work [41].” The set of demonstrations are used to generate a set of corresponding actions. These demonstrations are segmented and are used to train the model different task sequences. These sequences have set beginning and ending being their changepoints.)
“extracting video frames corresponding to each of the selected ones of the one or more subtask sequences;” (Learning NTG Networks, column 2, page 5; "For the localizer, we use the video frames as input and the corresponding action labels from the demonstrations as targets. For the edge classifier, we collect all pairs of source-target nodes connected by transitions, and use the action label from the demonstration as the target.", This section teaches how the localizer uses the selected frames of the actions of the sequences to further train the system. The actions are records and the different actions are stored as Nodes and Edge Nodes with corresponding video frames.)
“segmenting actions from the extracted video frames;” (Neural Task Graph Execution, column 1 and 2, page 4; "We propose the NTG execution engine that interacts with the environment by executing the task graph. The execution engine executes a task graph in two steps: (i) Node Localization: The execution engine first localizes the current node in the graph based on the visual observation. (ii) Edge Classification: For a given node, there can be multiple outgoing edges for transitions to different actions. The edge classifier checks the (latent) preconditions of each possible next action and picks the most fitting one. These two steps enable the execution engine to use the generated Conjugate Task Graph as a reactive policy which completes the task given observations.", This teaches how the different actions within the video clip are segmented into nodes. Each of these nodes are imported to a graph for further evaluation. The nodes are separated into node and edge nodes which are the different tasks contained throughout the sequence.)
Regarding claim 21, Finn discloses, “generating trajectory information from the segmented actions, the trajectory information comprising a sequence of waypoints and end-effector poses for robot manipulation;” (Meta-Imitation Learning with MAML, pp. 3-4; “In this section, we describe how we can extend the model-agnostic meta-learning algorithm (MAML) to the imitation learning setting. The model’s input,
o
t
, is the agent’s observation at time t, e.g. an image, whereas the output at is the action taken at time t, e.g. torques applied to the robot’s joints. We will denote a demonstration trajectory as
T
:
=
{
o
1
,
a
1
,
…
o
T
,
a
T
}
and use a mean squared error loss as a function of policy parameters ϕ as follows: [see equation (2)] We will primarily consider the one-shot case, where only a single demonstration
T
(
j
)
is used for the gradient update. However, we can also use multiple demonstrations to resolve ambiguity.” As seen in algorithm 1, equation 2 is used to determine the trajectory for a given task. This is used to train the models parameters on a specified task.)
“training a model using reinforcement learning based on the trajectory information and the end-effector poses;” (Algorithm 1 Meta-Imitation Learning with MAML, pp. 3; This model uses a training method called MAML. MAML is a form of reinforcement learning, it uses training periods to train a model to learn new tasks. The variant in this article uses imitation learning.)
“testing the trained model in a simulation environment; and” (Simulated Reaching, pp. 6; “The first experimental domain is a family of planar reaching tasks, as illustrated in Figure 3, where the goal of a particular task is to reach a target of a particular color, amid two distractors with different colors. This simulated domain allows us to rigorously evaluate our method and compare with prior approaches and baselines.” Finn discloses the use of their system in a virtual environment. This is seen in this experiment.)
“deploying the tested model to control physical robot operations in a manufacturing workcell.” (Real-World Placing, pp. 8; “With this goal in mind, we designed a robotic placing experiment using a 7-DoF PR2 robot arm and RGB camera, where the goal is to place a held item into a target container, such as a cup, plate, or bowl, while ignoring two distractors. We collected roughly 1300 demonstrations for meta-training using a diverse range of objects, and evaluated one-shot learning using held-out, unseen objects (see Figure 5). The policy is provided a single visual demonstration of placing the held item onto the target, but with varied positions of the target and distractors, as illustrated in Figure 3.” The article discloses testing the proposed learning method on a live system. This experiment used a robotic system to perform the trained actions of pick and place.)
Finn, Duan, and Liu fail to explicitly disclose the remaining elements of this claim. However, Huang discloses, “wherein the applying the learned subtask sequences to control robot operations comprises: extracting video frames from video clips corresponding to the learned subtask sequences;” (Learning NTG Networks, column 2, page 5; "For the localizer, we use the video frames as input and the corresponding action labels from the demonstrations as targets. For the edge classifier, we collect all pairs of source-target nodes connected by transitions, and use the action label from the demonstration as the target.", This section teaches how the localizer uses the selected frames of the actions of the sequences to further train the system. The actions are records and the different actions are stored as Nodes and Edge Nodes with corresponding video frames.)
“segmenting actions from the extracted video frames and assigning unique identifiers associated with the learned subtask sequences;” (Neural Task Graph Execution, column 1 and 2, page 4; "We propose the NTG execution engine that interacts with the environment by executing the task graph. The execution engine executes a task graph in two steps: (i) Node Localization: The execution engine first localizes the current node in the graph based on the visual observation. (ii) Edge Classification: For a given node, there can be multiple outgoing edges for transitions to different actions. The edge classifier checks the (latent) preconditions of each possible next action and picks the most fitting one. These two steps enable the execution engine to use the generated Conjugate Task Graph as a reactive policy which completes the task given observations." The system in this applicant is able to store actions as graphs. This will save the nodes and evaluate the different functions attached to the nodes in the graph.)
Claims 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Finn, Duan and Liu in view of Roitberg et al, (Roitberg et al, “Multimodal Human Activity Recognition for Industrial Manufacturing Processes in Robotic Workcells.”, 2015, hereinafter “Roitberg”).
Regarding claim 18, Finn discloses, extracting feature vectors from each video clip using a convolutional neural network to extract spatio-temporal features.” (Model Architectures for Meta-Imitation Learning, pp. 5; “We use a convolutional neural network (CNN) to represent the policy, similar to prior vision-based imitation and meta-learning methods [13, 27]. The policy observation includes both the camera image and the robot’s configuration, e.g. the joint angles and end-effector pose. … We include a diagram of the policy architecture with the bias transformation in Figure 2.” Finn uses a CNN to evaluate images from segments of videos. The videos used in Finn represent human actions which can be performed in manufacturing settings.)
Finn, Duan, and Liu fail to explicitly disclose the remaining elements of this claim. However, Roitberg discloses, “recording video of the human actions performing a task using robot vision;” (Sensor Data Synchronization, pp. 261; “We implemented a communication and control framework which unifies the sensor data over multiple machines and operating systems using the Robot Operating System (ROS) and ZeroC ICE. The master server manages the entire input as ROS messages, which can be easily saved and replayed as ROS BAG files. Limitations of the network bandwidth as well as the high resolution of the streamed videos occasionally lead to the loss of Kinect v2 data, which is subsequently corrected with a Kalman filter.” This article discloses a system that is able to take in video data and train robotic systems. This model will intake videos performed at a work station using different recording devices.)
“performing change point detection on the recorded video to identify time periods based on significant changes in the human actions;” (Feature Calculation and Preprocessing, pp. 262; “Due to the dynamic nature of human activities, motion features carry significant information about human behavior. Thus, we capture the change of feature values within certain time segments, i.e., the delta value as well as the variance within a segment.” This model is able to segment sections of video based on time stamps and actions performed. This will save the time frames where the human actions are performed.)
“identifying each of the plurality of subtasks for individual time periods identified by the change point detection;” (Hidden Markov Models, pp. 263; “For our activity recognition approach, we propose a strictly hierarchical two-stage machine learning framework, which combines discriminative and generative classification methods. The lowest level L1 is not subordinate to any other levels and is processed independently from them. Consequently, the hierarchical aspect is not present at this stage and we use a discriminative classifier with a HMM for this stage (Algorithms 1 and 2).” This article uses different machine learning models to identify activities performed in the segments of video.)
“generating video clips for each identified subtask based on a corresponding time period; and” (Kinect v2 Features, pp. 262; “We use the positions of the hands, wrists and elbows and their motion information in the segment of the last 0:5s. We put strong emphasis on the motion of the skeletal angles of the upper body joints as well as the orientation of the hip-, shoulder- and spine-line. Since the Kinect SDK does not provide any velocity information, we calculate the motion information in the last 0:5s, 1 s as well as the motion history in the frame segments of 0:5s to 1s and 1s to 2s.” This article discloses the different segments of time the activities were performed in. This is able to generate the sets of segments to train a robotic system)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Finn, Duan, Liu, and Roitberg. Finn teaches a method that uses reinforcement learning and limited demonstrations to teach a robotic system a task. Duan teaches a method called one-shot imitation learning and the goal of the system is teaching a robotic system using limited number of demonstrations. Liu teaches a federated system that is able to use local edge robots who can be trained to complete tasks and use a global server to distribute learned parameters from one edge system to another. Roitberg teaches a system that uses human activity recognition to train robotic system. One of ordinary skill would have motivation to combine different reinforcement learning methods which are able to teach robots a task after a limited number of demonstrations with a larger system able to execute and connect multiple robotic systems to distribute knowledge to different edge robots with teaching from another system which integrates human activity into trainable actions for industrial settings, “The contributions of this paper are two-fold. Firstly, we designed a multimodal interface integrated in a robotic workcell for HRI, tailored to recognize human activities and gestures. In spite of the diverse data sources and the complex communication framework, the system is easy to maintain and extend due to its compatibility with ROS. Secondly, we proposed a two-stages machine learning approach for the classification of human activities on multiple levels of abstraction. We use PCA for dimensionality reduction and combine a RBF-SVM estimator with a HHMM.” (Conclusion, pp. 266)
Regarding claim 20, Finn discloses, “for each subtask sequence, calculating a correctness metric based on a difference between the number of times observed and the number of times correct, normalized by a total number of observations;” (Algorithm 1 Meta-Imitation Learning with MAML, pp. 3; This algorithm teaches that the performed actions will be evaluated against a validation set. This will determine the updating parameter to train the model. This algorithm discloses the use of a gradient descent.)
“outputting the selected subtask sequence to train the associated robot.” (Meta-Imitation Learning with MAML, pp. 4; “During meta-training, each meta-optimization step entails the following: A batch of tasks is sampled and two demonstrations are sampled per task. Using one of the demonstrations,
θ
i
'
is computed for each task
T
i
using gradient descent with Equation 2. Then, the second demonstration of each task is used to compute the gradient of the meta-objective by using Equation 1 with the loss in Equation 2. Finally, θ is updated according to the gradient of the meta-objective. In effect, the pair of demonstrations serves as a training-validation pair. The algorithm is summarized in Algorithm 1.” Finn discloses that the learned tasks will be updated output to the robot system to train the system the demonstrated tasks. This algorithm discloses how this system is trained.)
Finn, Duan, and Liu fail to explicitly disclose the remaining elements of this claim. However, Roitberg discloses, “receiving multiple subtask sequences from multiple observations of a task, each subtask sequence having been observed a number of times and having been correct a number of times based on the quality evaluation;” (System Setup, pp. 260; “We propose a new multimodal HRI-system, that is integrated in a typical industrial workcell and equipped with three different kinds of sensors (Fig. 2). A Microsoft Kinect v2 RGB-D sensor is mounted in front of the person, facing diagonally from above. The Kinect v2 is used to detect and track the human skeleton. An Asus Xtion Pro is placed on the metal cage facing straight down towards the table, and is intended for detecting objects on the table.” The system in this article will use multiple different sensor to view a demonstration of an activity performed by user. The use can repeat the demonstration and the system will be able to evaluate previously learned tasks use ground truth datasets to confirm accuracy.)
“selecting a subtask sequence having a minimum correctness metric among subtask sequences that have been observed more than a threshold number of times; and” (Scenarios and Activities, pp. 261; “We analyzed the scenarios and derived a four-level hierarchy of activities (Fig. 5). The structure of the hierarchy is based on complexity, duration (which is highly correlated) and generalization. The activity of Level n (
L
n
) is linked by a set of grammar rules to sub-activities from
L
n
+
1
” This system is able to analyze the different divided tasks using different metrics. This will select and analyze each of the scenarios and their associated tasks.)
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Finn, Duan and Liu in view of Yang et al, (Yang et al, “Interactive-Imitation-Based Distributed Coordination Scheme for Smart Manufacturing”, 2021, hereinafter “Yang”).
Regarding claim 19, Finn, Duan, and Liu fail to explicitly disclose the elements of this claim. However, Yang discloses, “initializing a probability distribution for each subtask over a set of features;” (System Model, pp. 3601; “For the sequential production operations, scenarios are broken up into a series of episodes, which include a set of states S characterizing the configurations of all agents as well as the varying environment, a set of observations
O
1
,
…
,
O
N
, and a set of actions
A
1
,
…
,
A
N
for each agent. The function of
P
:
S
×
A
1
×
…
×
A
N
→
P
(
S
)
denotes the stochastic transition between the states, where P(S) denotes the set of probability distributions over S.” This model uses a probability distribution over all sets of actions and observations of states. This is distribution is initialized with the model.)
“sampling a feature vector for each subtask according to the probability distribution;” (System Model, pp. 3601; “Specifically, given the state
s
t
at time t, agents take actions of (
A
1
,
…
,
A
N
) and then
s
t
transits to
s
t
+
1
with the probability of
P
(
s
t
+
1
|
s
t
,
a
1
,
…
,
a
N
)
. According to the actions (i.e., manufacturing operations), agent
i
∈
N
attains an immediate reward
r
i
:
S
×
A
1
×
…
×
A
N
→
R
” This model will select a given event and evaluate it. This teaches that the model will sample a set of features of a task.)
“clustering the feature vector and applying a threshold to learn a binary quality checker function for each subtask;” (System Model, pp. 3201; “By selecting actions through a stochastic policy
π
i
:
O
i
×
A
i
→
0,1
, agent i tries to maximize its total expected return [See Equation (1)] where
γ
denotes the discount factor.” This model will group the sets of tasks and their related observations and states. This will then evaluate the features to learn the quality of the task this will evaluate each of the states and actions.)
“using the binary quality checker functions to predict the final quality evaluation;” (Interagent Imitation, pp. 3602; “To overcome this barrier, we propose a confidence-based matching method to screen out the imperfect expert demonstrations and reduce the gap between the expert and imitation policies. Let
ρ
o
p
t
(
s
,
a
)
and
ρ
n
o
n
(
s
,
a
)
denote the stateaction visitation distributions of the optimal and nonoptimal policies, respectively. Thus,
ρ
s
,
a
=
T
ρ
o
p
t
s
,
a
+
(
1
-
T
)
ρ
n
o
n
(
s
,
a
)
, where T denotes the weight factor of
ρ
o
p
t
(
s
,
a
)
. Let
ρ
'
s
,
a
=
T
ρ
θ
i
s
,
a
+
(
1
-
T
)
ρ
n
o
n
(
s
,
a
)
denote the approximation result of
ρ
(
s
,
a
)
. To minimize the gap (i.e., JSD) between
ρ
(
s
,
a
)
and
ρ
'
s
,
a
is equivalent to minimizing the gap between
ρ
o
p
t
(
s
,
a
)
and
ρ
θ
i
s
,
a
. This model will evaluate the demonstration and ensure accuracy of the policy based on an evaluation. This will try to learn the policies and maximize the accuracy to the expert’s policy without overfitting.)
“comparing the predicted final quality evaluation with an actual quality check from a product quality check system to generate the prediction reward; and” (Imitation-Driven Coordinated Manufacturing, pp. 3201; “The GAIL model learns an imitation policy
π
θ
i
(
i
∈
N
)
directly from capturing behaviors of the expert policy
π
E
[17], [18]. In the GAIL-based coordinated manufacturing scenario, the master industrial device acts as the expert and the slave devices act as the imitators.” This model will evaluate the different demonstrations and train form the expert policy listed in a manufacturing environment.)
“updating the probability distribution for each subtask based on the prediction reward;” (System Model, pp. 3601; “Specifically, given the state
s
t
at time t, agents take actions of (
A
1
,
…
,
A
N
) and then
s
t
transits to
s
t
+
1
with the probability of
P
(
s
t
+
1
|
s
t
,
a
1
,
…
,
a
N
)
. According to the actions (i.e., manufacturing operations), agent
i
∈
N
attains an immediate reward
r
i
:
S
×
A
1
×
…
×
A
N
→
R
” the agent in the system will attain an immediate reward. This will be used to update the model.)
“wherein a probability distribution database stores, for each subtask, a task identifier, a subtask identifier, features being used, and a probability estimation indicative of which feature will be more useful for subtask selection.” (System Model, pp. 3601; “For the sequential production operations, scenarios are broken up into a series of episodes, which include a set of states S characterizing the configurations of all agents as well as the varying environment, a set of observations
O
1
,
…
,
O
N
, and a set of actions
A
1
,
…
,
A
N
for each agent. The function of
P
:
S
×
A
1
×
…
×
A
N
→
P
(
S
)
denotes the stochastic transition between the states, where P(S) denotes the set of probability distributions over S.” The probability distribution would contain the sets of actions, observations, states and other identifying information to the actions performed and the environmental data.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Finn, Duan, Liu, and Yang. Finn teaches a method that uses reinforcement learning and limited demonstrations to teach a robotic system a task. Duan teaches a method called one-shot imitation learning and the goal of the system is teaching a robotic system using limited number of demonstrations. Liu teaches a federated system that is able to use local edge robots who can be trained to complete tasks and use a global server to distribute learned parameters from one edge system to another. Yang teaches imitation based learning systems to train robotic systems and evaluate the quality of the trained policies. One of ordinary skill would have motivation to combine different reinforcement learning methods which are able to teach robots a task after a limited number of demonstrations with a larger system able to execute and connect multiple robotic systems to distribute knowledge to different edge robots with a system that is able to evaluate learned policies to ensure quality control of policies, “As the mixed imperfect demonstrations directly degrade the imitation accuracy, we presented a confidence-based matching method to obtain high-fidelity imitation policies. Second, the SIL model was also leveraged to improve the imitation accuracy by following one agent’s own good historical experiences. As the ambiguous rewards may lead to suboptimal imitation policies, we presented an SVPG-based self-imitation method to learn an expected optimal policy. We verified the effectiveness of the IIDC algorithm by conducting the collaborative task of assembling precision components.”. (Conclusion, pp. 3607).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL MICHAEL GALVIN-SIEBENALER whose telephone number is (571)272-1257. The examiner can normally be reached Monday - Friday 8AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL M GALVIN-SIEBENALER/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights