DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This is a nonfinal in response to an RCE filed on 03/05/2026. Claims 1-13 remain pending. Claims 1 and 8 have been amended.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Park et al “A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder” in view of Wang et al (US 20210197335 A1) and AlRegib et al (US 20220327389 A1) (Hereinafter referred to as Park, Wang, and AlRegib respectively)
Regarding Claim 1, Park teaches a system for detecting off-nominal behavior (See at least Park Page 1 Abstract, the anomalous execution is interpreted as off-nominal behavior) comprising:
a trained manifold developed with a target behavior dataset (See at least Park Page 4 C. Training and Testing Framework and Page 3 Figure 2, the trained LSTM-VAE-based anomaly detector is interpreted as the trained manifold, which is developed with a sequence of non-anomalous/target behavior dataset) , wherein said target behavior data set corresponds to one or more tasks to be completed by a robotic system over a time interval (See at least Park Pages 1-2 Introduction, Page 4 C. Training and Testing Framework, and Page 3 Figure 2, the non-anomalous/target behavior dataset corresponds to a task to be completed by a robot in sequences over a time interval);
an encoder configured to receive at least one initial data set (See at least Park Page 3 C. An LSTM-based Variational Autoencoder and Figure 2, and Pages 6-7 Evaluation and Figure 6, the encoder receives an initial multimodal input at different times), said encoder being further configured to generate a lower dimensional data set from said at least one initial data set (See at least Park Page 1 Introduction, during encoding, the multimodal observations/initial data set is projected into a latent space, which is a lower dimensional data set), wherein said at least one initial data set corresponds to a performance of a first task of said one or more tasks over said time interval (See at least Park Page 1 Introduction, and Pages 5-7, V Experimental Setup and VI Evaluation, the initial data set/multimodal observation corresponds to a robot performing a task over a time interval);
a generator configured to generate a reconstructed data set by mapping said lower dimensional data set onto said trained manifold (See at least Park Page 1 Introduction, Page 2 A. Preliminary Autoencoder, and Page 3 Figure 2, the decoder/generator generates a reconstructed data set using the lower dimensional/compressed data set)…; and
…measure a divergence between said initial data set and said reconstructed data set along said time interval as said at least one initial data set is received, wherein a divergence beyond a pre-determined threshold is indicative of off-nominal behavior (See at least Park Page 4 IV Anomaly Detection and Pages 6-7 VI Evaluation and Figure 6, the deviation between the initial/observed data set and the reconstructed/predicted data set beyond a threshold value indicates anomalous execution/off-nominal behavior).
Park fails to disclose said generator being adversarially trained with a discriminator.
However, Wang teaches said generator being adversarially trained with a discriminator (See at least Wang Paragraphs 0010 and 0022-0026, the generator is trained with a discriminator during the adversarial training process).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in Park with Wang to adversarially train the generator with a discriminator. This modification, as taught by Wang, would allow the system to reach Nash equilibrium, so that the sample data generated by the generator is no different from the real sample data, and the discriminator cannot distinguish the generated sample data from the real sample data (See at least Wang Paragraph 0010).
Modified Park fails to explicitly disclose a classifier configured to receive feature representations, said feature representations comprising at least a difference between a first feature of said at least one initial data set and a second feature of said reconstructed data set, said classifier being further configured to measure a divergence between said initial data set and said reconstructed data set.
However, AlRegib teaches a classifier configured to receive feature representations, said feature representations comprising at least a difference between a first feature of said at least one initial data set and a second feature of said reconstructed data set (See at least AlRegib Paragraphs 0005, 0007, 0009, and 0028, the discriminator, which is a type of classifier, measures similarity and feature matching between the initial data and the reconstructed data to determine a reconstruction loss), said classifier being further configured to measure a divergence between said initial data set and said reconstructed data set (See at least AlRegib Paragraphs 0005, 0007, 0009, 0030, and 0032, the discriminator/classifier measures the reconstruction error/divergence between the reconstructed data and the initial data).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with AlRegib to utilize a classifier to measure a divergence between said initial data set and said reconstructed data set. Classifiers are utilized as an anomaly detector by comparing the reconstructed data with original data (See at least AlRegib Paragraph 0009). By using a classifier to measure the divergence, as taught by AlRegib, the system can detect an abnormality by measuring the divergence/reconstruction error between the initial data and the reconstructed data (See at least AlRegib Paragraph 0032), which would improve the autonomy of the system.
Regarding Claim 2, modified Park teaches said target behavior dataset includes only nominal behavior data (See at least Park Page 4 C. Training and Testing Framework, the target behavior dataset includes non-anomalous training data, which is interpreted as nominal behavior data).
Regarding Claim 3, modified Park teaches said at least one initial data set includes a plurality of measured state of said robotic system over said time interval (See at least Park Pages 6-7, VI Evaluation and Figure 6, the current observations are a plurality of measured states of the robot over said time interval) and said nominal behavior data corresponds to a set of target states for said robotic system over said time interval (See at least Park Page 4 C. Training and Testing Framework, the nominal behavior data/non-anomalous training corresponds to target states over said time interval).
Regarding Claim 5, modified Park fails to disclose said classifier is a fully-connected classifier.
However, Wang teaches said classifier is a fully-connected classifier (See at least Wang Paragraph 0013, the discriminator/classifier is a fully-connected neural network).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with Wang to have the classifier be a fully-connected classifier. Fully connected classifiers are conventional and well-understood in the art, and therefore, one of ordinary skill in the art would be motivated to utilize a fully connected classifier as the classifier.
Claims 4 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Wang and AlRegib, and in further view of Kranski et al (US 20210197335 A1) (Hereinafter referred to as Kranski)
Regarding Claim 4, modified Park fails to disclose said nominal behavior data comprises simulated behavior data.
However, Kranski teaches said nominal behavior data comprises simulated behavior data (See at least Kranski Paragraphs 0022 and 0061, the nominal behavior data to complete a task is simulated).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with Kranski to have the nominal behavior data comprise simulated behavior data. This modification, as taught by Kranski, would allow the system to train the robot to perform the task through simulated attempts rather than actually controlling a real robot (See at least Kranski Paragraphs 0022 and 0061), which would improve the efficiency and cost of the training.
Regarding Claim 7, modified Park teaches a robotics control platform coupled to said robotic system (See at least Park Page 5 C. Experimental Procedure, the user interface is interpreted as the robotics control platform), wherein…cause said robotic system to terminate an operation upon…determines divergence of said initial data set and said reconstructed data set beyond said pre-determined threshold (See at least Park Page 1 Introduction, Page 4 IV Anomaly Detection and Pages 6-7 VI Evaluation and Figure 6, the robot is stopped during anomalous task execution, which is determined based on the deviation between the initial/observed data set and the reconstructed/predicted data set beyond a threshold value).
Modified Park fails to explicitly disclose said classifier determines a divergence of said initial data set and said reconstructed data set.
However, AlRegib teaches said classifier determines a divergence of said initial data set and said reconstructed data set (See at least AlRegib Paragraphs 0005, 0007, 0009, 0030, and 0032 the discriminator, which is a type of classifier, measures the reconstruction error/divergence between the reconstructed data and the initial data).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with AlRegib to utilize a classifier to determine a divergence between said initial data set and said reconstructed data set. Classifiers are utilized as an anomaly detector by comparing the reconstructed data with original data (See at least AlRegib Paragraph 0009). By using a classifier to measure the divergence, as taught by AlRegib, the system can detect an abnormality by measuring the divergence/reconstruction error between the initial data and the reconstructed data (See at least AlRegib Paragraph 0032), which would improve the autonomy of the system.
Modified Park fails to explicitly disclose wherein said robotics control platform is configured to cause said robotic system to terminate an operation.
However, Kranski teaches wherein said robotics control platform is configured to cause said robotic system to terminate an operation (See at least Kranski Paragraph 0061, the control platform/teaching subsystem stops/terminates an operation of the robot).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with Kranski to have the robotics control platform cause said robotic system to terminate an operation. This modification, as taught by Kranski, would allow an operator to stop the operation of the robot when the robot failed at completing the task, which would prevent damage to the robot (See at least Kranski Paragraph 0061).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Wang and AlRegib, and in further view of Rozantsev et al (US 20220230376 A1) (Hereinafter referred to as Rozantsev)
Regarding Claim 6, modified Park fails to disclose said classifier is a long short-term memory classifier.
However, Rozantsev teaches said classifier is a long short-term memory classifier (See at least Rozantsev Paragraph 0040, the discriminator/classifier utilizes a LSTM unit).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Kranski with Rozantsev to have the classifier be a long short-term memory classifier. This modification, as taught by Rozantsev, would allow the system to efficiently gather information from the predictions of a prediction network for multiple consecutive time steps, and predict whether the sequences of input data is generated or coming from ground-truth data (See at least Rozantsev Paragraph 0040), thus, improving efficiency.
Claims 8-11 are rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Wang, AlRegib, Mantegazza et al “Sensing Anomalies as Potential Hazards: Datasets and Benchmarks”, and Hanspal et al (US 20240338561 A1) (Hereinafter referred to as Mantegazza and Hanspal respectively)
Regarding Claim 8, Park teaches a system for detecting off-nominal behavior (See at least Park Page 1 Abstract, the anomalous execution is interpreted as off-nominal behavior) comprising:
a robotic control platform configured to observe and control operations of a robotic system (See at least Park Page 1 Introduction, and Pages 5-6 V Experimental Setup, the long short-term memory-based variational autoencoder anomaly detector and user interface are interpreted as a robotic control platform, which observes the robot using the multimodal inputs and controls operations via the interface), said robotic control platform comprising:
a user interface configured to allow a user to remotely direct operations of said robotic system (See at least Park Page 5 C. Experimental Procedure, the user directs operations of the robot using the user interface);
a trained manifold developed with a target behavior dataset (See at least Park Page 4 C. Training and Testing Framework and Page 3 Figure 2, the trained LSTM-VAE-based anomaly detector is interpreted as the trained manifold, which is developed with a sequence of non-anomalous/target behavior dataset) , wherein said target behavior data set corresponds to one or more tasks to be completed by a robotic system over a time interval (See at least Park Pages 1-2 Introduction, Page 4 C. Training and Testing Framework, and Page 3 Figure 2, the non-anomalous/target behavior dataset corresponds to a task to be completed by a robot in sequences over a time interval);
an encoder configured to receive at least one initial data set (See at least Park Page 3 C. An LSTM-based Variational Autoencoder and Figure 2, and Pages 6-7 Evaluation and Figure 6, the encoder receives an initial multimodal input at different times), wherein said at least one initial data set corresponds to a performance of a first task of said one or more tasks over said time interval (See at least Park Page 1 Introduction, and Pages 5-7, V Experimental Setup and VI Evaluation, the initial data set/multimodal observation corresponds to a robot performing a task over a time interval), …said encoder being further configured to generate a lower dimensional data set from said at least one initial data set (See at least Park Page 1 Introduction, during encoding, the multimodal observations/initial data set is projected into a latent space, which is a lower dimensional data set);
a generator configured to generate a reconstructed data set by mapping said lower dimensional data set onto said trained manifold (See at least Park Page 1 Introduction, Page 2 A. Preliminary Autoencoder, and Page 3 Figure 2, the decoder/generator generates a reconstructed data set using the lower dimensional/compressed data set)…; and
…measure a divergence between said initial data set and said reconstructed data set along said time interval as said at least one initial data set is received, wherein a divergence beyond a pre-determined threshold is indicative of off-nominal behavior (See at least Park Page 4 IV Anomaly Detection and Pages 6-7 VI Evaluation and Figure 6, the deviation between the initial/observed data set and the reconstructed/predicted data set beyond a threshold value indicates anomalous execution/off-nominal behavior).
Park fails to disclose said generator being adversarially trained with a discriminator.
However, Wang teaches said generator being adversarially trained with a discriminator (See at least Wang Paragraphs 0010 and 0022-0026, the generator is trained with a discriminator during the adversarial training process).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in Park with Wang to adversarially train the generator with a discriminator. This modification, as taught by Wang, would allow the system to reach Nash equilibrium, so that the sample data generated by the generator is no different from the real sample data, and the discriminator cannot distinguish the generated sample data from the real sample data (See at least Wang Paragraph 0010).
Modified Park fails to explicitly disclose a classifier configured to receive feature representations, said feature representations comprising at least a difference between a first feature of said at least one initial data set and a second feature of said reconstructed data set, said classifier being further configured to measure a divergence between said initial data set and said reconstructed data set.
However, AlRegib teaches a classifier configured to receive feature representations, said feature representations comprising at least a difference between a first feature of said at least one initial data set and a second feature of said reconstructed data set (See at least AlRegib Paragraphs 0005, 0007, 0009, and 0028, the discriminator, which is a type of classifier, measures similarity and feature matching between the initial data and the reconstructed data to determine a reconstruction loss), said classifier being further configured to measure a divergence between said initial data set and said reconstructed data set (See at least AlRegib Paragraphs 0005, 0007, 0009, 0030, and 0032, the discriminator/classifier measures the reconstruction error/divergence between the reconstructed data and the initial data).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with AlRegib to utilize a classifier to measure a divergence between said initial data set and said reconstructed data set. Classifiers are utilized as an anomaly detector by comparing the reconstructed data with original data (See at least AlRegib Paragraph 0009). By using a classifier to measure the divergence, as taught by AlRegib, the system can detect an abnormality by measuring the divergence/reconstruction error between the initial data and the reconstructed data (See at least AlRegib Paragraph 0032), which would improve the autonomy of the system.
Modified Park fails to disclose said user interface being further configured to notify said user upon said classifier detecting said off-nominal behavior.
However, Mantegazza teaches said user interface being further configured to notify said user upon detecting said off-nominal behavior (See at least Mantegazza section 1, the anomaly/off-nominal behavior is reported to the user/operator).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with Mantegazza to notify a user upon the classifier detecting off-nominal behavior. By notifying the user of the off-nominal behavior, the user can analyze the reported off-nominal behavior and generate instructions accordingly (See at least Mantegazza section 1), which would also improve the safety and efficiency of the system.
Modified Park fails to disclose said user interface is further configured to allow said user to cause said at least one initial data set to be communicated to said encoder.
However, Hanspal teaches said user interface is further configured to allow said user to cause said at least one initial data set to be communicated to said encoder (See at least Hanspal Paragraphs 0064-0065, and 0084, the initial/input data set is communicated to the encoder according to the user input).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with Hanspal to allow said user to cause said at least one initial data set to be communicated to said encoder. This modification, as taught by Hanspal, would allow the user to train the encoder according to initial data set inputted by the user (See at least Hanspal Paragraphs 0064-0065, and 0084), thus, giving the user greater control over the system.
Regarding Claim 9, modified Park teaches said target behavior dataset includes only nominal behavior data (See at least Park Page 4 C. Training and Testing Framework, the target behavior dataset includes non-anomalous training data, which is interpreted as nominal behavior data).
Regarding Claim 10, modified Park teaches said at least one initial data set includes a plurality of measured state of said robotic system over said time interval (See at least Park Pages 6-7, VI Evaluation and Figure 6, the current observations are a plurality of measured states of the robot over said time interval) and said nominal behavior data corresponds to a set of target states for said robotic system over said time interval (See at least Park Page 4 C. Training and Testing Framework, the nominal behavior data/non-anomalous training corresponds to target states over said time interval).
Regarding Claim 11, modified Park fails to disclose said classifier is a fully-connected classifier.
However, Wang teaches said classifier is a fully-connected classifier (See at least Wang Paragraph 0013, the discriminator/classifier is a fully-connected neural network).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with Wang to have the classifier be a fully-connected classifier. Fully connected classifiers are conventional and well-understood in the art, and therefore, one of ordinary skill in the art would be motivated to utilize a fully connected classifier as the classifier.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Wang, AlRegib, Mantegazza, and Hanspal, and in further view of Rozantsev
Regarding Claim 12, modified Park fails to disclose said classifier is a long short-term memory classifier.
However, Rozantsev teaches said classifier is a long short-term memory classifier (See at least Rozantsev Paragraph 0040, the discriminator/classifier utilizes a LSTM unit).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with Rozantsev to have the classifier be a long short-term memory classifier. This modification, as taught by Rozantsev, would allow the system to efficiently gather information from the predictions of a prediction network for multiple consecutive time steps, and predict whether the sequences of input data is generated or coming from ground-truth data (See at least Rozantsev Paragraph 0040), thus, improving efficiency.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Wang, AlRegib, Mantegazza, and Hanspal, and in further view of Kranski
Regarding Claim 13, modified Park fails to disclose said nominal behavior data comprises simulated behavior data.
However, Kranski teaches said nominal behavior data comprises simulated behavior data (See at least Kranski Paragraphs 0022 and 0061, the nominal behavior data to complete a task is simulated).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the teachings disclosed in modified Park with Kranski to have the nominal behavior data comprise simulated behavior data. This modification, as taught by Kranski, would allow the system to train the robot to perform the task through simulated attempts rather than actually controlling a real robot (See at least Kranski Paragraphs 0022 and 0061), which would improve the efficiency and cost of the training.
Response to Arguments
Applicant's arguments filed 09/08/2026 have been fully considered but they are not persuasive. Applicant argues, on pages 5-6 of the remarks, that the prior art does not disclose every limitation of amended claims 1 and 8.
However, Examiner disagrees. Wang teaches a generator being adversarially trained with a discriminator. AlRegib teaches a classifier/discriminator that determines a difference between an initial data set and a reconstructed data set by measuring similarity and feature matching between the two data sets.
Thus, for these reasons, the claims still stand rejected under 103.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ESVINDER SINGH whose telephone number is (571)272-7875. The examiner can normally be reached Monday-Friday: 9 am-5 pm est.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached at 571-270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ESVINDER SINGH/Primary Examiner, Art Unit 3657
