DETAILED ACTION
Claims 1, 3-15, 17-20 are presented for examination.
This office action is in response to submission of application on 09-JUNE-2026.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07-DECEMBER-2021 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 30-DECEMBER-2025 has been entered.
Response to Amendment
The amendment filed 09-JUNE-2026 in response to the non-final office action mailed 09-MARCH-2026 has been entered. Claims 1, 3-15, 17-20 remain pending in the application.
With regards to the non-final office action’s rejection under 101, the amendments to the claims overcome the original rejection with regards to the claims being directed towards an abstract idea.
With regards to the non-final office action’s rejection under 103, the amendment to the claims have overcome the original rejection. However, upon a new search for the amended limitations, a new 103 rejection over Metzler in view of Asbag, further in view of previously present Gowal has been written.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 6-8, 10-11, 14-15, 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Metzler et al. (Pub. No. WO 2020088739 A1, filed on October 29th 2018, hereinafter Metzler) in view of Asbag et al. (Pub. No. US 11037286 B2, filed on September 28th 2017, hereinafter Asbag), further in view of Gowal et al. (“On the effectiveness of interval bound propagation for training verifiably robust models”, published August 29th 2019, hereinafter Gowal).
Regarding claim 1:
Claim 1 recites:
A method for training a machine-learning network, the method comprising: receiving input data from a sensor, wherein the input data includes a perturbation, wherein the input data is indicative of image, radar, sonar, or sound information; obtaining a worst-case bound on a classification error and loss for perturbed versions of the input data, utilizing at least bounding of one or more hidden layer values by an adversarial norm constraint; generating augmented training data by augmenting a training data set using at least a term promoting classification of adversarial inputs into respective additional abstain classes of at least two additional abstain classes corresponding to different perturbation conditions; training a classifier using the augmented training data, wherein the classifier includes a plurality of classes, including one or more correct classes and at least two additional abstain classes, wherein each additional abstain class of the at least two additional abstain classes is determined in response to at least bounding the input data; determining a first lower bound for each of the one or more correct classes of the plurality of classes; determining, using interval bound propagation over perturbed hidden layer values, a second lower bound for each of the at least two additional abstain classes; using at least one of the worst-bound, the first lower bound, and the second lower bound, outputting a classification in response to the input data indicating one of the plurality of classes, wherein the assignment of the input data to any respective class of the one or more correct classes and the at least two additional abstain classes is a valid assignment; in response to exceeding a convergence threshold: outputting a trained classifier, wherein the trained classifier is configured to detect at least one additional abstain class of the at least two additional abstain classes in response to obtaining the worst-case bound; and classifying, using one or more layers of the trained classifier, the input data as an abstain class in response to the input data including at least one of the perturbation and adversarial information; and in response to not exceeding the convergence threshold, continuing to train the classifier.
Metzler discloses receiving an input data from a sensor, wherein the input data includes a perturbation:
“…The system further comprises communication means for transmitting data from the surveillance sensors to the central computing unit and state derivation means for analyses of the surveillance data and derivation of at least one state…” (Metzler)
“…Ambiguous deductions can further be the result of unfavorable conditions for survey data acquisition such as poor light conditions, inauspicious robot position P10 when generating surveillance data or perturbing environmental influence…” (Metzler)
Metzler teaches transmitting data from surveillance centers to the central computing unit, where the input data may include perturbing environmental influence. This would be analogous to the central computing unit receiving input data from a sensor, wherein the data includes a perturbation.
Metzler discloses wherein the input data is indicative of image, radar, sonar, or sound information
“The plurality of surveillance sensors… comprise for example one or more RGB camera… microphone…” (Metzler)
Metzler teaches that the sensors may be, for example, an RGB camera or a microphone, which would be an example of image and sound information.
Metzler discloses hidden layers
“The Neural Network 20 comprises an input layer 17, a hidden layer 18 and an output layer 19” (Metzler)
Metzler teaches the use of hidden layers.
Bounding is taught by Asbag further below.
Metzler discloses generating augmented training data by augmenting a training data set using at least a term promoting classification of [adversarial] inputs:
Metzler recites: “In case of supervised machine learning also labeling information (i.e. assignment of the object classes to the data) is necessary.”
This teaches the use of supervised machine learning, which is a form of annotation of training data to possess labels. Here, Metzler uses labeling to assign different object classes to the data. This would be training data augmented by a term promoting classification of input into each of the additional classes of the plurality of classes. Furthermore, with Asbag’s teachings of abstain classes below, it would have been obvious to combine this augmentation and the use of abstain classes as is disclosed in the claim limitation.
Metzler discloses one or more correct classes:
Metzler, description: “Probabilistic classification algorithms further use statistical inference to find the best class for a given instance… consequently, providing an option to abstain a choice when its confidence value is too low.”
Furthermore, Metzler teaches classification algorithms that sort inputs into multiple correct classes, which would provide one or more correct classes.
Metzler discloses in response to exceeding a convergence threshold: outputting a trained classifier:
“in case the probability is above a defined threshold such that considering the additional data, subsequent classification results in a probability below the defined threshold. Said otherwise, if there is a too high uncertainty or unreliability of a classification, the system retrieves automatically additional data about a state pattern resp. one or more facility elements, such that additional information (e.g. parameters or features) describing the state is available, allowing for a higher certainty of assignment as "critical" or "non- critical" or possibly as "normal" or "anomalous".” (Metzler)
“At step 116, it is checked if the determined uncertainty 115 is above a defined threshold. If the result is "no", i.e. there is low uncertainty and the detected state 114 can be seen as correct or unambiguous, than the robot 100 continues its patrol resp. goes on to the next object to be surveyed” (Metzler)
Metzler teaches a process iteratively continuing if a value fails to meet a particular threshold, which discloses a further process continuing upon a classifier exceeding a convergence threshold. Furthermore, Metzler explicitly describes the successful meeting of a convergence threshold to trigger additional actions as well, describing a robot that when there is low uncertainty (i.e., the uncertainty has met the threshold) performs further actions.
Metzler discloses classifying, using one or more layers of the trained classifier, the input data as an abstain class in response to the input data including at least one of the perturbation and adversarial information:
Metzler teaches: “…Ambiguous deductions can further be the result of unfavourable conditions for survey data acquisition such as poor light conditions, inauspicious robot position P10 when generating surveillance data or perturbing environmental influence…” (Metzler)
The ambiguous deduction would be analogous to the abstain class as it is not classified to be in a particular classification outside of being unknown, and the perturbing environment influence would be at least one of the perturbation and adversarial information from the input data.
Metzler discloses and in response to not exceeding the convergence threshold, continuing to train the classifier:
“In an optional further stage of this aspect of the invention, such a generic classifier and/or detector can additionally be post-trained by real world pictures. […] For example, the real world pictures on which the detector and/or classifier is applied can be used as additional training resource to enhance the detector and/or classifier, e.g. to improve its real world success rate” (Metzler)
Metzler teaches the process of, in order to improve a particular value, continuing to train a classifier. This would be analogous to in response to not exceeding the convergence threshold, continuing to train the classifier.
However, Metzler does not disclose obtaining a worst-case bound on a classification error and loss for perturbed versions of the input data, utilizing at least bounding of one or more hidden layer values by an adversarial norm constraint. Instead, Asbag discloses:
Asbag in the same field of endeavor of reinforcement learning teaches setting a confidence value that is balanced between enhancing the quality of classification and loss of defects of interest (Column 2 line 35 – Column 3 line 15). The confidence value would therefore be the worst-case bound as it describes a performance metrics that must be balanced between classification error and loss for perturbed versions of the data.
Furthermore, Asbag teaches an adversarial norm constraint as it teaches the use of a threshold for particular abstain classes (Column 2 line 35 – Column 3 line 15) wherein this would act as an adversarial norm constraint as it bounds the inputs that will be classified into correct classes by the model rather than abstain classes.
Metzler and Asbag are analogous art to the present application because they are all in the same field of endeavor of reinforcement learning.
Asbag discloses classification into respective additional abstain classes of at least two additional abstain classes corresponding to different perturbation conditions:
Asbag teaches the amendment aspect of the limits at least two additional abstain classes corresponding to different perturbation conditions. For example, Asbag teaches two different rejection bins (as above, considered to be abstain classes) for two different rejected defect scenarios (different perturbation conditions), wherein a rejected defect that may be more than one class is labeled as “cannot decide”, while a rejected defect that is not part of any class is labeled as “unknown” (Column 7, lines 20-30).
Asbag discloses training a classifier using augmented training data, wherein the classifier includes a plurality of classes, [including one or more correct classes] and at least two additional abstain classes:
Asbag teaches training a classifier that has the ability to classify data into rejection bins in response to a low confidence score. (Column 2 line 35 – Column 3 line 15). These bins are classes that have been bound together (Column 3, lines 60-65), and act as abstain classes. These bins are also sorted by priority, with Key Defects of Interest being the highest priority abstain class. (Column 2 line 35 – Column 3 line 15). This would be analogous to detecting an additional abstain class in response to obtaining the worst-case bound, and further examples of abstain classes are given to provide at least two additional classes.
Asbag discloses wherein each additional abstain class of the at least two additional abstain classes is determined in response to at least bounding the input data:
Asbag teaches the bounding of input data by the confidence thresholds that they correspond to upon processing, which classifies them into rejection bins of multiple abstain classes (Column 2 line 35 – Column 3 line 15). This would be analogous to each additional abstain class of the plurality of additional abstain classes is determined in response to at least bounding the input data.
Asbag discloses wherein assignment of the input data to any respective class of the one or more correct classes and the at least two additional abstain classes is a valid assignment:
Asbag teaches valid class thresholds that determine valid assignments to classes, which would apply to the previously taught correct and abstain classes (Column 3, lines 45-51).
Asbag discloses wherein the trained classifier is configured to detect at least one additional abstain class of the at least two additional abstain classes in response to obtaining the worst-case bound:
Asbag teaches training a classifier that has the ability to classify data into rejection bins in response to a low confidence score. (Column 2 line 35 – Column 3 line 15). These bins are classes that have been bound together (Column 3, lines 60-65), and act as abstain classes. These bins are also sorted by priority, with Key Defects of Interest being the highest priority abstain class. (Column 2 line 35 – Column 3 line 15). This would be analogous to detecting an additional abstain class in response to obtaining the worst-case bound.
Neither Metzler nor Asbag fully disclose determining a first lower bound for each of the one or more correct classes of the plurality of classes; determining, using interval bound propagation over perturbed hidden layer values, a second lower bound for each of the at least two additional abstain classes; using at least one of the worst-case bound, the first lower bound, and the second lower bound outputting a classification in response to the input data indicating one of the plurality of classes. Instead, this limitation is disclosed by Gowal:
Gowal recites: “IBP’s goal is to find an upper bound on the optimal value of the problem (4). The simplest approach is to bound the activation zk of each layer by an axis-aligned bounding box” (Page 3) as well as “In the context of classification under adversarial perturbation, solving the optimization problem (8) for each target class y= ytrue” (Page 4)
Gowal teaches using interval bound propagation to determine a lower bound for multiple classes, as the optimization problem described requires the lower bound for the class. Therefore, both a first and second lower bound for various classes would be determined.
Gowal further recites: “Our experiments have shown that the proposed approach outperforms competing techniques in terms of verified bounds on adversarial error rates in image classification problems” (Page 7).
Therefore, Gowal discloses that using the previously determined bounds, a classification is output indicating one of the plurality of classes as it would be part of an image classification problem. Gowal could then combine with Metzler and Asbag in order to apply its determination of the lower bound to a classifier with correct classes and abstain classes.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 3, which is dependent upon claim 1:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 1 upon which claim 3 depends.
However, Metzler does not disclose wherein the plurality of classes includes original classes corresponding to the input data. However, Asbag teaches:
General description, paragraph 11, excerpt: “By way of non-limiting example, each class can be assigned to one of the following classification groups: “Key Defects of Interest (KDOI)” being the classification group with the highest priority, “Defects of Interest (DOI)”, and “False” being the classification group with the lowest priority.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag, further in view of Gowal that disclosed:
method of claim 1
And the teachings of Asbag that disclosed:
wherein the plurality of classes includes original classes corresponding to the input data
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 6, which is dependent upon claim 1:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 1 upon which claim 6 depends. Furthermore, Metzler teaches:
Description, excerpt: “…Ambiguous deductions can further be the result of unfavourable conditions for survey data acquisition such as poor light conditions, inauspicious robot position P10 when generating surveillance data or perturbing environmental influence…”
This discloses wherein the classifier does not classify the input data as the [original] classes when the input data includes perturbations as data with the perturbing influence need not be classified into an original class.
Metzler does not disclose wherein the plurality of classes includes original classes corresponding to the input data. However, Asbag teaches:
General description, paragraph 11, excerpt: “By way of non-limiting example, each class can be assigned to one of the following classification groups: “Key Defects of Interest (KDOI)” being the classification group with the highest priority, “Defects of Interest (DOI)”, and “False” being the classification group with the lowest priority.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag, further in view of Gowal that disclosed:
method of claim 1
wherein the classifier does not classify the input data as the [original] classes when the input data includes perturbations
And the teachings of Asbag that disclosed:
wherein the plurality of classes includes original classes corresponding to the input data
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 7, which is dependent upon claim 1:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 1 that claim 7 is dependent upon. Metzler in view of Asbag does not disclose bounding a training objective function by a worst-case upper bound utilizing an interval bound propagation (IBP) technique.
However, Gowal in the same field of endeavor of adversarial learning teaches:
Introduction, excerpt: “…IBP allows to define a loss to minimize an upper bound on the maximum difference between any pair of logits when the input can be perturbed…”
Metzler in view of Asbag and Gowal are analogous art because they are in the same field of endeavor.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag, further in view of Gowal that disclosed:
the method of claim 1
And the teachings of Gowal that disclosed:
bounding a training objective function by a worst-case upper bound utilizing an interval bound propagation (IBP) technique.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 8:
Claim 8 recites:
A system including a machine-learning network, comprising: an input interface configured to receive input data from a sensor, wherein the sensor includes a video, radar, LiDAR, sound, sonar, ultrasonic, motion, or thermal imaging sensor; a processor, in communication with the input interface, wherein the processor is configured to: receive input data from a sensor via the input interface, wherein the input data is indicative of image, radar, sonar, or sound information; generate augmented training data by augmenting a training data set using at least a term promoting classification of adversarial inputs into respective additional abstain classes of at least two additional abstain classes; training a classifier using the augmented training data, wherein the classifier includes a plurality of classes, including one or more correct classes and at least two additional abstain classes, wherein each additional abstain class of the at least two additional abstain classes is determined in response to at least bounding the input data including one or more perturbations; determine a first lower bound for each of the one or more correct classes of the plurality of classes; determine, using interval bound propagation, a second lower bound for each of the at least two additional abstain classes; using at least one of the first lower bound, and the second lower bound output a classification in response to the input data indicating one of the plurality of classes wherein assignment of the input data to any respective class of the one or more correct classes and the at least two additional abstain classes is a valid assignment; in response to the classifier exceeding a convergence threshold: output a trained classifier configured to detect at least one additional abstain class of the at least two additional abstain classes; and classify, using one or more layers of the trained classifier, the input data as an abstain class in response to the input data including at least one of the perturbation and adversarial information; and in response to the classifier not exceeding the convergence threshold, continue to train the classifier.
Metzler discloses an input interface configured to receive input data from a sensor, wherein the sensor includes a video, radar, LiDAR, sound, sonar, ultrasonic, motion, or thermal imaging sensor; a processor, in communication with the input interface, wherein the processor is configured to: receive an input data from a sensor, wherein the input data is indicative of image, radar, sonar, or sound information:
“The plurality of surveillance sensors… comprise for example one or more RGB camera… microphone…” (Metzler)
Metzler teaches that the sensors may be, for example, an RGB camera or a microphone, which would be an example of image and sound information.
Metzler discloses generate augmented training data by augmenting a training data set using at least a term promoting classification of [adversarial] inputs:
Metzler recites: “In case of supervised machine learning also labeling information (i.e. assignment of the object classes to the data) is necessary.”
This teaches the used of supervised machine learning, which is a form of annotation of training data to possess labels. Here, Metzler uses labeling to assign different object classes to the data. This would be training data augmented by a term promoting classification of input into each of the additional classes of the plurality of classes. Furthermore, with Asbag’s teachings of abstain classes below, it would have been obvious to combine this augmentation and the use of abstain classes as is disclosed in the claim limitation.
Metzler discloses one or more correct classes:
Metzler, description: “Probabilistic classification algorithms further use statistical inference to find the best class for a given instance… consequently, providing an option to abstain a choice when its confidence value is too low.”
Metzler teaches classification algorithms that sort inputs into multiple correct classes, which would provide one or more correct classes.
Metzler discloses in response to the classifier exceeding a convergence threshold: output a trained classifier:
“in case the probability is above a defined threshold such that considering the additional data, subsequent classification results in a probability below the defined threshold. Said otherwise, if there is a too high uncertainty or unreliability of a classification, the system retrieves automatically additional data about a state pattern resp. one or more facility elements, such that additional information (e.g. parameters or features) describing the state is available, allowing for a higher certainty of assignment as "critical" or "non- critical" or possibly as "normal" or "anomalous".” (Metzler)
“At step 116, it is checked if the determined uncertainty 115 is above a defined threshold. If the result is "no", i.e. there is low uncertainty and the detected state 114 can be seen as correct or unambiguous, than the robot 100 continues its patrol resp. goes on to the next object to be surveyed” (Metzler)
Metzler teaches a process iteratively continuing if a value fails to meet a particular threshold, which discloses a further process continuing upon a classifier exceeding a convergence threshold. Furthermore, Metzler explicitly describes the successful meeting of a convergence threshold to trigger additional actions as well, describing a robot that when there is low uncertainty (i.e., the uncertainty has met the threshold) performs further actions.
Metzler discloses classify, using one or more layers of the trained classifier, the input data as an abstain class in response to the input data including at least one of the perturbation and adversarial information:
Metzler teaches: “…Ambiguous deductions can further be the result of unfavourable conditions for survey data acquisition such as poor light conditions, inauspicious robot position P10 when generating surveillance data or perturbing environmental influence…” (Metzler)
The ambiguous deduction would be analogous to the abstain class as it is not classified to be in a particular classification outside of being unknown, and the perturbing environment influence would be at least one of the perturbation and adversarial information from the input data.
Metzler discloses and in response to the classifier not exceeding the convergence threshold, continuing to train the classifier:
“In an optional further stage of this aspect of the invention, such a generic classifier and/or detector can additionally be post-trained by real world pictures. […] For example, the real world pictures on which the detector and/or classifier is applied can be used as additional training resource to enhance the detector and/or classifier, e.g. to improve its real world success rate” (Metzler)
Metzler teaches the process of, in order to improve a particular value, continuing to train a classifier. This would be analogous to in response to not exceeding the convergence threshold, continuing to train the classifier.
However, Metzler does not teach classification into respective additional abstain classes of at least two additional abstain classes. Instead, this limitation is taught by Asbag:
Asbag teaches
Asbag teaches the amendment aspect of the limits at least two additional abstain classes For example, Asbag teaches two different rejection bins (as above, considered to be abstain classes) for two different rejected defect scenarios wherein a rejected defect that may be more than one class is labeled as “cannot decide”, while a rejected defect that is not part of any class is labeled as “unknown” (Column 7, lines 20-30).
However, Metzler does not teach train a classifier using the augmented training data, wherein the classifier includes a plurality of classes, including one or more correct classes and the at least two additional abstain classes corresponding to different perturbation conditions. Instead, this limitation is taught by Asbag:
Asbag teaches training a classifier that has the ability to classify data into rejection bins in response to a low confidence score. (Column 2 line 35 – Column 3 line 15). These bins are classes that have been bound together (Column 3, lines 60-65), and act as abstain classes. These bins are also sorted by priority, with Key Defects of Interest being the highest priority abstain class. (Column 2 line 35 – Column 3 line 15). This would be analogous to detecting an additional abstain class in response to obtaining the worst-case bound.
Asbag teaches the amendment aspect of the limits at least two additional abstain classes corresponding to different perturbation conditions. For example, Asbag teaches two different rejection bins (as above, considered to be abstain classes) for two different rejected defect scenarios (different perturbation conditions), wherein a rejected defect that may be more than one class is labeled as “cannot decide”, while a rejected defect that is not part of any class is labeled as “unknown” (Column 7, lines 20-30).
Furthermore, Metzler has previously taught classification algorithms that sort inputs into multiple correct classes, which would provide one or more correct classes when combined with Asbag.
Metzler does not disclose training a classifier using augmented training data, wherein the classifier includes a plurality of classes, including one or more correct classes and at least two additional abstain classes. Instead, this limitation is disclosed by Asbag:
Asbag teaches training a classifier that has the ability to classify data into rejection bins in response to a low confidence score. (Column 2 line 35 – Column 3 line 15). These bins are classes that have been bound together (Column 3, lines 60-65), and act as abstain classes. These bins are also sorted by priority, with Key Defects of Interest being the highest priority abstain class. (Column 2 line 35 – Column 3 line 15). This would be analogous to detecting an additional abstain class in response to obtaining the worst-case bound, and further examples of abstain classes are given to provide at least two additional classes.
Metzler, description: “Probabilistic classification algorithms further use statistical inference to find the best class for a given instance… consequently, providing an option to abstain a choice when its confidence value is too low.”
Furthermore, Metzler teaches classification algorithms that sort inputs into multiple correct classes, which would provide one or more correct classes when combined with Asbag.
Metzler does not disclose wherein each additional abstain class of the at least two additional abstain classes is determined in response to at least bounding the input data. Instead, this limitation is disclosed by Asbag:
Asbag teaches the bounding of input data by the confidence thresholds that they correspond to upon processing, which classifies them into rejection bins of multiple abstain classes (Column 2 line 35 – Column 3 line 15). This would be analogous to each additional abstain class of the plurality of additional abstain classes is determined in response to at least bounding the input data.
Metzler does not disclose wherein assignment of the input data to any respective class of the one or more correct classes and the at least two additional abstain classes is a valid assignment. Instead, this limitation is disclosed by Asbag:
Asbag teaches valid class thresholds that determine valid assignments to classes, which would apply to the previously taught correct and abstain classes (Column 3, lines 45-51).
Metzler does not disclose output a trained classifier configured to detect at least one additional abstain class of the at least two abstain classes. Instead, this limitation is disclosed by Asbag:
Asbag teaches training a classifier that has the ability to classify data into rejection bins in response to a low confidence score. (Column 2 line 35 – Column 3 line 15). These bins are classes that have been bound together (Column 3, lines 60-65), and act as abstain classes. These bins are also sorted by priority, with Key Defects of Interest being the highest priority abstain class. (Column 2 line 35 – Column 3 line 15). This would be analogous to detecting an additional abstain class in response to obtaining the worst-case bound.
Neither Metzler nor Asbag fully disclose determine a first lower bound for each of the one or more correct classes of the plurality of classes; determine, using interval bound propagation, a second lower bound for each of the at least two additional abstain classes; using at least one of the first lower bound, and the second lower bound, output a classification in response to the input data indicating one of the plurality of classes. Instead, this limitation is disclosed by Gowal:
Gowal recites: “IBP’s goal is to find an upper bound on the optimal value of the problem (4). The simplest approach is to bound the activation zk of each layer by an axis-aligned bounding box” (Page 3) as well as “In the context of classification under adversarial perturbation, solving the optimization problem (8) for each target class y= ytrue” (Page 4)
Gowal teaches using interval bound propagation to determine a lower bound for multiple classes, as the optimization problem described requires the lower bound for the class. Therefore, both a first and second lower bound for various classes would be determined.
Gowal further recites: “Our experiments have shown that the proposed approach outperforms competing techniques in terms of verified bounds on adversarial error rates in image classification problems” (Page 7).
Therefore, Gowal discloses that using the previously determined bounds, a classification is output indicating one of the plurality of classes as it would be part of an image classification problem. Gowal could then combine with Metzler and Asbag in order to apply its determination of the lower bound to a classifier with correct classes and abstain classes.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 10, which is dependent upon claim 8:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 8 that claim 10 is dependent upon. Metzler in view of Asbag does not disclose wherein the processor is further configured to utilize interval bound propagation […] with perturbed versions of the input data.
However, Gowal teaches:
Introduction, excerpt: “…IBP allows to define a loss to minimize an upper bound on the maximum difference between any pair of logits when the input can be perturbed…”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag that disclosed:
the method of claim 8
And the teachings of Gowal that disclosed:
wherein the processor is further configured to utilize interval bound propagation […] with perturbed versions of the input data.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 11, which is dependent upon claim 10:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 10 that claim 11 is dependent upon. Metzler in view of Asbag does not disclose wherein the processor is further configured to compute an upper bound associated with training of the machine-learning network.
However, Gowal teaches:
Introduction, excerpt: “…IBP allows to define a loss to minimize an upper bound on the maximum difference between any pair of logits when the input can be perturbed…”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag further in view of Gowal that disclosed:
the method of claim 10
And the teachings of Gowal that disclosed:
wherein the processor is further configured to compute an upper bound associated with training of the machine-learning network.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Claim 14 recites a system that parallels the method and system of claims 1 and 8. It contains no limitations that are not found within claims 1 and 8, and as evidenced by the identical amendments is intended as a counterpart to claim 1 and 8. Therefore, the analysis discussed above with respect to claims 1 and 8 also applies to claim 14. Accordingly, claims 14 is rejected based on substantially the same rationale as set forth above with respect to claims 1 and 8.
Regarding claim 15 which is dependent upon claim 14:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 14 upon which claim 15 depends. Furthermore, Metzler teaches:
Description, excerpt: “…The state detector resp. an underlying computing unit is further configured to trigger an action of the robot by the action controller in case an ambiguity is noticed…”
This discloses wherein instructions further cause the processor to operate a physical system based on output data, wherein the physical system is […] a robot […].
Regarding claim 17, which is dependent upon claim 14:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 14 upon which claim 17 depends.
However, Metzler does not disclose wherein the plurality of classes includes original classes corresponding non-perturbation classification associated with the input data. However, Asbag teaches:
General description, paragraph 11, excerpt: “By way of non-limiting example, each class can be assigned to one of the following classification groups: “Key Defects of Interest (KDOI)” being the classification group with the highest priority, “Defects of Interest (DOI)”, and “False” being the classification group with the lowest priority.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag that disclosed:
method of claim 14
And the teachings of Asbag that disclosed:
wherein the plurality of classes includes original classes corresponding non-perturbation classification associated with the input data
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 18, which is dependent upon claim 14:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 14 that claim 18 is dependent upon. Metzler in view of Asbag does not disclose wherein the instructions further cause the processor to compute an upper bound associated with training of the machine-learning network.
However, Gowal teaches:
Introduction, excerpt: “…IBP allows to define a loss to minimize an upper bound on the maximum difference between any pair of logits when the input can be perturbed…”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag, further in view of Gowal that disclosed:
the method of claim 14
And the teachings of Gowal that disclosed:
wherein the instructions further cause the processor to compute an upper bound associated with training of the machine-learning network.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 19 which is dependent upon claim 14:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 14 upon which claim 19 depends.
However, Metzler does not disclose wherein the plurality of classes except the plurality of additional abstain classes are utilized to classify a non-perturbation class.
Asbag teaches:
General description, paragraph 11 excerpt: “By way of non-limiting example, each class can be assigned to one of the following classification groups: “Key Defects of Interest (KDOI)” being the classification group with the highest priority, “Defects of Interest (DOI)”, and “False” being the classification group with the lowest priority. The prioritized rejection bins can consist, accordingly, of “KDOI” CND rejection bin; “DOI” CND rejection bin, “False” CND rejection bin and “unknown (UNK)” rejection bin, and wherein priorities of CND rejection bins correspond to priorities of respective classification groups”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag, further in view of Gowal that disclosed:
method of claim 14
And the teachings of Asbag that disclosed:
wherein the plurality of classes except the plurality of additional abstain classes are utilized to classify a non-perturbation class
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler, Asbag, and Gowal. This would have provided the advantage of improving automated classification (Asbag, Column 4, lines 15-20), as well as the advantage of tighter bounds in later stages of training (Gowal, “Perhaps surprisingly, our results show that neural networks can easily adapt to make the rather loose bound provided by IBP much tighter.”)
Regarding claim 20, which is dependent upon claim 14:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 14 upon which claim 20 depends. Furthermore, Metzler teaches:
Description, excerpt: “…the criticality classification and optionally the normality classification is implemented with at least one of a rule-based system, based on expert knowledge, in particular comprising… a neural network …”
This discloses wherein the machine-learning network is a neural network.
Claims 4 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Metzler in view of Asbag, , further in view of Gowal, further in view of Guan et al. (Pub. No. CN 108446506 A, published August 24th 2018, hereinafter Guan).
Regarding claim 4, which is dependent upon claim 1:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 1 that claim 4 is dependent upon. Metzler in view of Asbag, further in view of Gowal does not disclose determining a hidden value upper bound and hidden value lower bound associated with a hidden value of a network layer of the machine- learning network.
However, Guan in the same field of endeavor of reinforcement learning teaches:
Step 4.3 description, excerpt: “…respectively represent the upper limit and lower limit of k moment j-th supporting layer output. and ? [sic] j respectively represent the upper limit of the j-th hidden node threshold value and the lower limit...”
Metzler in view of Asbag, further in view of Gowal and Guan are analogous art because they are in the same field of endeavor.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag, further in view of Gowal that disclosed:
the method of claim 1
And the teachings of Guan that disclosed:
determining a hidden value upper bound and hidden value lower bound associated with a hidden value of a network layer of the machine- learning network.
This would have provided to Metzler in view of Asbag, further in view of Gowal the advantage of more efficient system modeling (Guan: “interval feedback neural network due to its own structure with memory, the adaptive time-varying characteristics, can solve the feedforward neural network to problem of order dynamic system modeling, so it can be used as effective means of uncertain system modelling”).
Regarding claim 5, which is dependent upon claim 1:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 1 that claim 5 is dependent upon. Metzler in view of Asbag, further in view of Gowal does not disclose wherein the one or more hidden layer values is associated with a last layer of the machine-learning network.
However, Guan teaches:
Step 4.7.1. description, excerpt: “…obtaining the hidden node weight value upper limit and lower limit of the correction value to the output layer node…”
This output layer would be the associated last layer.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag, further in view of Gowal that disclosed:
the method of claim 1
And the teachings of Guan that disclosed:
wherein the one or more hidden layer values is associated with a last layer of the machine-learning network
This would have provided to Metzler in view of Asbag, further in view of Gowal the advantage of more efficient system modeling (Guan: “interval feedback neural network due to its own structure with memory, the adaptive time-varying characteristics, can solve the feedforward neural network to problem of order dynamic system modeling, so it can be used as effective means of uncertain system modelling”).
Claims 9 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Metzler in view of Asbag, , further in view of Gowal, further in view of Beggel et al. (Pub. No. EP 3477553 A1, published May 1st 2019, hereinafter Beggel).
Regarding claim 9, which is dependent upon claim 8:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 8 that claim 9 is dependent upon. Metzler in view of Asbag, further in view of Gowal does not disclose wherein the classifier is further configured to detect the at least one additional abstain class […] in response to the input data including one or more perturbations.
However, Beggel in the same field of endeavor of adversarial learning teaches:
Description of embodiments, excerpt: “…The Adversarial Autoencoder induces a prior distribution on the latent low dimensional space. This prior distribution can be predetermined and can be input into the Adversarial Autoencoder… Alternatively, a mixture of Gaussians distribution with one or more dedicated rejection classes (for anomalies) can be used, especially when the number of different anomaly classes is known...”
The dedicated rejection classes for anomalies would be the at least one additional abstain class.
Metzler in view of Asbag, further in view of Gowal and Beggel are analogous art because they are in the same field of endeavor.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag, further in view of Gowal that disclosed:
the method of claim 8
And the teachings of Beggel that disclosed:
wherein the classifier is further configured to detect the at least one additional abstain class […] in response to the input data including one or more perturbations.
This would have provided to Metzler in view of Asbag, further in view of Gowal the advantage of reliably identifying anomalous data (Beggel, “The method can reliably identify anomalies in images that were not contained in the training set”).
Regarding claim 12, which is dependent upon claim 8:
Metzler in view of Asbag, further in view of Gowal teaches the method of claim 8 that claim 12 is dependent upon. Metzler in view of Asbag, further in view of Gowal does not disclose wherein the processor is further configured to compute an upper bound and lower bound of the input data.
However, Beggel teaches:
Figure 2 description, excerpt: “…In this approach, the kernel-transformed normal data can be separated from the origin by a decision boundary whereas the kernel-transformed anomalies lie on the other side of the boundary closer to the origin…”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Metzler in view of Asbag that disclosed:
the method of claim 8
And the teachings of Beggel that disclosed:
wherein the processor is further configured to compute an upper bound and lower bound of the input data.
This would have provided to Metzler in view of Asbag, further in view of Gowal the advantage of reliably identifying anomalous data (Beggel, “The method can reliably identify anomalies in images that were not contained in the training set”).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Metzler in view of Asbag, , further in view of Gowal, further in view of Guan, further in view of Beggel.
Claim 13 recites a system that parallels the method of claim 4. Therefore, the analysis discussed above with respect to claim 4 also applies to claim 13. Accordingly, claim 13 is rejected based on substantially the same rationale as set forth above with respect to claim 4.
Response to Arguments
Applicant’s arguments filed 09-JUNE-2026 have been fully considered, but the examiner believes that not all are fully persuasive.
Regarding the applicant’s remarks on the non-final office action’s 103 rejection of the claims, the applicant argues that Metzler in view of Asbag does not teach the amended limitations of these claims. As such, the applicant argues that all claims dependent on the above would additionally not be obvious under 103. However, the examiner believes that Metzler in view of Asbag does teach the amended limitation generating augmented training data by augmenting a training data set using at least a term promoting classification of adversarial inputs into respective additional abstain classes of at least two additional abstain classes corresponding to different perturbation conditions for the reasons described below:
The applicant states that “a rejection bin or low-confidence fallback state is fundamentally different from the presently claimed learned abstain-class architecture”. However, the present specification states that an abstain option is a rejection option (Paragraph 18), matching the language used by Asbag. Furthermore, as abstain classes are known in the art, they are used for a low-confidence fallback state when the model would provide an uncertain output. For that reason, the examiner believes that Asbag’s rejection bins are at a minimum analogous to an abstain class.
Therefore, Asbag teaches the amendment aspect of the limits at least two additional abstain classes corresponding to different perturbation conditions. For example, Asbag teaches two different rejection bins (as above, considered to be abstain classes) for two different rejected defect scenarios (different perturbation conditions), wherein a rejected defect that may be more than one class is labeled as “cannot decide”, while a rejected defect that is not part of any class is labeled as “unknown” (Column 7, lines 20-30).
However, the examiner agrees that the prior art of the original office action does not teach determining a first lower bound [for each of the one or more correct classes of the plurality of classes]; determining, using interval bound propagation over perturbed hidden layer values, a second lower bound [for each of the at least two additional abstain classes]; using at least one of the worst-case bound, the first lower bound, and the second lower bound, outputting a classification in response to the input data indicating one of the plurality of classes. Therefore, the examiner has written a new rejection under 103 using previously presented art Gowal to address these limitations and respectfully requests applicant’s consideration of the following:
Gowal recites: “IBP’s goal is to find an upper bound on the optimal value of the problem (4). The simplest approach is to bound the activation zk of each layer by an axis-aligned bounding box” (Page 3) as well as “In the context of classification under adversarial perturbation, solving the optimization problem (8) for each target class y= ytrue” (Page 4)
Gowal teaches using interval bound propagation to determine a lower bound for multiple classes, as the optimization problem described requires the lower bound for the class. Therefore, both a first and second lower bound for various classes would be determined.
Gowal further recites: “Our experiments have shown that the proposed approach outperforms competing techniques in terms of verified bounds on adversarial error rates in image classification problems” (Page 7).
Therefore, Gowal discloses that using the previously determined bounds, a classification is output indicating one of the plurality of classes as it would be part of an image classification problem. Gowal could then combine with Metzler and Asbag in order to apply its determination of the lower bound to a classifier with correct classes and abstain classes.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDRIA JOSEPHINE MILLER whose telephone number is (703)756-5684. The examiner can normally be reached Monday-Thursday: 7:30 - 5:00 pm, every other Friday 7:30 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.J.M./Examiner, Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights