Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application and claims filed 12/15/2023. Claims 1-18 are pending and have been examined. Claims 1-18 are rejected.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/15/2023, 12/11/2024, and 09/09/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The present application claims foreign priority based on Indian patent application number 202241058188 filed October 12, 2022. The examiner notes that a certified copy (in English) of the above-noted application was retrieved on 01/12/2024. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 14, 15, 16, 17 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 14 recites the limitation " the difference" in the phrase " determining whether the difference between the combined output and the output of the trained AI model…". There is insufficient antecedent basis for this limitation in the claim.
Claim 15 recites the limitation "the model" in the phrase "by transmitting the model to a third party". There is insufficient antecedent basis for this limitation in the claim. (Examiner’s Note: Claim 15 depends on Claim 12, which recites two distinct models: "a validation model" and "the trained AI model". When Claim 15 refers simply to "the model," it is unclear which of the two models is being transmitted to the server.)
Claim 16 recites the limitation “each validation model” in the phrase “wherein the output of each validation model is inferenced”. There is insufficient antecedent basis for this limitation in the claim. (Examiner’s Note: Claim 16 depends on claim 12 which strictly recites the deployment of "a validation model" (singular). By using the term "each," Claim 16 implies a plurality of validation models that do not exist in the parent claim.)
Claim 17 recites the limitation "the changes or deviations" in the phrase "maintain a record of the changes or deviations that occurred during on-device training of the validation model". There is insufficient antecedent basis for this limitation in the claim. (Examiner's Note: Regarding Claim 17, the phrase "the changes or deviations that occurred during on-device training of the validation model" lacks proper antecedent basis because it was not previously recited. Independent Claim 12 recites "a plurality of anticipated configurational changes" and a "set of actual configurational deviations," but both of these limitations are explicitly associated with the trained AI model, not the validation model. Therefore, it is unclear what "the changes or deviations" regarding the validation model are referring to.)
Claim Objections
Claim 5 objected to because of the following informalities: “without storage of validation dataset on the device.” should read “without storage of a validation dataset on the device.” Appropriate correction is required.
Claim 11 objected to because of the following informalities: “determining the difference based on one of an error function…” should read “determining the difference based on an error function…” Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Examiner’s Note: Some rejections will include an Examiner’s Note (labeled ‘EN’) to provide additional context or rationale explaining the basis for the rejection.
Claims 1-7, 10, 12-16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Jung (US 20210056411 A1), hereinafter "Jung" in view of Li et al. ("Learning to Detect Malicious Clients for Robust Federated Learning"), hereinafter "Li", and further in view of Tong et al. (US 20230139521 A1), hereinafter "Tong".
Claim 1
Jung teaches,
A method for validating a trained artificial intelligence (AI) model on a device, the method comprising: (Para 2, "The present disclosure generally relates to automatically validating whether newly learned neural network model parameters improve a performance of the neural network trained by automatically-labeled training and validation data." Para 9, "The validation is particularly important when the transfer learning is executed on an edge device because it is necessary to validate whether the newly learned neural network model parameters improve the performance of the neural network before updating the pre-existing model parameters." Para 43, "The second phase of the neural network training is performed on an edge device 150 with one or more processors 170. The second phase neural network training utilizes the pre-trained neural network 112 at the first phase as the basic network. The systems and methods in the present disclosure are applied to the refinement of a neural network model at a transfer learning process after the edge device 150 is deployed into a locally constrained physical environment." - EN: this denotes the pre-trained neural network 112 deployed on the edge device 150 and refined by on-device transfer learning, which reads on the "trained artificial intelligence (AI) model" on a "device". Automatically validating whether its newly learned model parameters improve performance constitutes "validating a trained artificial intelligence (AI) model on a device".)
deploying, at the device, a validation model (...) (Para 61, "As shown by block 503, the edge device 150 inputs training data into a second neural network for auto-labeling, wherein the second neural network corresponds to a slightly overfitted neural network model. In some implementations, the slightly overfitted neural network utilizes a copy of the same inference neural network model as the pre-trained neural network (e.g., pre-trained neural network 112 shown in FIG. 1) to generate the slightly overfitted neural network by relaxing parameters such that the requirements of non overfitting are relaxed." Para 58, "In some implementations, the slightly-overfitted neural network process 500b is performed on the edge learning module 180 on the edge device 150." Para 8, "the proposed label approximations and consequent output of the transfer learning using the approximated labels by the neural network are validated for degree of changes and correctness." - EN: under the broadest reasonable interpretation (BRI), the second, slightly overfitted neural network generated and executed on the edge device 150 constitutes a "validation model" deployed "at the device", because its output gives the second confidence condition used to validate the approximated labels and the consequent output of the on-device training of the deployed model. See also Para 28, "The second neural network is a version of the first neural network overfitted to the environment.")
providing, at the device, input data to each of the validation model and the trained AI model for receiving an output from each of the validation model and the trained AI model, (...) (Para 63, "In some implementations, the confidence upgrade process using a slightly overfitted neural network process may be formalized as follows:" Para 68, "yi: Softmax output for input xi by the pre-trained base neural network model M" Para 69, "yi': Softmax output for input xi by the overfitted neural network model Mi’" Para 95, "In some implementations, the confidence upgrade process 700 is performed by the edge learning module 180 (e.g., the edge learning module 180 shown in FIG. 1) on the edge device 150." - EN: Per the Para 63, yi=M(xi) and yi'=Mk'(xi), the same input vector xi is provided to the pre-trained base model M and to the slightly overfitted (validation) network Mk', and a Softmax output is received from each, on the edge device. This denotes providing, at the device, input data to each of the validation model and the trained AI model and receiving an output from each.)
combining, at the device, the output of each of the validation model and the trained AI model; and (Para 97, "As shown in box 703, the edge device 150 calculates a weighted sum of the first confidence condition, the second confidence condition, the third confidence condition, in order to adjust the final values of the Softmax function... Then, the final Softmax values are scaled-up from the initial Softmax values, as a weighted adjustment (i.e., the weighted sum of the first confidence condition, the second confidence condition, and the third confidence condition)." Para 71, "Ci(xi): output of confidence upgrade for input xi" - EN: equation (2) of Para 63, Ci(xi) = yi + w'(yi'), forms the confidence upgrade output by adding the Softmax output yi of the deployed base model to the weighted Softmax output w'(yi') of the overfitted (validation) network, on the edge device. This constitutes "combining, at the device, the output of each of the validation model and the trained AI model".)
Jung does not explicitly teach:
generated by applying a plurality of anticipated configurational changes associated with the trained AI model requiring validation;
the output of the validation model being further based on one or more actual configurational deviations that occurred during training of the trained AI model since deployment of the trained AI model on the device;
However, Li teaches:
generated by applying a plurality of anticipated configurational changes associated with the trained AI model requiring validation; (Section 4.2, p. 5, "We use the test data of the three datasets to generate the model weights for training the corresponding detection model. This is done by using the test data to train the same LR, CNN, and RNN models in a centralized setting and collecting the model weights of each update step. We then use the collected model weights to train the corresponding detection model." Section 3.3, p. 3, "To train such a spectral anomaly detection model, we rely on the centralized training process, which provides unbiased model updates." - EN: the detection (validation) model is generated by training it upon the collected model weights of each update step of the same model that requires validation, gathered in advance of deployment. Each collected update step represents the parameter configuration reached by a step of training of that model, and the succession of update steps thereby represents the step-to-step parameter changes the model undergoes during training. Under BRI, these collected weight-update steps are a "plurality of anticipated configurational changes associated with the trained AI model requiring validation", because they establish, in advance of the on-device training, the changes that training of the same model is anticipated to produce, and generating the detection model by training upon them constitutes generating the validation model by applying them.)
the output of the validation model being further based on one or more actual configurational deviations that occurred during training of the trained AI model since deployment of the trained AI model on the device; (Section 3.4, pp. 3-4, "After obtaining the spectral anomaly detection model, we apply it in every round of the FL model training to detect malicious client updates. Through encoding and decoding, each client's update will incur a reconstruction error." Section 3.3, p. 3, "We feed the malicious and the benign model updates into our encoder to get their latent vectors" Section 1, p. 1, "Each client transfers the local model updates to the central server for immediate aggregation, while keeping the raw data in their local storage." - EN: each client's model update is the set of actual weight deviations produced by training the distributed model on the client device after the global model is deployed to the clients. The detection (validation) model receives these actual updates as its input, so its output (latent vector, reconstruction, and resulting reconstruction error) is "further based on" actual configurational deviations that occurred during training since deployment. In the combination, the second (validation) neural network of Jung is generated from, and further evaluates, the actual parameter deviations of Jung's on-device trained model, whose parameter history Jung already maintains (Jung, Para 146, Para 177).)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the on-device validation of training of a deployed neural network of Jung with the detection model generated from collected model weight updates and applied to actual model updates of Li during the generation and operation of Jung's second (validation) neural network. The motivation for doing so would be to validate the parameter changes produced by on-device training directly from the deviations themselves, without depending on labeled validation data, so that harmful or abnormal updates are detected even when they cannot be exactly anticipated in advance. As Li elaborates regarding the benefit of this spectral anomaly detection methodology in Section 1, pages 1-2, "We show that in such a low-dimensional latent feature space, the abnormal (i.e., malicious) model updates from clients can be easily differentiated as their essential features are drastically different from those of the normal updates, leading to targeted defense."
The combination of Jung and Li does not explicitly teach:
validating the trained AI model based on a comparison of the combined output and the output of the trained AI model.
However, Tong teaches:
validating the trained AI model based on a comparison of the combined output and the output of the trained AI model. (Para 55, "The comparison module 413 compares the output generated by the validation neural networks 410 with the output generated by the neural network 405. Based on the comparison, the comparison module 413 generates a comparison output indicative of the difference between the neural network 405 output and the validation neural network 410 output(s) via data path 445. The comparison module 413 compares the comparison output with a predetermined comparison threshold to determine whether the comparison output is greater than the predetermined comparison threshold." Para 56, "If the comparison output is greater than the predetermined comparison threshold, the comparison module 413 generates an alert and transmits the alert and the neural network 405 output to the server 145." Para 57, "The server 145 may initiate an update for one or more neural networks 405 based on the comparison output, such as causing the neural network 405 to update corresponding weights and biases using a loss function that incorporates the comparison output." - EN: Tong validates a deployed, trained neural network 405 by comparing, for the same input, the output of its validation path (one or more validation neural networks 410) against the trained network's own output, determining a comparison output indicative of the difference, comparing the difference with a predetermined threshold, generating an alert when the threshold is exceeded, and having the server initiate an update of the trained network based on the comparison output ("The present disclosure relates to validating, e.g., cross-checking, neural network output with output from multiple other neural network models.", Para 1). In the system of Jung as modified by Li, the output of the on-device validation path for a given input is the combined output Ci(xi) of Jung, that is, the trained model output yi combined with the weighted validation network output w'(yi') (Jung, Para 63). Applying Tong's comparison, thresholding, and update triggering to the outputs available in Jung's system therefore constitutes "validating the trained AI model based on a comparison of the combined output and the output of the trained AI model". Additionally, Claim 1 of Tong itself recites "generate an alert when a difference between the output generated by the first neural network and the output generated by the second neural network is greater than a predetermined comparison threshold.")
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the combined-output confidence mechanism on the edge device of Jung, as modified by Li, with the output-comparison model validation with threshold-triggered alerting and updating of Tong at the on-device validation stage. The motivation for doing so would be to detect in real time, without manually labeled ground truth, when on-device training has degraded the deployed model, and to trigger a corrective update only when the deviation between the validation-side output and the model output is significant. As Tong elaborates regarding the benefit of this cross-checking methodology in paragraph 58, "In these implementations, the ground truth data for the output generated by the neural network 405 is the output generated by the validation neural networks 410 based on the same received sensor data."
Claim 2
Regarding claim 2, Jung in view of Li further in view of Tong teaches all the limitations of claim 1 as cited above and Li further teaches:
The method as claimed in claim 1, comprising:
generating, at the device or outside the device, the validation model based on a training dataset and a validation dataset, prior to the deploying of the validation model at the device. (Section 4.2, p. 5, "We use the test data of the three datasets to generate the model weights for training the corresponding detection model. This is done by using the test data to train the same LR, CNN, and RNN models in a centralized setting and collecting the model weights of each update step. We then use the collected model weights to train the corresponding detection model. The trained anomaly detection model is available to the server when it processes the clients' updates in FL model training for each of the above FL tasks." Section 3.3, p. 3, "To train such a spectral anomaly detection model, we rely on the centralized training process, which provides unbiased model updates." - EN: the detection (validation) model is generated at the central server, which constitutes generating the validation model "outside the device"; the claim recites the location in the alternative, so a showing on either suffices. Under BRI, the test data used to train the LR, CNN, and RNN models in the centralized setting constitutes the "training dataset", because it is the dataset where the centralized training is performed, and the collected model weights with which the detection (validation) model is itself trained constitute the "validation dataset", because they are the dataset from which the validation model learns to validate model updates. The validation model is thereby generated based on both datasets: the training dataset from which the update steps are produced, and the validation dataset upon which the validation model is trained. The trained detection model being available before it processes any client updates constitutes generation "prior to the deploying of the validation model at the device".)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the on-device validation framework of Jung, as modified by Li, with the server-side generation of the detection model from a training dataset and a validation dataset of Li prior to its deployment. The motivation for doing so would be to relieve the resource-constrained edge device of the computational burden of building the validation model and to ensure a fully generated validation model is ready before any on-device training update requires validation. As Li elaborates regarding the benefit of this server-side generation methodology in Section 4.2, page 5, "The trained anomaly detection model is available to the server when it processes the clients' updates in FL model training for each of the above FL tasks."
Claim 3
Regarding claim 3, Jung in view of Li further in view of Tong teaches all the limitations of claim 1 as cited above and Tong further teaches:
The method as claimed in claim 1, wherein the validating of the trained AI model comprises one of successfully or unsuccessfully validating the trained AI model in response to determining whether a difference between the combined output and the output of the trained AI model is one of higher or lower than a predefined threshold. (Para 55, "The comparison module 413 compares the comparison output with a predetermined comparison threshold to determine whether the comparison output is greater than the predetermined comparison threshold." Para 56, "If the comparison output is greater than the predetermined comparison threshold, the comparison module 413 generates an alert and transmits the alert and the neural network 405 output to the server 145." Para 57, "If the comparison output is less than or equal to the predetermined comparison threshold, the comparison module 413 transmits the comparison output to the server 145." - EN: the comparison output is the determined difference between the validation-side output (the combined output, see claim 1) and the trained model's output. Determining whether that difference is greater than the predetermined comparison threshold, generating an alert when it is (an unsuccessful validation of the trained AI model), and following the less-than-or-equal branch when it is not (a successful validation), constitutes "one of successfully or unsuccessfully validating the trained AI model in response to determining whether a difference... is one of higher or lower than a predefined threshold".)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the combined-output validation on the edge device of Jung, as modified by Li, with the threshold-based success or failure determination of Tong during the validating step. The motivation for doing so would be to make the validation outcome an objective, automatic decision in which only deviations large enough to indicate degradation of the deployed model are treated as failures requiring attention. As Tong elaborates regarding the benefit of this threshold-based validation methodology in paragraph 56, "For example, the comparison module 413 can generate the alert to indicate that the comparison output is greater than the predetermined comparison threshold for further review purposes."
Claim 4
Regarding claim 4, Jung in view of Li further in view of Tong teaches all the limitations of claim 3 as cited above and Jung further teaches:
The method as claimed in claim 3, comprising:
one of retraining or discarding training of the trained AI model in response to unsuccessfully validating the trained AI model. (Para 198, "In some implementations, the method 900 further includes adjusting the updated currently-existing model parameters when the difference between the original model parameters and the updated currently-existing model parameters lies outside the threshold, wherein adjusting the updated currently-existing model parameters further comprises: setting the updated currently-existing model parameters to a previously existing model parameters, performing a factory reset on the updated currently-existing model parameters to the original model parameters, or updating the updated currently-existing model parameters to a new set of model parameters over a network." - EN: setting the updated currently-existing model parameters to previously existing model parameters or performing a factory reset to the original model parameters constitutes "discarding training of the trained AI model", and updating the updated currently-existing model parameters to a new set of model parameters over a network constitutes "retraining"; the claim recites these responses in the alternative, so a showing on either suffices. Jung thereby teaches the claimed corrective responses, performed when a validation determination is adverse.)
Claim 5
Regarding claim 5, Jung in view of Li further in view of Tong teaches all the limitations of claim 1 as cited above and Tong further teaches:
The method as claimed in claim 1, wherein the validating of the trained AI model comprises validating, in real-time, the trained AI model using the validation model without storage of validation dataset on the device. (Para 54, "Thus, the validation neural networks 410 can generate output based on the same sensor data received by the neural network 405, i.e., the same input." Para 58, "In these implementations, the ground truth data for the output generated by the neural network 405 is the output generated by the validation neural networks 410 based on the same received sensor data." Para 56, "In various implementations, the neural network 405 can operate in parallel with the validation neural networks 410." - EN: the ground truth used for the validation is generated on the fly by the validation networks from the same received sensor data, rather than retrieved from a validation dataset stored on the device, and the trained network operating in parallel with the validation networks on received data constitutes validating "in real-time". In the combination, the validation neural networks 410 of Tong correspond to the validation model as set forth at claim 1, that is, the second neural network of Jung as modified by Li, which likewise generates its output from the same input data xi that is provided to the trained AI model on the device (Jung, Para 63), and whose validation dataset, the collected model weights of Li, remains at the server as set forth at claim 2, such that no validation dataset is stored on the device.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the on-device validation of Jung, as modified by Li, with the live cross-checking of Tong, in which the validation networks generate the reference output from the same input received by the deployed model. The motivation for doing so would be to validate the deployed model continuously during normal operation without reserving device storage for a validation dataset. As Tong elaborates regarding the benefit of this live cross-checking methodology in paragraph 54, "Thus, the validation neural networks 410 can generate output based on the same sensor data received by the neural network 405, i.e., the same input."
Claim 6
Regarding claim 6, Jung in view of Li further in view of Tong teaches all the limitations of claim 1 as cited above and Li further teaches:
The method as claimed in claim 1, comprising:
determining the plurality of anticipated configurational changes associated with an on-device training of the trained AI model; (Section 4.2, p. 5, "This is done by using the test data to train the same LR, CNN, and RNN models in a centralized setting and collecting the model weights of each update step." - EN: collecting the model weights of each update step of the same model that will be trained in the field determines, in advance, the plurality of changes the model is anticipated to undergo. In the combination, these anticipate the parameter updates produced by the on-device training of Jung's deployed model.)
creating a set of anticipated deviation features based on the plurality of anticipated configurational changes; and (Section 3.3, p. 3, "To avoid the curse of dimensionality, we employ a low-dimensional representation, called a surrogate vector, of each model update vector by random sampling." - EN: the surrogate vector created from each model update vector is a feature representation of an anticipated configurational change; the set of surrogate vectors constitutes the claimed "set of anticipated deviation features".)
generating the validation model based on the set of anticipated deviation features. (Section 4.2, p. 5, "We then use the collected model weights to train the corresponding detection model." Section 3.3, p. 3, "To train such a spectral anomaly detection model, we rely on the centralized training process, which provides unbiased model updates." - EN: training the detection (validation) model on the surrogate-vector representations of the collected update steps constitutes generating the validation model based on the set of anticipated deviation features.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the generation of the validation network from collected model updates in the system of Jung as modified by Li with the surrogate-vector deviation features of Li during the generation of the validation model. The motivation for doing so would be to keep the validation model compact enough to train and execute efficiently on resource-constrained hardware by representing each high-dimensional parameter change as a low-dimensional feature vector. As Li elaborates regarding the benefit of this representation methodology in Section 3.3, page 3, "To avoid the curse of dimensionality, we employ a low-dimensional representation, called a surrogate vector, of each model update vector by random sampling."
Claim 7
Regarding claim 7, Jung in view of Li further in view of Tong teaches all the limitations of claim 6 as cited above and Li further teaches:
The method as claimed in claim 6, comprising:
training the validation model offline with a validation dataset prior to the deploying of the validation model at the device. (Section 3.3, p. 3, "To train such a spectral anomaly detection model, we rely on the centralized training process, which provides unbiased model updates." Section 4.2, p. 5, "We use the test data of the three datasets to generate the model weights for training the corresponding detection model... The trained anomaly detection model is available to the server when it processes the clients' updates in FL model training for each of the above FL tasks." - EN: training the detection (validation) model through the centralized training process, completed before the model processes any client updates from the field, constitutes training the validation model "offline" and "prior to the deploying of the validation model at the device" in the combination. The dataset with which the detection (validation) model is trained is the dataset of collected model weights, which constitutes the "validation dataset", consistent with the mapping set forth at claim 2; training the validation model with that dataset constitutes training the validation model "with a validation dataset".)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the validation model generation in the system of Jung as modified by Li with the offline, centralized training of the detection model of Li before deployment. The motivation for doing so would be to establish a stable and unbiased baseline for the validation model under controlled conditions before it is relied upon to judge on-device training updates. As Li elaborates regarding the benefit of this offline centralized training methodology in Section 3.3, page 3, "To train such a spectral anomaly detection model, we rely on the centralized training process, which provides unbiased model updates."
Claim 10
Regarding claim 10, Jung in view of Li further in view of Tong teaches all the limitations of claim 1 as cited above and Tong further teaches:
The method as claimed in claim 1, wherein validating of the trained AI model comprises:
determining a difference between the combined output and the output of the trained AI model; and (Para 55, "Based on the comparison, the comparison module 413 generates a comparison output indicative of the difference between the neural network 405 output and the validation neural network 410 output(s) via data path 445." - EN: as set forth at claim 1, in the combination the validation-side output for a given input is the combined output Ci(xi) of Jung; generating the comparison output indicative of the difference between the two outputs constitutes "determining a difference between the combined output and the output of the trained AI model".)
comparing the difference with a predefined threshold to validate the trained AI model. (Para 55, "The comparison module 413 compares the comparison output with a predetermined comparison threshold to determine whether the comparison output is greater than the predetermined comparison threshold. The predetermined comparison threshold may be selected based on empirical analysis." - EN: the "predetermined comparison threshold" constitutes the "predefined threshold", and the threshold determination validates the trained network's behavior as set forth at claim 1.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the combined-output validation on the edge device of Jung, as modified by Li, with the difference determination and predetermined threshold comparison of Tong during the validating step. The motivation for doing so would be to reduce the validation decision to a single quantitative measure that the device can check automatically and tune empirically for the deployment at hand. As Tong elaborates regarding the benefit of this difference-and-threshold methodology in paragraph 55, "The comparison module 413 compares the comparison output with a predetermined comparison threshold to determine whether the comparison output is greater than the predetermined comparison threshold. The predetermined comparison threshold may be selected based on empirical analysis."
Claim 12
Jung teaches,
A system for validating a trained artificial intelligence (AI) model on a device, the system comprising: (Para 2, "The present disclosure generally relates to automatically validating whether newly learned neural network model parameters improve a performance of the neural network trained by automatically-labeled training and validation data." Para 9, "The validation is particularly important when the transfer learning is executed on an edge device because it is necessary to validate whether the newly learned neural network model parameters improve the performance of the neural network before updating the pre-existing model parameters.")
a validation model; and at least one processor, wherein the at least one processor is configured to: (Para 43, "The second phase of the neural network training is performed on an edge device 150 with one or more processors 170." Para 206, "in some implementations the device 1100 includes one or more processing units 1102 (e.g., NMP, microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, and/or the like)" - EN: the edge device's one or more processors, which execute the edge learning and validation processes, constitute the "at least one processor"; the "validation model" is mapped at claim 1 (Jung's second, slightly overfitted neural network, as modified by Li).)
The remaining limitations of claim 12 are substantially the same as method claim 1, therefore claim 12 is rejected under the same rationale as claim 1.
Claims 13 and 14 recite substantially the same limitations as method claims 2 and 3 respectively. Therefore, claims 13 and 14 are rejected under the same rationale as claims 2 and 3.
Claim 15
Regarding claim 15, Jung in view of Li further in view of Tong teaches all the limitations of claim 12 as cited above and Li further teaches:
The system as claimed in claim 12, wherein the validation is performed by at least one of a local validation dataset pushed into the device or by transmitting the model to a third party or centralized server for validation. (Section 1, p. 1, "Each client transfers the local model updates to the central server for immediate aggregation, while keeping the raw data in their local storage." Section 3.4, pp. 3-4, "After obtaining the spectral anomaly detection model, we apply it in every round of the FL model training to detect malicious client updates." - EN: the claim recites alternatives, and a showing on a single alternative suffices. Transferring the local model updates, that is, the model's trained parameters, from the client device to the central server, where the detection model validates them in every round, constitutes "transmitting the model to a third party or centralized server for validation". In the combination, the edge device of Jung transmits its model updates to the central server hosting the detection model of Li, where they are screened, in addition to the on-device validating performed at the device as set forth at claim 12. Thereby satisfying the "transmitting the model to a third party or centralized server for validation" alternative.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the on-device validation system of Jung, as modified by Li and Tong, with the transfer of the model updates to a central server for validation of Li. The motivation for doing so would be to allow a resource-rich central server to screen model changes from many devices while the raw user data never leaves each device. As Li elaborates regarding the benefit of this centralized detection methodology in Section 1, page 2, "Third, by detecting and removing the malicious updates in the central server, their negative impacts can be fully eliminated."
Claim 16
Regarding claim 16, Jung in view of Li further in view of Tong teaches all the limitations of claim 12 as cited above and Tong further teaches:
The system as claimed in claim 12, wherein the output of each validation model is inferenced. (Para 51, "FIG. 4 is a diagram of an example validation network 400 for comparing an output generated by a neural network 405, e.g., a first neural network, with outputs generated by one or more validation neural networks 410, e.g., a plurality of second neural networks." Para 68, "It is understood that multiple validation neural networks 410 may be used in which the output of the neural network 405 is compared with corresponding outputs from each validation neural network 410." - EN: each validation neural network generates its output by executing the trained network on the received input data, which constitutes the output of each validation model being "inferenced". Jung likewise receives an inference output from its validation network, "yi': Softmax output for input xi by the overfitted neural network model Mk'" (Jung, Para 69).)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the validation model inference on the edge device of Jung, as modified by Li, with the plurality of validation neural networks each generating an inference output of Tong. The motivation for doing so would be to corroborate the validation result across several independently trained networks so that the bias of any single validation network does not decide the outcome. As Tong elaborates regarding the benefit of this multi-network cross-checking methodology in paragraph 32, "The validation neural networks can be trained on different datasets that can be partial observations with different bias from the real-world underlying distribution."
Claim 18
Jung teaches,
A non-transitory computer readable recording medium including a program executes a controlling method for validating a trained artificial intelligence (AI) model on a device, the method comprising: (Para 38, "In accordance with some implementations, a non-transitory computer readable storage medium has stored therein instructions, which, when executed by one or more processors of an electronic device, cause the electronic device to perform or cause performance of any of the methods described herein." Para 135, "In some implementations, the method 900 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory).")
The remaining limitations of claim 18 are substantially the same as method claim 1, therefore claim 18 is rejected under the same rationale as claim 1.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Jung in view of Li and Tong as applied to claim 7 above, and further in view of Bandler et al. ("Neural Inverse Space Mapping EM-Optimization"), hereinafter "Bandler".
Claim 8
Regarding claim 8, Jung in view of Li further in view of Tong teaches all the limitations of claim 7 as cited above and Li further teaches:
The method as claimed in claim 7, wherein generating the validation model comprises generating the validation model based on the training of the validation model with the validation dataset (...) (Section 4.2, p. 5, "We then use the collected model weights to train the corresponding detection model. The trained anomaly detection model is available to the server when it processes the clients' updates in FL model training for each of the above FL tasks." Section 3.3, p. 3, "To train such a spectral anomaly detection model, we rely on the centralized training process, which provides unbiased model updates." - EN: as set forth at claims 2 and 7, the dataset of collected model weights with which the detection (validation) model is trained constitutes the "validation dataset"; training the detection (validation) model with that dataset constitutes generating the validation model based on the training of the validation model with the validation dataset.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the offline training of the validation model in the system of Jung, as modified by Li and Tong, with the generation of the detection model by training it upon the collected model weights of Li, such that the validation model is generated based on its training with the validation dataset. The motivation for doing so would be to obtain the detection capability of the validation model from the validation dataset itself, so that the validation model can be constructed without labeled examples of abnormal updates. As Li elaborates regarding the benefit of this training-based generation methodology in Section 1, page 2, the spectral anomaly detection framework "works in both the unsupervised and semi-supervised settings".
The combination of Jung, Li, and Tong does not explicitly teach:
generated using an inverse space mapping technique.
However, Bandler teaches:
generated using an inverse space mapping technique. (Abstract, p. 1007, "We present neural inverse space mapping (NISM) optimization for electromagnetics-based design of microwave structures. The inverse of the mapping from the fine to the coarse model parameter spaces is exploited for the first time in a space mapping algorithm." Section II.B, p. 1007, "We realize parameter extraction, which consists of finding the coarse model parameters that makes the characterizing coarse responses Rcs as close as possible to the previously calculated Rfs. We continue by training the simplest neural network N that implements the inverse of the mapping from the fine to the coarse parameter space at the available points." - EN: Bandler's neural network N is trained with a dataset of parameter pairs generated by the parameter extraction step of the inverse space mapping algorithm, each pair relating a point in the fine model parameter space to the extracted point in the coarse model parameter space, and the trained network implements the inverse of the mapping. Bandler thereby teaches generating, using an inverse space mapping technique, the dataset with which a neural network is trained.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the generation of the validation model by training it with the validation dataset in the system of Jung, as modified by Li and Tong, with the generation of the training dataset by the inverse space mapping technique of Bandler during the offline generation of the validation model, such that the validation dataset with which the validation model is trained is generated using the inverse space mapping technique. The motivation for doing so would be to generate the validation dataset analytically, by a simple parameter extraction procedure over paired evaluations of the two models, rather than by manually labeling or separately gathering the dataset, thereby reducing the cost of preparing the validation dataset before deployment. As Bandler elaborates regarding the benefit of this inverse space mapping methodology in the Abstract, page 1007, "NISM optimization does not require up-front EM simulations, multipoint parameter extraction, or frequency mapping. It employs a simple statistical parameter extraction procedure."
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Jung in view of Li and Tong as applied to claim 1 above, and further in view of Joshi et al. (US 20210319300 A1), hereinafter "Joshi".
Claim 9
Regarding claim 9, Jung in view of Li further in view of Tong teaches all the limitations of claim 1 as cited above and Jung further teaches:
The method as claimed in claim 1, comprising:
maintaining a record of the one or more (...) (Para 140, "In some implementations, the updated model parameters are also stored in a database (e.g., memory 160 shown in FIG. 1) for model parameter records. Previous validation measurements with corresponding model parameters in the model parameter records may provide useful information regarding the newly proposed model parameters." Para 146, "Save param(Mt+1) to a history of model parameters HP" Para 177, "HP: a history of model parameters" - EN: Jung saves each set of updated model parameters produced by the on-device training of the deployed model to a database of model parameter records and to a history of model parameters HP. This constitutes the claimed "maintaining a record". That the recorded updates are the "actual configurational deviations" is shown by Li below.)
Jung does not explicitly teach:
actual configurational deviations occurred during training of the trained AI model to create a set of actual deviation features,
However, Li teaches:
actual configurational deviations occurred during training of the trained AI model to create a set of actual deviation features, (Section 3.4, pp. 3-4, "After obtaining the spectral anomaly detection model, we apply it in every round of the FL model training to detect malicious client updates. Through encoding and decoding, each client's update will incur a reconstruction error." Section 3.3, p. 3, "To avoid the curse of dimensionality, we employ a low-dimensional representation, called a surrogate vector, of each model update vector by random sampling." Section 3.3, p. 3, "We feed the malicious and the benign model updates into our encoder to get their latent vectors" - EN: as at claim 1, each client's model update is the set of actual weight deviations produced by training the deployed model on the device, that is, the "actual configurational deviations". Li represents each update as a model update vector, reduced to a surrogate vector; these representations constitute the "set of actual deviation features", using the same surrogate-vector representation relied upon at claim 6 for the anticipated deviation features. In the combination, each update recorded in Jung's model parameter records and history HP is represented as such a feature vector, which constitutes maintaining the record of the actual configurational deviations "to create a set of actual deviation features".)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the record of updated model parameters of Jung, as modified by Li and Tong, with the representation of each model update as a deviation feature of Li at the record-maintaining step. The motivation for doing so would be to store the changes the deployed model has actually undergone in the same feature form as the anticipated changes from which the validation model was generated, so that the validation model can evaluate them directly. As Li elaborates regarding the benefit of this feature representation methodology in Section 3.3, page 3, "These embeddings are expected to retain those important features that capture the essential variability in the data instances."
The combination of Jung, Li, and Tong does not explicitly teach:
the set of actual deviation features being a subset of a set of anticipated deviation features; and
determining, by the validation model, a validation coefficient as the output of the validation model based on the set of actual deviation features and the input data,
wherein the combining comprises combining the validation coefficient of the validation model and the output of the trained AI model to generate a corrected output for the trained AI model.
However, Joshi teaches:
the set of actual deviation features being a subset of a set of anticipated deviation features; and (Para 41, " Due to the exponential drift factor (t/t0).sup.−v (see, equation 1), the spread of the distribution changes over time, and that can be corrected by γ.sub.j. That is, the multiplicative coefficient γ.sub.j can be used to compensate for the change in the spread of the conductance distribution over time." Para 12, "The method is based on the observation that the conductance of an electronic device such as a PCM device has an exponential relation to the time." - EN: in Joshi, the stored synaptic weights of a deployed network (Para 52) deviate according to a known drift model (equation 1), so the family of deviations the weights will undergo is anticipated in advance, and the deviation actually measured at any calibration time tc is one instance within that family. Joshi thereby teaches actual deviations that lie within the deviations anticipated in advance. In the combination, the anticipated deviation features are those created from the model updates collected before deployment (Li, Section 4.2, p. 5, as relied upon at claim 1), and the actual deviation features created from the updates that occurred on the device fall within that set, such that "the set of actual deviation features" is "a subset of a set of anticipated deviation features".)
determining, by the validation model, a validation coefficient as the output of the validation model based on the set of actual deviation features and the input data, (Para 43, " In certain embodiments, the scaling factor γ.sub.j may be calibrated. To update γ.sub.j, the quantity Γ.sub.j|t0 is first computed (see, equation 7.1.a below) right after programming all the devices 12 in the array 10: " Para 44, " Next, another quantity Γ.sub.j|tc is computed (see, equation 7.1.b below) at a time tc after programming all the devices in the array 10, and γ.sub.j is then updated based on the ratio of Γ.sub.j|t0 to Γ.sub.j|tc, as given by equation 7.1.c below: " Para 47, "The procedure can be repeated throughout the operation of the system 1, at several distinct desired time instants tc." - EN: under BRI, γj constitutes the "validation coefficient", because it quantifies the correction needed for the deviated network. Joshi determines γj from the deviation actually measured between programming time t0 and calibration time tc, together with an applied input, because each quantity Γj is the output current measured under the calibration voltage input Vc. In the combination, the validation model of claim 1 makes this determination: it receives the input data of claim 1 (Jung, Para 63) and the actual deviation features (Li, Section 3.4, pp. 3-4), and, per Joshi, determines the validation coefficient as its output based on both.)
wherein the combining comprises combining the validation coefficient of the validation model and the output of the trained AI model to generate a corrected output for the trained AI model. (Para 40, " A current I.sub.j as obtained at the output of the j.sup.th column of the crossbar array structure may be erroneous, owing notably to a drift in the electrical conductance of, e.g., the PCM devices in that column. The current I.sub.j can nevertheless be corrected by using multiplicative coefficient γ.sub.j and an additive parameter β.sub.j, as shown in equation 7 below: Ī.sub.j=γ.sub.j×I.sub.j+β.sub.j (7)" Para 52, "the electronic devices 12 are programmed at step S10 for the electronic devices 12 to store synaptic weights pertaining to connections to nodes of a single layer of an ANN. The output currents obtained at the output lines 18 are obtained according to a multiply-accumulate operation." - EN: multiplying γj with the output current Ij of the array storing the trained network's weights generates the corrected output Īj. In the combination, the combining of claim 1 is performed as taught by Joshi, in place of the weighted addition of Jung (Para 63, equation (2)): the validation model outputs the validation coefficient, and the combining comprises multiplying, at the device, that coefficient with the output of the trained AI model to generate the corrected output. The corrected output serves as the combined output that is compared with the output of the trained AI model for the validating of claim 1 (Tong, Para 55).)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the on-device validation with combined outputs of Jung, as modified by Li and Tong, with the coefficient-based output correction of Joshi at the combining and validating steps, such that the combining of claim 1 is performed as the multiplication of the validation coefficient with the model output. The motivation for doing so would be to correct the output of the deployed model at inference time using a coefficient computed from the deviations actually measured since deployment, thereby retaining the accuracy of the deployed model without retraining it. As Joshi elaborates regarding the benefit of this coefficient-based correction methodology in paragraph 26, "The present embodiments make it possible to retain accuracy in the values stored in the electronic devices 12 over longer time periods, compared to known methods of drift correction in similar applications, in particular when applied to crossbar-based inference accelerators."
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Jung in view of Li and Tong as applied to claim 10 above, and further in view of Bergmann et al. ("Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings”), hereinafter "Bergmann".
Claim 11
Regarding claim 11, Jung in view of Li further in view of Tong teaches all the limitations of claim 10 as cited above and Tong further teaches:
The method as claimed in claim 10, wherein determining of the difference comprises determining the difference (...) (Para 55, "Based on the comparison, the comparison module 413 generates a comparison output indicative of the difference between the neural network 405 output and the validation neural network 410 output(s) via data path 445." - EN: the determining of the difference is as set forth at claim 10, wherein the validation-side output is the combined output in the combination.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the combined-output validation on the edge device of Jung, as modified by Li, with the difference determination of Tong. The motivation for doing so would be to reduce the validation decision to a single quantitative measure that the device can check automatically and tune empirically for the deployment at hand. As Tong elaborates regarding the benefit of this difference-and-threshold methodology in paragraph 55, "The comparison module 413 compares the comparison output with a predetermined comparison threshold to determine whether the comparison output is greater than the predetermined comparison threshold. The predetermined comparison threshold may be selected based on empirical analysis."
The combination of Jung, Li, and Tong does not explicitly teach:
based on one of an error function between the combined output and the output of the trained AI model.
However, Bergmann teaches:
based on one of an error function between the combined output and the output of the trained AI model. (Abstract, p. 1, "Anomalies are detected when the outputs of the student networks differ from that of the teacher network." Section 3.2, p. 4, "First, we propose to compute the regression error of the mixture's mean µ(r,c) with respect to the teacher's surrogate label:" Section 3.2, pp. 4-5, "The intuition behind this score is that the student networks will fail to regress the teacher's output within anomalous regions during inference since the corresponding descriptors have not been observed during training." - EN: the regression error of equation (8) is the squared L2-distance between the mean output of the student networks and the corresponding output of the teacher network for the same input, and thereby constitutes an "error function between" the outputs of two neural networks. In the combination, the comparison output of Tong as set forth at claim 10 is determined by computing the regression error of Bergmann between the two outputs whose difference is determined, which constitutes "determining the difference based on one of an error function between the combined output and the output of the trained AI model".)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the determination of the comparison output indicative of the difference on the edge device of Jung, as modified by Li and Tong, with the computation of the difference between the outputs of two neural networks as a regression error of Bergmann at the determining of the difference. The motivation for doing so would be to determine the difference as a defined, computable magnitude that can be compared directly against the predefined threshold and obtained efficiently on the resource-constrained device. As Bergmann elaborates regarding the benefit of this regression error methodology in Section 3.2, page 5, "Note that e(r,c) is non-constant even for M = 1, where only a single student is trained and anomaly scores can be efficiently obtained with only a single forward pass through the student and teacher network, respectively."
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Jung in view of Li and Tong as applied to claim 12 above, and further in view of Szeto et al. (US 20170124487 A1), hereinafter "Szeto".
Claim 17
Regarding claim 17, Jung in view of Li further in view of Tong teaches all the limitations of claim 12 as cited above and Szeto teaches:
The system as claimed in claim 12, wherein the at least one processor is further configured to:
maintain a record of the changes or deviations that occurred during on-device training of the validation model. (Para 214, "It is therefore in accordance with described embodiments that every time an update, batch update, or other type of model training occurs, a version for that resulting prediction engine variant is produced such that earlier variants of models may be unambiguously referenced" Para 175, "Unlike previously known platforms, the PredictionIO or machine learning platform tracks every instance and variant of created prediction engine variants and trained machine learning models associated with any given code base including the version of the software, the time, the tenant, organizational ID, and data sources utilized in its creation, and provides a unique ID for the deployment." Para 220, "According to one embodiment, a model management system is utilized to track the model variants and the various underlying changes to the source code, algorithm parameters, weightings, source data, range of data, etc., which result in any given trained model variant." - EN: producing and tracking a version for every occurrence of model training, together with the underlying changes in weightings and parameters that result in each trained model variant, constitutes maintaining "a record of the changes or deviations that occurred during" training of a model, and Szeto applies this tracking to every trained machine learning model, including models that are updated regularly as new data comes in (Szeto, Para 209). In the combination, the second (validation) neural network of Jung as modified by Li is generated and trained on the edge device 150 (Jung, Para 58, Para 61), and the at least one processor of Jung applying the version tracking of Szeto to that network maintains a record of the changes or deviations that occurred during on-device training of the validation model.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the on-device generation and training of the validation model in the system of Jung, as modified by Li and Tong, with the tracking of a version and its underlying changes for every occurrence of model training of Szeto at the validation model. The motivation for doing so would be to enable the device to identify and roll back a validation model update that degrades validation performance, so that a corrupted validation model does not silently invalidate the on-device validation of the deployed model. As Szeto elaborates regarding the benefit of this model version tracking methodology in paragraph 48, "Such a rollback mechanism is beneficial when a newer version of a trained model is not performing as expected or yields sub-par results in comparison to a previously known and utilized version of the trained model."
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAYMUR RAHMAN ALI whose telephone number is (571)272-0007. The examiner can normally be reached Mon-Fri. 9:30-6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571)270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NAYMUR RAHMAN ALI/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights