DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1 and 5 are objected to because of the following informalities: the term “AI” needs to be spelled out at least the first time its mentioned, such as “Artificial intelligence (AI)”. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims’ subject matter eligibility will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”).
With respect to claim 1.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Is the claim to a process, machine, manufacture, or composition of matter? Yes—claim 1 recites a method, which is a process.
Step 2A, prong one: Does the claim recite an abstract idea, law of nature or natural phenomenon? Yes—the limitations identified below each, under its broadest reasonable interpretation, covers mental processes abstract idea grouping (concepts performed in the human mind (including an observation, evaluation, judgment, opinion)), see MPEP 2106.04(a)(2), subsection III and the 2019 PEG, but for the recitation of generic computer components:
“diagnosing learnings of the extracted self-learned AI models on (Mental processes- concept of observation and evaluation).
Step 2A, prong two: Does the claim recite additional elements that integrate the judicial exception into a practical application? No—the judicial exception is not integrated into a practical application.
“extracting the self-learned AI models from the edge devices to a version controlled database;” involves the mere gathering of data, which is insignificant extra-solution activity. See MPEP § 2106.05(g).
“performing federated learning on the selected group of AI models to get a re-baselined model” and “a digital twin environment of the corresponding edge device”: mere instructions to “apply it” because it only includes high level of generality description to apply the abstract idea. See MPEP § 2106.05(f).
“pushing the re-baselined model into the plurality of edge devices using firmware over the air”: involves the mere gathering of data, which is insignificant extra-solution activity. See MPEP § 2106.05(g).
The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No—there are no additional limitations beyond the mental processes identified above. The limitation treated above, are directed to the well-understood, routine, and conventional activity of storing and retrieving information in memory. See MPEP § 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). It also includes limitations that Merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The additional element is insignificant application, which is similar to examples of activities that the courts have found to be insignificant extra-solution activity, in accordance with MPEP 2106.05(g), Insignificant Extra-Solution Activity. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 2.
Step 1: A method, as above.
Step 2A Prong 1: The judicial exceptions of claim 1 are incorporated.
Step 2A Prong 2, Step 2B: The judicial exceptions are not integrated into a practical application. The claim recites “wherein each edge device of the plurality of edge devices run a specified version of the AI model”: This limitation merely recites generic training, Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984 (see MPEP § 2106.05(f)). The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as disclosed above. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. The above limitation, mere instructions to apply as
it recites only the idea of a solution or outcome, MPEP 2106.05(f). Thus, in examining the claim
elements as recited by the limitations individually and as an ordered combination, as a whole the
independent claims do not recite what have the courts have identified as "significantly more".
The claim is not patent eligible.
Claim 3.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the diagnosis of learnings is done by testing a performance of the extracted self-learned AI models on critical tasks.”: This limitation merely specifies mental processes- concept of observation and evaluation.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 4.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the selection of the group of AI models is based on a performance of the AI models in critical tasks”: This limitation merely specifies mental processes- concept of observation and evaluation.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claims 5-8
Step 1: The claims recite an system; therefore, they fall into the statutory category of machines.
Step 2A Prong 1: The claims recite the same mental processes as claims 1-4, respectively.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Claims 5-8 recite generic computer components, namely “control system for re-baselining a plurality of AI models, the control system comprising: a processor; a memory; and at least one network interface, wherein the plurality of AI models reside in a plurality of independent edge devices,
wherein the AI models are adapted to self-learn in the plurality of independent edge devices,
wherein the plurality of independent edge devices are connected to the at least one network interface”. As before, the mere recitation that the method is to be performed on a generic computer amounts to a mere instruction to apply the exception on the computer. See MPEP § 2106.05(f). With that exception, the analysis mirrors that of claims 1-4, respectively.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The analysis, with the one exception noted above, mirrors that of claims 1-4, respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ito et al. (“An On-Device Federated Learning Approach for Cooperative Model Update Between Edge Devices”, VOLUME 9, June 11, 2021, pages 92986- 92998) in view of Lu et al. (“Communication-Efficient Federated Learning and Permissioned Blockchain for Digital Twin Edge Networks”, IEEE INTERNET OF THINGS JOURNAL, VOL. 8, NO. 4, FEBRUARY 15, 2021).
Regarding claim 1.
Ito teaches a method of re-baselining a plurality of AI models residing in a plurality of independent edge devices, the AI models adapted to self-learn in the edge devices, the method comprising:
extracting the self-learned AI models from the edge devices to a version controlled database (see page 92990, section B, “First, edge devices independently execute a sequential training by using OS-ELM algorithm. They also compute the intermediate results U and V by Equation 15. Second, they upload their intermediate results to a server.”, also see page 92987, “A federated learning framework was proposed by Google in 2016 [7]–[9]. Their main idea is to build a global federated model at a server side by collecting locally trained results from distributed client devices.”);
diagnosing learnings of the extracted self-learned AI models onsee page 92987, “As a common manner, a server side in federated learning systems has no access to local data in client devices. There is a risk that a client may get out of normal behaviors in the federated model training. In [15], a dimensionality reduction based anomaly detection approach is utilized to detect anomalous model updates from clients in a federated learning system… malicious clients are identified by clustering their submitted features”);
selecting a group of self-learned AI models based on said diagnosis (see page 92987, “malicious clients are identified by clustering their submitted features, and then the final global model is generated by excluding updates from the malicious clients.”, also see page 92990, “Regarding the client selection strategy that determines which models of client devices are merged, in this paper we… A client selection strategy that can improve anomaly detection accuracy by excluding unsatisfying local models is proposed in [19]. Our proposed on-device federated learning can be combined with these client selection strategies in order to improve the accuracy or efficiency, though such a direction is our future work”);
performing federated learning on the selected group of AI models to get a re-baselined model (see page 92990, “Edge devices can share their trained results by exchanging their intermediate results U and V in the proposed on-device federated learning approach, which can mitigate the privacy issues since they do not share raw data for the cooperative model update.”, also see page 92994, “In each communication round, two patterns are trained separately based on a single global model. Then, these locally trained models are averaged, and the global model is updated, which will be used for local train of the next round.”);
validating the re-baselined model see page 92987, “malicious clients are identified by clustering their submitted features, and then the final global model is generated by excluding updates from the malicious clients.”, also see abstract, “Our approach is evaluated with anomaly detection tasks generated from a driving dataset of cars, a human activity dataset, and MNIST dataset.”);
and pushing the re-baselined model into the plurality of edge devices using firmware over the air (see page 92986, “retraining or customizing a model is required at edge devices as the model is becoming outdated due to environmental changes over time (i.e., concept drift). Generally, retraining the model later to reflect environmental changes for each edge device is a complicated task, because the server machine needs to collect training data from the edge device, train a new model based on the collected data, and then deliver the new model to the edge device.”, also see page 92990, “Figure 5 shows a flowchart of the proposed cooperative model update between two devices: Device-A and Device B. Assuming Device-A sends its intermediate results and Device-B receives them for updating its model, their cooperative model update is performed by the following steps.”).
Ito do not teach AI models on a digital twin environment.
Lu teaches AI models on a digital twin environment (see page 2283, figure 4, Digital twin-empowered DNN,
PNG
media_image1.png
330
422
media_image1.png
Greyscale
).
Both Ito and Lu pertain to the problem of edge devices federated learning, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Ito and Lu to teach the above limitations. The motivation for doing so would be “we integrate digital twins with edge networks and propose the digital twin edge networks (DITENs) to fill the gap between physical edge networks and digital systems. Then, we propose a blockchain-empowered federated learning scheme to strengthen communication security and data privacy protection in DITEN. Furthermore, to improve the efficiency of the integrated scheme, we propose an asynchronous aggregation scheme and use digital twin empowered reinforcement learning to schedule relaying users and allocate spectrum resources. Theoretical analysis and numerical results confirm that the proposed scheme can considerably enhance both communication efficiency and data security for IoT applications.” (see Lu Abstract).
Regarding claim 2.
Ito and Lu teach the method of claim 1,
Ito further teaches wherein each edge device of the plurality of edge devices run a specified version of the AI model (see page 92990, “they upload their intermediate results to a server. We assume that the input weight matrix α and the bias vector b are the same in the edge devices. They download necessary intermediate results from the server if needed. They update their model based on their own intermediate results and those downloaded from the server by Equation 8.”).
Regarding claim 3.
Ito and Lu teach the method of claim 1,
Ito further teaches wherein the diagnosis of learnings is done by testing a performance of the extracted self-learned AI models on critical tasks (see page 92987, “As a common manner, a server side in federated learning systems has no access to local data in client devices. There is a risk that a client may get out of normal behaviors in the federated model training. In [15], a dimensionality reduction based anomaly detection approach is utilized to detect anomalous model updates from clients in a federated learning system… malicious clients are identified by clustering their submitted features”).
Regarding claim 4.
Ito and Lu teach the method of claim 1,
Ito further teaches wherein the selection of the group of AI models is based on a performance of the AI models in critical tasks (see page 92987, “malicious clients are identified by clustering their submitted features, and then the final global model is generated by excluding updates from the malicious clients.”, also see page 92990, “Regarding the client selection strategy that determines which models of client devices are merged, in this paper we… A client selection strategy that can improve anomaly detection accuracy by excluding unsatisfying local models is proposed in [19]. Our proposed on-device federated learning can be combined with these client selection strategies in order to improve the accuracy or efficiency, though such a direction is our future work”).
Regarding claim 5.
Ito teaches a control system for re-baselining a plurality of AI models, the control system comprising: a processor; a memory; and at least one network interface,
wherein the plurality of AI models reside in a plurality of independent edge devices,
wherein the AI models are adapted to self-learn in the plurality of independent edge devices,
wherein the plurality of independent edge devices are connected to the at least one network interface (see page 92990, “Figure 4 illustrates a cooperative model update of the proposed on-device federated learning.”
PNG
media_image2.png
366
604
media_image2.png
Greyscale
), wherein the at least one network interface is configured to extract the self-learned AI models from the edge devices to a version controlled database stored in the memory (see page 92990, section B, “First, edge devices independently execute a sequential training by using OS-ELM algorithm. They also compute the intermediate results U and V by Equation 15. Second, they upload their intermediate results to a server.”, also see page 92987, “A federated learning framework was proposed by Google in 2016 [7]–[9]. Their main idea is to build a global federated model at a server side by collecting locally trained results from distributed client devices.”); wherein the processor is configured to:
diagnose the learnings of the extracted self-learned AI models see page 92987, “As a common manner, a server side in federated learning systems has no access to local data in client devices. There is a risk that a client may get out of normal behaviors in the federated model training. In [15], a dimensionality reduction based anomaly detection approach is utilized to detect anomalous model updates from clients in a federated learning system… malicious clients are identified by clustering their submitted features”);
select a group of self-learned AI models based on said diagnosis (see page 92987, “malicious clients are identified by clustering their submitted features, and then the final global model is generated by excluding updates from the malicious clients.”, also see page 92990, “Regarding the client selection strategy that determines which models of client devices are merged, in this paper we… A client selection strategy that can improve anomaly detection accuracy by excluding unsatisfying local models is proposed in [19]. Our proposed on-device federated learning can be combined with these client selection strategies in order to improve the accuracy or efficiency, though such a direction is our future work”);
perform federated learning on the selected group of AI models to get a re-baselined model (see page 92990, “Edge devices can share their trained results by exchanging their intermediate results U and V in the proposed on-device federated learning approach, which can mitigate the privacy issues since they do not share raw data for the cooperative model update.”, also see page 92994, “In each communication round, two patterns are trained separately based on a single global model. Then, these locally trained models are averaged, and the global model is updated, which will be used for local train of the next round.”);
validate the re-baselined model see page 92987, “malicious clients are identified by clustering their submitted features, and then the final global model is generated by excluding updates from the malicious clients.”, also see abstract, “Our approach is evaluated with anomaly detection tasks generated from a driving dataset of cars, a human activity dataset, and MNIST dataset.”);
and push the re-baselined model into the plurality of edge devices through the network interface (see page 92986, “retraining or customizing a model is required at edge devices as the model is becoming outdated due to environmental changes over time (i.e., concept drift). Generally, retraining the model later to reflect environmental changes for each edge device is a complicated task, because the server machine needs to collect training data from the edge device, train a new model based on the collected data, and then deliver the new model to the edge device.”, also see page 92990, “Figure 5 shows a flowchart of the proposed cooperative model update between two devices: Device-A and Device B. Assuming Device-A sends its intermediate results and Device-B receives them for updating its model, their cooperative model update is performed by the following steps.”).
Ito do not teach AI models on a digital twin environment.
Lu teaches AI models on a digital twin environment (see page 2283, figure 4, Digital twin-empowered DNN,
PNG
media_image1.png
330
422
media_image1.png
Greyscale
).
Both Ito and Lu pertain to the problem of edge devices federated learning, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Ito and Lu to teach the above limitations. The motivation for doing so would be “we integrate digital twins with edge networks and propose the digital twin edge networks (DITENs) to fill the gap between physical edge networks and digital systems. Then, we propose a blockchain-empowered federated learning scheme to strengthen communication security and data privacy protection in DITEN. Furthermore, to improve the efficiency of the integrated scheme, we propose an asynchronous aggregation scheme and use digital twin empowered reinforcement learning to schedule relaying users and allocate spectrum resources. Theoretical analysis and numerical results confirm that the proposed scheme can considerably enhance both communication efficiency and data security for IoT applications.” (see Lu Abstract).
Claim 6-8 recites a system to perform the method recited in claims 2-4. Therefore the rejection of claims 2-4 above applies equally here.
Related prior art:
Pezzillo et al. (US 20190370687 A1) teaches a trained instance of an ML model is received at the edge computing devices via communications networks from an ML model manager. Feedback data including labeled observations is generated by the execution of the trained instance of the ML model at the edge computing devices on unlabeled observations captured by the edge computing devices. The feedback data is transmitted from the edge computing devices to a machine learning model manager.
PHAM et al. (US 20200125942 A1) teaches ensemble models require increased processing power and memory resources, leaving them unsuitable for execution directly on the user device. Moreover, such ensemble models may require a combination of results from disparate model types (e.g., combining output of a neural network and output of a linear regression). This may require different combinatory functions because each model type may produce different output.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IMAD M KASSIM whose telephone number is (571)272-2958. The examiner can normally be reached 10:30AM-5:30PM, M-F (E.S.T.).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297 - 4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IMAD KASSIM/Primary Examiner, Art Unit 2129
Prosecution Insights