DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Germany on 02/09/2024 (i.e. DE10 2024 201 190.8.). It is noted, however, that applicant has not filed a certified copy of the application as required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 02/07/2025 and 09/19/2025 have been considered and placed in the applicant file.
Claim Objections
Claims 1, 3, 5, 14 and 18 are objected to because of the following informalities:
Claim 1, line 11, the term “a synthetic representation of the local data based on the parameterization using a generative Al function” should be changed to “a synthetic representation of the local data based on the parameterization using a generative artificial intelligence (AI) function” to correct for typographical errors. Appropriate corrections required.
Claim 3, line 2, the term “wherein the generative Al function is configured to” should be changed to “wherein the generative artificial intelligence (AI) function is configured to” to correct for typographical errors. Appropriate corrections required.
Claim 3, line 9, the term “and inputting the natural language prompt into the generative Al function to generate the synthetic representation” should be changed to “and inputting the natural language prompt into the generative artificial intelligence (AI) function to generate the synthetic representation” to correct for typographical errors. Appropriate corrections required.
Claim 5, line 1, the term “The method according to any claim 1” should be corrected “The method according to in order to correct for typographical errors.
Claim 14, line 12, the term “and a computing unit configured to generate a synthetic representation of the local data based on the parameterization using a generative AI function” should be changed to “and a computing unit configured to generate a synthetic representation of the local data based on the parameterization using a generative artificial intelligence (AI) function” to correct for typographical errors. Appropriate corrections required.
Claim 18, line 2, the term “the generative AI function is configured to” should be changed to “the generative artificial intelligence (AI) function is configured to” to correct for typographical errors. Appropriate corrections required.
Claim 18, line 9, the term “inputting the natural language prompt into the generative Al function to generate the synthetic representation” should be changed to “inputting the natural language prompt into the generative artificial intelligence (AI) function to generate the synthetic representation” to correct for typographical errors. Appropriate corrections required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means” or “step” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph because the claim limitation(s) do not recite(s) sufficient structure, materials, or acts to entirely perform the recited function.
Claims 1, 3, 11-12, 14 and 18 recites limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
Claim 1; recites the limitation, “receiving, at the model aggregator device…” [Line 4].
Claim 1; recites the limitation, “generating, at the model aggregator device…” [Line 9].
Claim 1; recites the limitation, “evaluating, at the model aggregator device…” [Line 12].
Claim 1; recites the limitation, “updating, at the model aggregator device…” [Line 16].
Claim 3; recites the limitation, “wherein the generative Al function is configured to…” [Line 1].
Claim 11; recites the limitation, “wherein the machine learned model is an image processing function configured to…” [Line 1].
Claim 12; recites the limitation, “wherein the machine learned model is configured to…” [Line 1].
Claim 14; recites the limitation, “an interface unit configured to…” [Line 1-2].
Claim 14; recites the limitation, “a computing unit configured to…” [Line 10].
Claim 18; recites the limitation, wherein the generative Al function is configured to…” [Line 1].
Claim 18; recites the limitation, wherein the generative Al function to…” [Line 9].
Because these claim limitation(s) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claims 1, 3, 11-12, 14 and 18:
“the model aggregator device” (Fig. 1, #MAD. Paragraph [0150, 0125-0127]- the model aggregator device is described as comprising both the model aggregator device MAD and a plurality of clients at different locations. The model aggregator device MAD is described as being hosted on a server, interfacing with the clients via a network and controlling, coordinating, and steering the federated learning procedures in system 1. (thus the model aggregator device MAD has sufficient structure/material/algorithm associated with it, wherein the model aggregator device MAD is a server,).
“the generative Al function” (Fig. 1 and. Paragraph [0029,]-the generative Al function is described as generating a synthetic representation of local data based on the parameterization. According to some examples, the generative AI function may comprise a transformer network, in particular, a transformer-based (deep) neural network. (thus the generative Al function has sufficient structure/material/algorithm associated with it, wherein the generative Al function is a neural network)).
“an interface unit” (Fig. 1, #IU. Paragraph [0131-0136 and 0152-0154]-the interface unit IU is described as including suitable components for interfacing with one or more networks to communicate with local sites, such as transmitters, receivers, ports, controllers, or other suitable components. Additionally, Figure 1 illustrates the interface unit IU as a black box within the model aggregator device MAD (thus the interface unit IU has sufficient structure/material/algorithm associated with it, wherein the interface unit is one or more transmitters, receivers, ports, that facilitate communication over a network)).
“a computing unit” (Fig. 1, #CU, Paragraph [0131-0135 and 0155-0158]-the computing unit CU is described as comprising working storage and one or more processors. The computing unit CU is also described as containing sub-units SYNTH, AGGR, and MGMT. (thus the computing unit CU has sufficient structure/material/algorithm associated with it, wherein the computing unit CU is one or more processors).
“image processing function” (Fig. 1, Paragraph [0088-0089]-the image processing function is described as a machine learning model being configured to provide an image processing result based on image data, the local data comprises training image data, the parameterization comprises a parameterization of the training image data, the synthetic representation comprises synthetic image data generated by the generative Al function based on the parameterization of the training image data (thus the image processing function has sufficient structure/material/algorithm associated with it, wherein the image processing function is a neural network for processing image data)).
“machine learning model” (Fig. 1, Paragraph [0019]-the machine learning model is described as being In particular, the machine-learned model can comprise a neural network, a support vector machine, a decision tree and/or a Bayesian network, and/or the trained function can be based on k-means clustering, Q-learning, genetic algorithms, a transformer network and/or association rules. (thus the machine learning model has sufficient structure/material/algorithm associated with it, wherein the machine learning model is a a neural network)).
If applicant does not intend to have these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-5, and 8-17 are rejected under 35 U.S.C. 103 as being unpatentable over ROTH et al. (US 20220366220 A1), hereinafter referenced as ROTH in view of CARETTE et al. (US 20260212206 A1), hereinafter referenced as CARETTE and in further view of ICKIN et al. (US 20230088561 A1), hereinafter referenced as ICKIN.
Regarding claim 1, ROTH explicitly teaches a computer-implemented method for federated learning (Fig. 1-3. Paragraph [0056]-ROTH discloses FIG. 1 is a block diagram illustrating an example architecture for federated learning by one or more neural networks 108, 116, 124, 134 to process medical data. Please also see Fig. 2-3) of a machine learned model (Fig. 1-3, #134, #234 and #358 called a global model. Paragraph [0060]. Please also read paragraph [0056-0057]) in a model aggregator device (Fig. 1-3, #132, #226 and #350 called a federated server. Paragraph [0056]-ROTH discloses a federated server 132 is a computing system comprising hardware components and memory containing software instructions that, when executed, train a global model 134 according to neural network data values collected from one or more edge devices or clients 102, 110, 118. A global model 134 is data values and software instructions that, when executed, perform image segmentation, classification, or any other neural network operations. Federated learning or federated training is neural network training using data and/or local neural network models 108, 116, 124 from edge devices or clients 102, 110, 118 at a plurality of locations. A federated server 132 performs federated training or federated learning of a global model 134 by aggregating neural network data values received from edge devices or clients 102, 110, 118. Please also see Fig. 4 and read paragraph [0522-0523]), the method comprising:
receiving, at the model aggregator device (Fig. 1-3, #132, #226 and #350 called a federated server. Paragraph [0060]. In paragraph [0077]-ROTH discloses FIG. 3 is a block diagram illustrating an architecture to perform federated learning using learnable aggregation weights 352, 354, 356. A federated server 350 updates a global model w 358 by dynamically selecting neural network data, such as neural network weights, gradients, or any other neural network data. Dynamically selecting neural network weights comprises selecting one or more sets of data from neural network weights according to one or more dynamic values, such as learnable aggregation weights 352, 354, 356. Please also read paragraph [0078-0080]), from a first local site (Fig. 1-3, #102, #110, #118, #202, #208, #216, #302, #314, #326 called edge devices or clients. Paragraph [0067]. In paragraph [0057]-ROTH discloses edge devices or clients 102, 110, 118 are computing systems located at medical, computing, or any other type of facility, where each facility may be located in geographically diverse positions, and comprise at least a computing system containing hardware and memory with software instructions that, when executed, train one or more local models 108, 116, 124, such as neural networks, to perform image segmentation, classification, or any other deep learning operation further described herein. Local models 108, 116, 124 are data values and software instructions that, when executed, perform deep learning operations, such as image segmentation or classification, to assist in or facilitate medical diagnosis) that is remote from the model aggregator device (Fig. 1-3, #132, #226 and #350 called a federated server. Paragraph [0060]. In paragraph [0062]-ROTH discloses one or more edge devices or client 102, 110, 118 computing systems are distributed globally or located at separate locations having separate local testing data 106, 114, 122. One or more edge devices or client 102, 110, 118 computing systems are located in different countries, cities, or other locales such as separate buildings or rooms in a building. One or more edge devices or client 102, 110, 118 facilities, such as a hospital, comprise testing services 104, 112, 120), a local update of the machine learned model (Fig. 1-3, #134, #234 and #358 called a global model. Paragraph [0060]. Please also see Fig. 4-5, and read paragraph [0069-0076 and 0093-0097]) and a parameterization (Fig. 1-3, #308, #310, #312, #320, #322, #324, #332, #334, #336, #352, #354, #356 called parameterized aggregation weights. Paragraph [0071, 0080 and 0088]. In paragraph [0080]-ROTH discloses each learnable aggregation weights a.sub.k 352, 354, 356 corresponds to a single client 302, 314, 328. Each learnable aggregation weights a.sub.k 352, 354, 356 corresponds to one or more groupings of edge devices or clients 302, 314, 328. Please also read paragraph [0068-0076 and 0115]) of local data (Fig. 1-3, #204, #210, #216, #304, #316, and #328 called local data. Paragraph [0071 and 0088]. In paragraph [0057]-ROTH discloses testing services 104, 112, 120 generate local testing data 106, 114, 122 usable for training one or more local models 108, 116, 124 by a client training framework implementing a training algorithm such as described below in conjunction with FIG. 5 (wherein local testing data corresponds to local data used to train local models). In paragraph [0091]-ROTH discloses during communication round t, once one or more edge devices or clients 302, 314, 326 finish updating their local models 306, 318, 330 in parallel, a federated server 350 gathers or transfers 338, 342, 346 neural network weights, such as local model 306, 318, 330 weight or scalar data values, as well as gathers or transfers 340, 344, 348 local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 from each edge device or client 302, 314, 326. Please also see Fig. 4-5, and read paragraph [0057, 0063, 0069-0076]), wherein the local update was generated at the first local site (Fig. 1-3, #102, #110, #118, #202, #208, #216, #302, #314, #326 called edge devices or clients. Paragraph [0067]) based on the local data (Fig. 1-3, #204, #210, #216, #304, #316, and #328 called local data. Paragraph [0071 and 0088]. In paragraph [0091]-ROTH discloses a federated server 350 comprising a federated learning framework, federated training framework, or other software framework collaboratively trains a global model w 358 using T communication rounds. During each communication round, a federated server 350 transfers 338, 342, 346 or sends out a global model w 358 to all edge devices or clients 302, 314, 326. One or more edge devices or clients 302, 314, 326 finish updating local models 306, 318, 330 in parallel. Please also see Fig. 4-5, and read paragraph [0057, 0063, 0069-0076]).
ROTH fails to explicitly teach generating, at the model aggregator device, a synthetic representation of the local data based on the parameterization using a generative Al function
However, CARETTE explicitly teaches generating, at the model aggregator device (Fig. 2, #200 called a server. Paragraph [0037]. In paragraph [0032]-CARETTE discloses FIG. 1 shows a block diagram of a first embodiment of a collaborative learning system 300 based on the present disclosure. The collaborative learning system 300 comprises a server 200 and at least one client device 100A. Further in paragraph [0067]-CARETTE discloses partial generative model 230 received by the first client device 100A may be trained locally, using the authentic sensor data 110 as additional training data, to become a trained partial generative model 230T. The trained partial generative model 230T may then be used to update the global generative model 210. Each sub-model sent to the client device 100A as part of the partial generative model 230 may become trained sub-models 230-2T, 230-3T, and 230-6T as part of the trained partial generative model 230T. Once given sufficient training from the authentic sensor data 110, these trained sub-models may be ready to be sent to the server 200 and to be incorporated into the corresponding global generative sub-models 210-2, 210-3, and 210-6. The server 200 may comprise an update algorithm configured to receive partial generative sub-models trained by the client device 100A and to incorporate the trained partial generative sub-models into the corresponding global generative sub-models. Please also read paragraph [0037 and 0134]), a synthetic representation (Fig. 2. Paragraph [0067]-CARETTE discloses the trained sub-models of the trained partial generative model 230T may also generate synthetic sensor data 310 with statistical properties corresponding to the authentic sensor data 110, which may expand a pool of data available to the local decision model 330 to be trained locally. Since both the authentic sensor data 110 and synthetic sensor data 310 may remain restricted to the client device 100A, a high standard of privacy may be maintained) of the local data (Fig. 2. Paragraph [0034]-CARETTE discloses the authentic sensor data 110 may be any data obtained from a measurement by one or more sensors connected to the first client device 100A. Types of sensors that may record physical data may include image sensors. Such sensors may record data related to physical phenomena that may relate to biometric or behavioral data of the user A (wherein authentic data and synthetic data is used as training data). Please also read paragraph [0036, 0047, 0067, and 0096]) based on the parameterization using a generative AI function (Fig. 2. Paragraph [0044]-CARETTE discloses the client device 100A comprises a respective client device circuitry 104 configured to receive the model information 240 from the server 200 (wherein model information 240 includes authentic statistical information 114 (e.g. a vector of values and metadata that describes the specific data context/type and statistical properties of characteristics or features of interest of authentic sensor data 110 corresponding to a statistical parameters), instructions for calculations, such as a profiling function and scoping function, etc.). Based on the received model information 240, the client device circuitry 104 is configured to determine the user-specific authentic statistical information 114 based on the user-specific authentic sensor data 110 and send the user-specific authentic statistical information 114 to the server 200. Further in paragraph [0072]-CARETTE discloses the profiling function 242 may comprise a set of statistical parameters to have one or more corresponding values calculated to obtain a set of statistical properties. The set of statistical parameters may be learned within the server 200 based on updates from one or more users of the collaborative learning system 30 (wherein the profiling function 242 may output authentic sensor data 110 in a more organized form and compatible form where data has been given meaning and context). Further in paragraph [0074]-CARETTE discloses the scoping function 246 may take the form of an embedding configured to receive as input the authentic sensor data 110 and generate as output embedded, or scoped, authentic sensor data 116 (wherein scoping function 246 may also filter unnecessary information). Please also read paragraph [0039-0046, 0069, 0076-0080 and 0100-0104]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, the method comprising: receiving, at the model aggregator device, from a first local site that is remote from the model aggregator device, a local update of the machine learned model and a parameterization of local data, wherein the local update was generated at the first local site based on the local data, with the teachings of CARETTE of having generating, at the model aggregator device, a synthetic representation of the local data based on the parameterization using a generative Al function.
Wherein ROTH’s method having generating, at the model aggregator device, a synthetic representation of the local data based on the parameterization using a generative Al function.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and CARETTE concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while CARETTE’s systems and methods improves federated learning and the ability to provide synthetic sensor data that accurately reflects the authentic sensor data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and CARETTE et al. (US 20260212206 A1), Abstract and Paragraph [0084].
Although ROTH explicitly teaches updating, at the model aggregator device, the machine learned model (Fig. 1-3, #134, #234 and #358 called a global model. Paragraph [0060]-ROTH discloses FIG. 1 is a block diagram illustrating an example architecture for federated learning by one or more neural. Please also read paragraph [0056-0057]) based on the local update (Fig. 3. Paragraph [0097]-ROTH discloses as a result of a training round t in T satisfying a condition, such as being a multiple of a specific value t.sup.0 or being a i.sup.th iteration or round, a server, such as a federated server or any other server further described herein, learns a global parameterization value b and computes parameterized global aggregation weights a for a round tin T, as described above in conjunction with FIGS. 3 and 4. As a result of a training round t in T not satisfying a condition, such as being a multiple of a specific value t.sup.0 or being a t.sup.th iteration or round, a server, such as a federated server or any other server further described herein, sets current global aggregation weights a for a current global training iteration or round t to previous global aggregation weights a from a previous global training iteration or round t−1. A server, such as a federated server or any other server, updates a global model w for a current training iteration or round t according to neural network data values, such as weights, obtained from K edge devices or client according to current global aggregation weights a for current round t. Please also read paragraph [0082, 0098, and 0115]).
ROTH fails to explicitly teach evaluating, at the model aggregator device, the local update using the synthetic representation to obtain an evaluation result indicative of performance of the local update; updating, at the model aggregator device, the machine learned model based on the evaluation result.
However, ICKIN explicitly teaches evaluating, at the model aggregator device (Fig. 8, #100 called a device or a master. Paragraph [0103]-ICKIN discloses FIG. 8 is a block diagram of a device, such as a master 100 (also referred to as a master node 100) for generating a synthetic dataset. Device 100 includes a processor circuit 44 a communication interface 42 coupled to the processor circuit, and a memory 46 coupled to the processor circuit 44. The memory 46 includes machine-readable computer program instructions that, when executed by the processor circuit, cause the processor circuit to perform the operations. Please also see Fig. 1, 5 and 7A, and read paragraph [0099-0102]), the local update using the synthetic representation (Fig. 2. Paragraph [0064]-ICKIN discloses Federated training methods are illustrated in FIG. 2, which shows a Federated Learning system 50. In FIG. 2, a synthetic training data generation process is used by each worker 200-1 to 200-N to create realistic, training datasets S′ that include similar but not identical training samples conditioned on the original training datasets S. That is, each worker 200-1 to 200-N, which has access to a real training dataset of size S, uses a generative model to generate a synthetic training dataset S′ having the same size as the real training dataset S. The real and synthetic datasets may be combined into a new training dataset of size 2S. In paragraph [0093]-ICKIN discloses FIG. 5 illustrates an example exchange between a master 100 and a worker 200-N in a federated learning system. The master 100 evaluates the trained weights at block 504, and in this example decides that the worker 200-N should use synthetic data for training. The master 100 indicates this to the worker 200-N vie a message 505 instructing the worker 200-N to generate synthetic training data) to obtain an evaluation result indicative of performance of the local update (Fig. 5. Paragraph [0094]-ICKIN discloses at block 506, the worker 200-N generates a synthetic training dataset S′ using. The worker 200-N generates a quality metric for the synthetic dataset, such as the KL divergence of synthetic training dataset, and transmits the quality metric to the master 100 in a message 507. The master evaluates the quality metric and decides that the worker 200-N should proceed with using the synthetic training dataset S′, which it indicates to the worker 200-N in a message 509. In paragraph [0095]-ICKIN discloses the worker 200-N then combines the real training dataset S with the synthetic training dataset S′ at block 510 and trains the ML model using the combined dataset (S, S′) at block 512. In paragraph [0096]-ICKIN discloses the worker 200-N then sends the re-trained weights to the master 100 in a message 513. The master 100 combines the re-trained weights with trained weights from other workers 200 at block 514, and transmits the combined weights to the workers 200 in a message 515. Further in paragraph [0090]-ICKIN discloses the larger combined training dataset may be used to train a NN model. In paragraph [0091]-ICKIN discloses the performance of the trained model may be evaluated on the real test set. Please also read paragraph [0097]);
updating, at the model aggregator device (Fig. 8, #100 called a device or a master. Paragraph [0103]), the machine learned model based on the evaluation result (Fig. 5. Paragraph [0094]-ICKIN discloses the master evaluates the quality metric and, in this example, decides that the worker 200-N should proceed with using the synthetic training dataset S′, which it indicates to the worker 200-N in a message 509. In paragraph [0095]-ICKIN discloses the worker 200-N then combines the real training dataset S with the synthetic training dataset S′ at block 510 and trains the ML model using the combined dataset (S, S′) at block 512. In paragraph [0096]-ICKIN discloses the worker 200-N then sends the re-trained weights to the master 100 in a message 513. The master 100 combines the re-trained weights with trained weights from other workers 200 at block 514, and transmits the combined weights to the workers 200 in a message 515. In paragraph [0097]-ICKIN discloses the decision to require synthetic data may depend on the quality of weights provided by a given worker and/or based on other consideration).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, the method comprising: receiving, at the model aggregator device, from a first local site that is remote from the model aggregator device, a local update of the machine learned model and a parameterization of local data, wherein the local update was generated at the first local site based on the local data, with the teachings of ICKIN of having evaluating, at the model aggregator device, the local update using the synthetic representation to obtain an evaluation result indicative of performance of the local update; updating, at the model aggregator device, the machine learned model based on the evaluation result.
Wherein ROTH’s method having updating, at the model aggregator device, the machine learned model based on the evaluation result and the local update
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and ICKIN concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while ICKIN’s systems and methods improves model performance in federated learning using synthetic data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and ICKIN et al. (US 20230088561 A1), Abstract and Paragraph [0063].
Regarding claim 2, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 1, ROTH further teaches wherein the parameterization includes a parameterization of data used for at least one of validating or testing the local update at the first local site (Fig. 1-3. Paragraph [0092]-ROTH discloses at each communication round t, each edge device or client 302, 314, 326 receives a copy of all neural network weights w.sub.1, . . . , w.sub.k, such as weights and/or gradients, from a federated server 350 as a result of a model transfer 338, 342, 346. Each edge device or client 302, 314, 326 keeps neural network weights w.sub.1, . . . , w.sub.k, such as weights and/or gradients, fixed while calculating local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 and local b. During each training iteration s on each edge device or client 302, 314, 326, each edge device or client 302, 314, 326 samples a batch or set x from its local data 304, 316, 328, such as validation or training data, and computes current local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 from local b.sup.s-1 before forwarding x into a local model 306, 318, 330 with weight Σ.sub.k=1.sup.Ka.sub.kw.sub.k. Each edge device or client 302, 314, 326 k computes loss function L(b.sup.s-1, x) and updates local b.sup.s,k based on a computation (softmax) or estimation (Dirichlet distribution) of a gradient. Each edge device or client 302, 314, 326 transfers 340, 344, 348 local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 and local b to a federated server 350. Please also read paragraph [0082, 0091, 0095-0104, and 0115]).
Regarding claim 4, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 1, ROTH fails to explicitly teaches further comprising:
adding the synthetic representation to an existing test data set accessible for the model aggregator device for at least one of validating or testing the machine learned model to generate an extended test data set, and wherein the evaluating evaluates the local update based on the extended test data set.
However, ICKIN explicitly teaches further comprising: adding the synthetic representation to an existing test data set accessible for the model aggregator device for at least one of validating or testing the machine learned model to generate an extended test data set (Fig. 5. Paragraph [0090]-ICKIN discloses the synthetic dataset may then be appended to the original dataset to provide a larger combined training dataset. The larger combined training dataset may be used to train a NN model), and wherein the evaluating evaluates the local update based on the extended test data set (Fig. 5. Paragraph [0094]-ICKIN discloses the master evaluates the quality metric and, in this example, decides that the worker 200-N should proceed with using the synthetic training dataset S′, which it indicates to the worker 200-N in a message 509. In paragraph [0095]-ICKIN discloses the worker 200-N then combines the real training dataset S with the synthetic training dataset S′ at block 510 and trains the ML model using the combined dataset (S, S′) at block 512. In paragraph [0096]-ICKIN discloses the worker 200-N then sends the re-trained weights to the master 100 in a message 513. The master 100 combines the re-trained weights with trained weights from other workers 200 at block 514, and transmits the combined weights to the workers 200 in a message 515. In paragraph [0097]-ICKIN discloses the decision to require synthetic data may depend on the quality of weights provided by a given worker and/or based on other consideration. Please also read paragraph [0005 and 0090-0092]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, the method comprising: receiving, at the model aggregator device, from a first local site that is remote from the model aggregator device, a local update of the machine learned model and a parameterization of local data, wherein the local update was generated at the first local site based on the local data, with the teachings of ICKIN of having further comprising:
adding the synthetic representation to an existing test data set accessible for the model aggregator device for at least one of validating or testing the machine learned model to generate an extended test data set, and wherein the evaluating evaluates the local update based on the extended test data set.
Wherein ROTH’s method having further comprising: adding the synthetic representation to an existing test data set accessible for the model aggregator device for at least one of validating or testing the machine learned model to generate an extended test data set, and wherein the evaluating evaluates the local update based on the extended test data set.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and ICKIN concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while ICKIN’s systems and methods improves model performance in federated learning using synthetic data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and ICKIN et al. (US 20230088561 A1), Abstract and Paragraph [0063].
Regarding claim 5, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to any claim 1, ROTH fails to explicitly teaches further comprising: determining a data quality of the synthetic representation, and wherein the evaluating evaluates the local update based on the data quality.
However, ICKIN explicitly teaches further comprising: determining a data quality of the synthetic representation, and wherein the evaluating evaluates the local update based on the data quality (Fig. 5. Paragraph [0094]-ICKIN discloses the master evaluates the quality metric and, in this example, decides that the worker 200-N should proceed with using the synthetic training dataset S′, which it indicates to the worker 200-N in a message 509. In paragraph [0095]-ICKIN discloses the worker 200-N then combines the real training dataset S with the synthetic training dataset S′ at block 510 and trains the ML model using the combined dataset (S, S′) at block 512. In paragraph [0096]-ICKIN discloses the worker 200-N then sends the re-trained weights to the master 100 in a message 513. The master 100 combines the re-trained weights with trained weights from other workers 200 at block 514, and transmits the combined weights to the workers 200 in a message 515. In paragraph [0097]-ICKIN discloses the decision to require synthetic data may depend on the quality of weights provided by a given worker and/or based on other consideration. Please also read paragraph [0005, 0090-0092 and 0097]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of ICKIN of having further comprising: determining a data quality of the synthetic representation, and wherein the evaluating evaluates the local update based on the data quality.
Wherein ROTH’s method having further comprising: determining a data quality of the synthetic representation, and wherein the evaluating evaluates the local update based on the data quality.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and ICKIN concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while ICKIN’s systems and methods improves model performance in federated learning using synthetic data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and ICKIN et al. (US 20230088561 A1), Abstract and Paragraph [0063].
In paragraph [0065]-ROTH discloses supervision is strong, wherein input information provides direct identification of an object, feature, style, or other aspect of an item, such as an image, in local testing data 106, 114, 122. Supervision is weak, wherein input information provides partial identification of an object, feature, style, or other aspect of local testing data 106, 114, 122. Strong supervision is input information such as bounding boxes, where one or more objects or features are outlined in local testing data 106, 114, 122. In paragraph [0066] In at least one embodiment, weak supervision comprises input information such as points, where individual locations in local testing data 106, 114, 122 are identified as being within an object or objects. In at least one embodiment, weak supervision comprises input information such as lines, where each point in a line within local testing data 106, 114, 122 are identified by said weak supervision as being within an object or objects. In at least one embodiment, weak supervision comprises input information such as tags or labels, where a tag or label identifies that local testing data 106, 114, 122 contains a specific object or objects or is of a specific classification [0088]-ROTH discloses each edge device or client 202, 208, 214 of K edge devices or clients 202, 208, 214 trains its local model 206, 208, 214 using local data 204, 210, 216 in order to minimize
Regarding claim 8, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 1, although ROTH explicitly teaches wherein the local data (Fig. 2-3, #204, #210, #216, #304, #316, and #328 called local data. Paragraph [0062, 0071 and 0088]) includes a plurality of independent data items (Fig. 2-3. Paragraph [0063]-ROTH discloses local testing data 106, 114, 122 is data values comprising images or other data usable for training local models 108, 116, 124. Local testing data 106, 114, 122 comprises supervision from testing services 104, 112, 120. Please also read paragraph [0064-0067]), and the parameterization includes for each independent data item, an item-parametrization of the independent data item (Fig. 2-3. Paragraph [0092]-ROTH discloses at each communication round t, each edge device or client 302, 314, 326 receives a copy of all neural network weights w.sub.1, . . . , w.sub.k, such as weights and/or gradients, from a federated server 350 as a result of a model transfer 338, 342, 346. Each edge device or client 302, 314, 326 keeps neural network weights w.sub.1, . . . , w.sub.k, such as weights and/or gradients, fixed while calculating local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 and local b. During each training iteration s on each edge device or client 302, 314, 326, in an embodiment, each edge device or client 302, 314, 326 samples a batch or set x from its local data 304, 316, 328, such as validation or training data, and computes current local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 from local b.sup.s-1 before forwarding x into a local model 306, 318, 330 with weight Σ.sub.k=1.sup.Ka.sub.kw.sub.k. Please also read paragraph [0087-0091]).
ROTH fails to explicitly teach and one or more statistical properties of the plurality of independent data items.
However, CARETTE explicitly teaches and one or more statistical properties of the plurality of independent data items (Fig. 2. Paragraph [0044]-CARETTE discloses the authentic statistical information 114 may include statistical properties summarizing characteristics or features of interest of the authentic sensor data 110. The authentic statistical information 114 may include a vector of values. Each value may correspond to a statistical parameter, each of which may correspond to one or more statistical parameters of the global generative model 210. Please also read paragraph [0039-0041, 0069, 0071-0075 and 0100-0104]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of CARETTE of having and one or more statistical properties of the plurality of independent data items
Wherein ROTH’s method having wherein the local data includes a plurality of independent data items, and the parameterization includes for each independent data item, an item- parametrization of the independent data item, and one or more statistical properties of the plurality of independent data items.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and CARETTE concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while CARETTE’s systems and methods improves federated learning and the ability to provide synthetic sensor data that accurately reflects the authentic sensor data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and CARETTE et al. (US 20260212206 A1), Abstract and Paragraph [0084].
Regarding claim 9, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 1, ROTH further teaches further comprising:
generating a modified parameterization based on the parameterization (Fig. 2. Paragraph [0096]-ROTH discloses FIG. 5 illustrates pseudocode 502 to perform learnable federated averaging using learnable aggregation weights in a global federated training architecture, according to at least one embodiment. At pseudocode 502 line 504, a server, such as a federated server or any other server further described herein, defines data structures representing initial global aggregation weights a and parameterization values b, as described above in conjunction with FIG. 3. In paragraph [0097]-ROTH discloses at pseudocode 502 lines 514, 516, 518, as a result of a training round t in T satisfying a condition, such as being a multiple of a specific value t.sup.0 or being a i.sup.th iteration or round, a server, such as a federated server or any other server further described herein, learns a global parameterization value b and computes parameterized global aggregation weights a for a round tin T. At pseudocode 502 line 524, a server, such as a federated server updates a global model w for a current training iteration or round t according to neural network data values, such as weights, obtained from K edge devices or client according to current global aggregation weights a for current round t. Please also read paragraph [0088-0095 and 0115]).
ROTH fails to explicitly teach wherein the synthetic representation is generated based on the modified parameterization.
However, CARETTE explicitly teaches wherein the synthetic representation is generated based on the modified parameterization (Fig. 2. Paragraph [0069]-CARETTE discloses the partial generative model 230 may be further personalized for the client device 100A through such an attention-like mechanism to generate the synthetic data 310 with similar statistical properties compared to the authentic sensor data 110. In paragraph [0074]-CARETTE discloses the scoping function 246 may thus ensure that the generated synthetic sensor data 310 has statistical properties that are similar from the authentic sensor data 110. The scoping function 246 may ensure that the synthetic sensor data 310 generated by the trained partial generative model 230T is aligned with the authentic sensor data 110. The scoping function 246 may also filter unnecessary or irrelevant data to ensure an effective and more efficient training. The scoping function 246 may take the form of an embedding configured to receive as input the authentic sensor data 110 and generate as output embedded, or scoped, authentic sensor data 116. Please also read paragraph [0072-0073 and 0096-0104]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of CARETTE of having wherein the synthetic representation is generated based on the modified parameterization.
Wherein ROTH’s method having wherein the synthetic representation is generated based on the modified parameterization.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and CARETTE concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while CARETTE’s systems and methods improves federated learning and the ability to provide synthetic sensor data that accurately reflects the authentic sensor data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and CARETTE et al. (US 20260212206 A1), Abstract and Paragraph [0084].
Regarding claim 10, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 1, ROTH further teaches wherein the local data comprises protected information, and the parameterization does not comprise the protected information (Fig. 1-3. Paragraph [0088]-ROTH discloses when being trained on imaging data from a specific location, training may take place at that location, or at least in a manner that protects confidentiality of imaging data or restricts imaging data from being transferred off-premises (e.g., to comply with HIPAA regulations, privacy regulations, etc.). Please also read paragraph [0512]).
Regarding claim 11, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 1, although ROTH explicitly teaches wherein the machine learned model (Fig. 1-3, #134, #234 and #358 called a global model. Paragraph [0060]. Please also see Fig. 4-5, and read paragraph [0069-0076 and 0093-0097]) is an image processing function configured to generate an image processing result based on image data, the local data includes training image data, the parameterization includes a parameterization of the training image data (Fig. 1-3. Paragraph [0091]-ROTH discloses a federated server 350 comprising a federated learning framework, federated training framework, or other software framework collaboratively trains a global model w 358 using T communication rounds. During each communication round, a federated server 350 transfers 338, 342, 346 or sends out a global model w 358 to all edge devices or clients 302, 314, 326. One or more edge devices or clients 302, 314, 326 finish updating local models 306, 318, 330 in parallel. In at least one embodiment, during communication round t, once one or more edge devices or clients 302, 314, 326 finish updating their local models 306, 318, 330 in parallel, a federated server 350 gathers or transfers 338, 342, 346 neural network weights, such as local model 306, 318, 330 weight or scalar data values, as well as gathers or transfers 340, 344, 348 local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 from each edge device or client 302, 314, 326. Please also see Fig. 4-5, and read paragraph [0069-0076 and 0093-0097]).
ROTH fails to explicitly teaches and the synthetic representation includes synthetic image data generated by the generative AI function based on the parameterization of the training image data.
However, CARETTE explicitly teaches and the synthetic representation includes synthetic image data generated by the generative AI function based on the parameterization of the training image data (Fig. 2. Paragraph [0067]-CARETTE discloses the partial generative model 230 received by the first client device 100A may be trained locally, using the authentic sensor data 110 as additional training data, to become a trained partial generative model 230T. The trained partial generative model 230T may then be used to update the global generative model 210. Once given sufficient training from the authentic sensor data 110, these trained sub-models may be ready to be sent to the server 200 and to be incorporated into the corresponding global generative sub-models 210-2, 210-3, and 210-6. The trained sub-models of the trained partial generative model 230T may also generate synthetic sensor data 310 with statistical properties corresponding to the authentic sensor data 110, which may expand a pool of data available to the local decision model 330 to be trained locally. Such an embodiment may also allow the client device to perform the data exchange and generation of synthetic sensor data 310 more efficiently, since both are achieved by means of the trained partial generative model 230T. Please also read paragraph [0039-0046, 0069, 0072-0080 and 0100-0104]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, the method comprising: receiving, at the model aggregator device, from a first local site that is remote from the model aggregator device, a local update of the machine learned model and a parameterization of local data, wherein the local update was generated at the first local site based on the local data, with the teachings of CARETTE of having generating, at the model aggregator device, a synthetic representation of the local data based on the parameterization using a generative Al function.
Wherein ROTH’s method having generating, at the model aggregator device, a synthetic representation of the local data based on the parameterization using a generative Al function.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and CARETTE concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while CARETTE’s systems and methods improves federated learning and the ability to provide synthetic sensor data that accurately reflects the authentic sensor data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and CARETTE et al. (US 20260212206 A1), Abstract and Paragraph [0084].
Regarding claim 12, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 11, ROTH further teaches wherein the machine learned model (Fig. 1-3, #108, #116, #124 called local models. Paragraph [0057]. Further in paragraph [0067]-ROTH discloses local models 108, 116, 124 perform any other deep learning operation further described herein. Local models 108, 116, 124 are, individually, a 3D U-Net. Local models 108, 116, 124 are individually any type of neural network further described herein to perform one or more neural network operations [0056]-ROTH discloses FIG. 1 is a block diagram illustrating an example architecture for federated learning by one or more neural networks 108, 116, 124, 134 to process medical data. Federated learning or federated training is neural network training using data and/or local neural network models 108, 116, 124 from edge devices or clients 102, 110, 118 at a plurality of locations. A federated server 132 performs federated training or federated learning of a global model 134 by aggregating neural network data values received from edge devices or clients 102, 110, 118, such as hospital computing systems, where each hospital is located at different geographic locations, and training said global model 134 using said neural network data values) is configured to generate the image processing result (Fig. 1-3. Paragraph [0057]-ROTH discloses edge devices or clients 102, 110, 118 are computing systems located at medical, computing, or any other type of facility, where each facility may be located in geographically diverse positions, and comprise at least a computing system containing hardware and memory with software instructions that, when executed, train one or more local models 108, 116, 124, such as neural networks, to perform image segmentation, classification, or any other deep learning operation further described herein. Local models 108, 116, 124 are data values and software instructions that, when executed, perform deep learning operations, such as image segmentation or classification, to assist in or facilitate medical diagnosis. Local models 108, 116, 124 perform any other deep learning operation further described herein to facilitate any other deep learning application) based on medical image data (Fig. 1, #106, #114, and #122 called local testing data. Paragraph [0063]-ROTH discloses local testing data 106, 114, 122 is data values comprising images or other data usable for training local models 108, 116, 124. Local testing data 106, 114, 122 comprises supervision from testing services 104, 112, 120. In paragraph [0065]-ROTH discloses supervision is strong, wherein input information provides direct identification of an object, feature, style, or other aspect of an item, such as an image, in local testing data 106, 114, 122. Supervision is weak, wherein input information provides partial identification of an object, feature, style, or other aspect of local testing data 106, 114, 122. Strong supervision is input information such as bounding boxes, where one or more objects or features are outlined in local testing data 106, 114, 122. Please also read paragraph [0066]), the image processing result being selected from a detection result of a medical finding in the medical image data, a classification of a medical finding in the medical image data, or a segmentation of the medical image data (Fig. 1-3. Paragraph [0067]-ROTH discloses local models 108, 116, 124 perform image segmentation to facilitate medical diagnosis or any other application of image segmentation. Local models 108, 116, 124 perform other image operations, such as classification or object identification to facilitate medical diagnosis or any other image-based application. Local models 108, 116, 124 perform any other deep learning operation further described herein), and the training image data includes medical image data (Fig. 2-3, #204, #210, #216, #304, #316, and #328 called local data. Paragraph [0071 and 0088]. In paragraph [0071]-ROTH discloses local data 204, 210, 216 is local testing data as described above in conjunction with FIG. 1. Local data 204, 210, 216 is training data usable to train one or more local models 206, 212, 218 such that a training framework updates one or more neural network weight values of said one or more local models 206, 212, 218. Please also read paragraph [0063]).
Regarding claim 13, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 1, ROTH further teaches further comprising:
providing the updated machine learned model to a second local site that is different from the first local site (Fig. 1-3, #204, #210, #216, #304, #316, and #328 called local data. Paragraph [0071 and 0088]. In paragraph [0091]-ROTH discloses a federated server 350 comprising a federated learning framework, federated training framework, or other software framework collaboratively trains a global model w 358 using T communication rounds. During each communication round, a federated server 350 transfers 338, 342, 346 or sends out a global model w 358 to all edge devices or clients 302, 314, 326. One or more edge devices or clients 302, 314, 326 finish updating local models 306, 318, 330 in parallel. During communication round t, once one or more edge devices or clients 302, 314, 326 finish updating their local models 306, 318, 330 in parallel, a federated server 350 gathers or transfers 338, 342, 346 neural network weights, such as local model 306, 318, 330 weight or scalar data values, as well as gathers or transfers 340, 344, 348 local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 from each edge device or client 302, 314, 326. Please also see Fig. 4-5, and read paragraph [0057, 0063, 0069-0076]).
Regarding claim 14, ROTH explicitly teaches a model aggregator device (Fig. 1-3, #132, #226 and #350 called a federated server. Paragraph [0060]) for federated learning (Fig. 1-3. Paragraph [0056]-ROTH discloses FIG. 1 is a block diagram illustrating an example architecture for federated learning by one or more neural networks 108, 116, 124, 134 to process medical data) of a machine learned model (Fig. 1-3, #134, #234 and #358 called a global model. Paragraph [0060]. Please also read paragraph [0056-0057]), the model aggregator device (Fig. 1-3. Paragraph [0056]-ROTH discloses a federated server 132 is a computing system comprising hardware components and memory containing software instructions that, when executed, train a global model 134 according to neural network data values collected from one or more edge devices or clients 102, 110, 118. A global model 134 is data values and software instructions that, when executed, perform image segmentation, classification, or any other neural network operations. Federated learning or federated training is neural network training using data and/or local neural network models 108, 116, 124 from edge devices or clients 102, 110, 118 at a plurality of locations. A federated server 132 performs federated training or federated learning of a global model 134 by aggregating neural network data values received from edge devices or clients 102, 110, 118. Please also see Fig. 4 and read paragraph [0522-0523]) comprising:
an interface unit (Fig. 1-3, #126, #128, #130 called a communication medium. Paragraph [0058]-ROTH discloses one or more edge devices or clients 102, 110, 118 at different locations transmit one or more neural network weights over any communication medium 126, 128, 130, such as Internet or other computing communication networks. In at least one embodiment, a communication medium 126, 128, 130 is any electronic communication infrastructure further described herein, such as Ethernet or any network infrastructure to facilitate Internet communications. A communication medium 126, 128, 130 is secured using one or more cryptographic communication protocols) configured to receive, from a first local site (Fig. 1-3, #102, #110, #118, #202, #208, #216, #302, #314, #326 called edge devices or clients. Paragraph [0067]. In paragraph [0057]-ROTH discloses edge devices or clients 102, 110, 118 are computing systems located at medical, computing, or any other type of facility, where each facility may be located in geographically diverse positions, and comprise at least a computing system containing hardware and memory with software instructions that, when executed, train one or more local models 108, 116, 124, such as neural networks, to perform image segmentation, classification, or any other deep learning operation further described herein. Local models 108, 116, 124 are data values and software instructions that, when executed, perform deep learning operations, such as image segmentation or classification, to assist in or facilitate medical diagnosis) that is remote from the model aggregator device (Fig. 1-3, #132, #226 and #350 called a federated server. Paragraph [0060]. In paragraph [0062]-ROTH discloses one or more edge devices or client 102, 110, 118 computing systems are distributed globally or located at separate locations having separate local testing data 106, 114, 122. One or more edge devices or client 102, 110, 118 computing systems are located in different countries, cities, or other locales such as separate buildings or rooms in a building. One or more edge devices or client 102, 110, 118 facilities, such as a hospital, comprise testing services 104, 112, 120), a local update of the machine learned model (Fig. 1-3, #134, #234 and #358 called a global model. Paragraph [0060]-ROTH discloses FIG. 1 is a block diagram illustrating an example architecture for federated learning by one or more neural. Please also read paragraph [0056-0057]) and a parameterization (Fig. 1-3, #308, #310, #312, #320, #322, #324, #332, #334, #336, #352, #354, #356 called parameterized aggregation weights. Paragraph [0071, 0080 and 0088]. In paragraph [0080]-ROTH discloses each learnable aggregation weights a.sub.k 352, 354, 356 corresponds to a single client 302, 314, 328. Each learnable aggregation weights a.sub.k 352, 354, 356 corresponds to one or more groupings of edge devices or clients 302, 314, 328. Please also read paragraph [0068-0076 and 0115]) of local data (Fig. 1-3, #204, #210, #216, #304, #316, and #328 called local data. Paragraph [0071 and 0088]. In paragraph [0057]-ROTH discloses testing services 104, 112, 120 generate local testing data 106, 114, 122 usable for training one or more local models 108, 116, 124 by a client training framework implementing a training algorithm such as described below in conjunction with FIG. 5 (wherein local testing data corresponds to local data used to train local models). In paragraph [0091]-ROTH discloses during communication round t, once one or more edge devices or clients 302, 314, 326 finish updating their local models 306, 318, 330 in parallel, a federated server 350 gathers or transfers 338, 342, 346 neural network weights, such as local model 306, 318, 330 weight or scalar data values, as well as gathers or transfers 340, 344, 348 local aggregation weights a 308, 310, 312, 320, 322, 324, 332, 334, 336 from each edge device or client 302, 314, 326. Please also see Fig. 4-5, and read paragraph [0057, 0063, 0069-0076]), wherein the local update was generated at the first local site (Fig. 1-3, #102, #110, #118, #202, #208, #216, #302, #314, #326 called edge devices or clients. Paragraph [0067]) based on the local data (Fig. 1-3, #204, #210, #216, #304, #316, and #328 called local data. Paragraph [0071 and 0088]. In paragraph [0091]-ROTH discloses a federated server 350 comprising a federated learning framework, federated training framework, or other software framework collaboratively trains a global model w 358 using T communication rounds. During each communication round, a federated server 350 transfers 338, 342, 346 or sends out a global model w 358 to all edge devices or clients 302, 314, 326. One or more edge devices or clients 302, 314, 326 finish updating local models 306, 318, 330 in parallel. Please also see Fig. 4-5, and read paragraph [0057, 0063, 0069-0076]); and
ROTH fails to explicitly teach a computing unit configured to generate a synthetic representation of the local data based on the parameterization using a generative AT function,
However, CARETTE explicitly teaches a computing unit (Fig. 2, #200 called a server circuitry. Paragraph [0037]-CARETTE discloses the server 200 may be any piece of computer hardware or software that provides functionality for other programs or devices, including the client device 100A. The server 200 is communicatively connectable to one or more client devices. The server 200 may serve multiple other users, for instance user B and user C, and corresponding client devices 100B and 100C, etc. The server comprises a server memory 202. The server memory 202 may be configured to store data and resources related to data, such as machine learning models. The server 200 comprises a server circuitry 204. The server circuitry 204 may provide various functionalities, such as sharing data or resources from multiple clients with other multiple clients, or performing computation for multiple clients, including the client device 100A. Please also read paragraph [0134]) configured to generate a synthetic representation (Fig. 2. Paragraph [0067]-CARETTE discloses the trained sub-models of the trained partial generative model 230T may also generate synthetic sensor data 310 with statistical properties corresponding to the authentic sensor data 110, which may expand a pool of data available to the local decision model 330 to be trained locally. Since both the authentic sensor data 110 and synthetic sensor data 310 may remain restricted to the client device 100A, a high standard of privacy may be maintained) of the local data (Fig. 2. Paragraph [0034]-CARETTE discloses the authentic sensor data 110 may be any data obtained from a measurement by one or more sensors connected to the first client device 100A. Types of sensors that may record physical data may include image sensors. Such sensors may record data related to physical phenomena that may relate to biometric or behavioral data of the user A (wherein authentic data and synthetic data is used as training data). Please also read paragraph [0036, 0047, 0067, and 0096]) based on the parameterization using a generative AI function (Fig. 2. Paragraph [0044]-CARETTE discloses the client device 100A comprises a respective client device circuitry 104 configured to receive the model information 240 from the server 200 (wherein model information 240 includes authentic statistical information 114 (e.g. a vector of values and metadata that describes the specific data context/type and statistical properties of characteristics or features of interest of authentic sensor data 110 corresponding to a statistical parameters), instructions for calculations, such as a profiling function and scoping function, etc.). Based on the received model information 240, the client device circuitry 104 is configured to determine the user-specific authentic statistical information 114 based on the user-specific authentic sensor data 110 and send the user-specific authentic statistical information 114 to the server 200. Further in paragraph [0072]-CARETTE discloses the profiling function 242 may comprise a set of statistical parameters to have one or more corresponding values calculated to obtain a set of statistical properties. The set of statistical parameters may be learned within the server 200 based on updates from one or more users of the collaborative learning system 30 (wherein the profiling function 242 may output authentic sensor data 110 in a more organized form and compatible form where data has been given meaning and context). Further in paragraph [0074]-CARETTE discloses the scoping function 246 may take the form of an embedding configured to receive as input the authentic sensor data 110 and generate as output embedded, or scoped, authentic sensor data 116 (wherein scoping function 246 may also filter unnecessary information). Please also read paragraph [0039-0046, 0069, 0076-0080 and 0100-0104]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of CARETTE of having generating, at the model aggregator device, a synthetic representation of the local data based on the parameterization using a generative Al function.
Wherein ROTH’s method having generating, at the model aggregator device, a synthetic representation of the local data based on the parameterization using a generative Al function.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and CARETTE concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while CARETTE’s systems and methods improves federated learning and the ability to provide synthetic sensor data that accurately reflects the authentic sensor data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and CARETTE et al. (US 20260212206 A1), Abstract and Paragraph [0084].
Although ROTH explicitly teaches and update the machine learned model based on the local update (Fig. 3. Paragraph [0097]-ROTH discloses as a result of a training round t in T satisfying a condition, such as being a multiple of a specific value t.sup.0 or being a i.sup.th iteration or round, a server, such as a federated server or any other server further described herein, learns a global parameterization value b and computes parameterized global aggregation weights a for a round tin T, as described above in conjunction with FIGS. 3 and 4. As a result of a training round t in T not satisfying a condition, such as being a multiple of a specific value t.sup.0 or being a t.sup.th iteration or round, a server, such as a federated server or any other server further described herein, sets current global aggregation weights a for a current global training iteration or round t to previous global aggregation weights a from a previous global training iteration or round t−1. A server, such as a federated server or any other server, updates a global model w for a current training iteration or round t according to neural network data values, such as weights, obtained from K edge devices or client according to current global aggregation weights a for current round t. Please also read paragraph [0082, 0098, and 0115]).
ROTH fails to explicitly teach evaluate the local update using the synthetic representation to obtain an evaluation result indicative of performance of the local update, and update the machine learned model based on the evaluation result.
However, ICKIN explicitly teaches evaluate the local update using the synthetic representation to obtain an evaluation result indicative of performance of the local update, and update the machine learned model based on the evaluation result (Fig. 5. Paragraph [0094]-ICKIN discloses at block 506, the worker 200-N generates a synthetic training dataset S′ using. The worker 200-N generates a quality metric for the synthetic dataset, such as the KL divergence of synthetic training dataset, and transmits the quality metric to the master 100 in a message 507. The master evaluates the quality metric and decides that the worker 200-N should proceed with using the synthetic training dataset S′, which it indicates to the worker 200-N in a message 509. In paragraph [0095]-ICKIN discloses the worker 200-N then combines the real training dataset S with the synthetic training dataset S′ at block 510 and trains the ML model using the combined dataset (S, S′) at block 512. In paragraph [0096]-ICKIN discloses the worker 200-N then sends the re-trained weights to the master 100 in a message 513. The master 100 combines the re-trained weights with trained weights from other workers 200 at block 514, and transmits the combined weights to the workers 200 in a message 515. Further in paragraph [0090]-ICKIN discloses the larger combined training dataset may be used to train a NN model. In paragraph [0091]-ICKIN discloses the performance of the trained model may be evaluated on the real test set. Please also read paragraph [0097]).
Regarding claim 15, ROTH explicitly teaches a non-transitory computer program product comprising program elements that induce a computing unit of a model aggregator device (Fig. 1-3, #132, #226 and #350 called a federated server. Paragraph [0060]) for federated learning (Fig. 1-3. Paragraph [0056]-ROTH discloses FIG. 1 is a block diagram illustrating an example architecture for federated learning by one or more neural networks 108, 116, 124, 134 to process medical data) of a machine learned model (Fig. 1-3, #134, #234 and #358 called a global model. Paragraph [0060]. Please also read paragraph [0056-0057]) to perform the method (Fig. 1-3. Paragraph [0056]-ROTH discloses a federated server 132 is a computing system comprising hardware components and memory containing software instructions that, when executed, train a global model 134 according to neural network data values collected from one or more edge devices or clients 102, 110, 118. A global model 134 is data values and software instructions that, when executed, perform image segmentation, classification, or any other neural network operations. Federated learning or federated training is neural network training using data and/or local neural network models 108, 116, 124 from edge devices or clients 102, 110, 118 at a plurality of locations. A federated server 132 performs federated training or federated learning of a global model 134 by aggregating neural network data values received from edge devices or clients 102, 110, 118. Please also see Fig. 7A-B, and read paragraph [0522-0523]) when the program elements are loaded into a memory of the computing unit (Fig. 1-3. Paragraph [0585]-ROTH discloses code is stored on a computer-readable storage medium, for example, in form of a computer program comprising a plurality of instructions executable by one or more processors. A computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers of transitory signals. Code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein. Please also see Fig. 7A-B and read paragraph [0056-0060 and 0522-0523] (wherein machine learning models may, for example, include generative adversarial models and synthetically generate data, such as ground truth data))
Although ROTH explicitly teaches a non-transitory computer program product comprising program elements that induce a computing unit of a model aggregator device for federated learning of a machine learned model to perform the method when the program elements are loaded into a memory of the computing unit.
ROTH fails to explicitly teach the method according to claim 1
However, ROTH in view of CARETTE and in further view of ICKIN explicitly teach the method according to claim 1 (Please see the rejection for claim 1 further above).
Regarding claim 16, ROTH explicitly teaches a non-transitory computer-readable medium on which program elements are stored, the program elements being readable and executable by a computing unit of a model aggregator device (Fig. 1-3, #132, #226 and #350 called a federated server. Paragraph [0060]) for federated learning (Fig. 1-3. Paragraph [0056]-ROTH discloses FIG. 1 is a block diagram illustrating an example architecture for federated learning by one or more neural networks 108, 116, 124, 134 to process medical data) of a machine learned model (Fig. 1-3, #134, #234 and #358 called a global model. Paragraph [0060]. Please also read paragraph [0056-0057]) to cause the model aggregator device to perform the method (Fig. 1-3. Paragraph [0056]-ROTH discloses a federated server 132 is a computing system comprising hardware components and memory containing software instructions that, when executed, train a global model 134 according to neural network data values collected from one or more edge devices or clients 102, 110, 118. A global model 134 is data values and software instructions that, when executed, perform image segmentation, classification, or any other neural network operations. Federated learning or federated training is neural network training using data and/or local neural network models 108, 116, 124 from edge devices or clients 102, 110, 118 at a plurality of locations. A federated server 132 performs federated training or federated learning of a global model 134 by aggregating neural network data values received from edge devices or clients 102, 110, 118. Please also see Fig. 7A-B, and read paragraph [0522-0523]) when the program elements are executed by the computing unit (Fig. 1-3. Paragraph [0585]-ROTH discloses code is stored on a computer-readable storage medium, for example, in form of a computer program comprising a plurality of instructions executable by one or more processors. A computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers of transitory signals. Code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein. Please also see Fig. 7A-B and read paragraph [0056-0060 and 0522-0523] (wherein machine learning models may, for example, include generative adversarial models and synthetically generate data, such as ground truth data)).
Although ROTH explicitly teaches a non-transitory computer-readable medium on which program elements are stored, the program elements being readable and executable by a computing unit of a model aggregator device for federated learning of a machine learned model to cause the model aggregator device to perform the method when the program elements are executed by the computing unit.
ROTH fails to explicitly teach the method according to claim 1.
However, ROTH in view of CARETTE and in further view of ICKIN explicitly teach the method according to claim 1 (Please see the rejection for claim 1 further above).
Regarding claim 17, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 10, ROTH further teaches wherein the protected information is protected personal information (Fig. 1-3. Paragraph [0088]-ROTH discloses when being trained on imaging data from a specific location, training may take place at that location, or at least in a manner that protects confidentiality of imaging data or restricts imaging data from being transferred off-premises (e.g., to comply with HIPAA regulations, privacy regulations, etc.). Please also read paragraph [0512]).
Claims 3 are rejected under 35 U.S.C. 103 as being unpatentable over ROTH et al. (US 20220366220 A1), hereinafter referenced as ROTH in view of CARETTE et al. (US 20260212206 A1), hereinafter referenced as CARETTE and in further view of ICKIN et al. (US 20230088561 A1), hereinafter referenced as ICKIN and in further view of AZARIAN et al. (US 20250166236 A1), hereinafter referenced as AZARIAN.
Regarding claim 3, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 1, ROTH in view of CARETTE and in further view of ICKIN fails to explicitly teaches wherein the generative Al function is configured to generate the synthetic representation based on a natural language prompt indicating the synthetic representation to be generated, and the generating includes obtaining the natural language prompt based on the parameterization, and inputting the natural language prompt into the generative Al function to generate the synthetic representation.
However, AZARIAN explicitly teaches wherein the generative Al function (Fig. 2. Paragraph [0035]-AZARIAN discloses FIG. 2 depicts additional details of an example image generation system 200 that implements diffusion processing. One or more components of the image generation system 200 may be implemented or run on one or more processors, which may be coupled to one or more memories. One or more processors may be configured to perform one or more functions of one or more components of the image generation system 200, such as by executing instructions stored in one or more memories, or without requiring instructions from memory (wherein the diffusion processor 212 comprises an encoder-decoder neural network architecture and the image generation system 200 may comprise one or more neural networks including transformers, artificial neural networks (ANN), generative adversarial models). Further in paragraph [0077]-AZARIAN discloses training and/or inference may be distributed amongst devices in a decentralized or federated fashion) is configured to generate the synthetic representation (Fig. 2. Paragraph [0036]-AZARIAN discloses the image generation system 200, which may be the same as image generation system 100, generates an output image 202 based on one or more conditionings 204) based on a natural language prompt indicating the synthetic representation to be generated (Fig. 2. Paragraph [0036]-AZARIAN discloses the one or more conditionings 204 may include a text prompt 206 that is the same as or similar to the text prompt 104 of FIG. 1. The text prompt 206 can be provided to a text encoder 208 configured to encode the text prompt 206 into a text embedding 210. The text encoder 208 encodes a text prompt 206 into a latent feature representation that captures the semantic meaning of the text (wherein text encoder 208 comprising a transformer, such as BERT, CLIP or another language model). Further in paragraph [0037]-AZARIAN discloses the text embedding 210 is then provided to a diffusion processor 212. The diffusion processor 212 comprises an encoder-decoder neural network architecture. Cross-attention aligns the text semantics of text embedding 210 with spatial image regions throughout the generation process performed by the diffusion processor 212. Please also read paragraph [0032]), and the generating includes obtaining the natural language prompt based on the parameterization, and inputting the natural language prompt into the generative Al function to generate the synthetic representation (Fig. 2. Paragraph [0036]-AZARIAN discloses the text prompt 206 can be provided to a text encoder 208 configured to encode the text prompt 206 into a text embedding 210. The text encoder 208 encodes a text prompt 206 into a latent feature representation that captures the semantic meaning of the text (wherein text encoder 208 comprising a transformer, such as BERT, CLIP or another language model). Further in paragraph [0037]-AZARIAN discloses the text embedding 210 is then provided to a diffusion processor 212. The diffusion processor 212 comprises an encoder-decoder neural network architecture. Cross-attention aligns the text semantics of text embedding 210 with spatial image regions throughout the generation process performed by the diffusion processor 212. In paragraph [0040]-AZARIAN discloses one or more conditioning parameters 304 can be utilized to tailor, refine, or direct an image generation process. Please also read paragraph [0048-0054 and 0061-0067]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further review of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of AZARIAN of having wherein the generative Al function is configured to generate the synthetic representation based on a natural language prompt indicating the synthetic representation to be generated, and the generating includes obtaining the natural language prompt based on the parameterization, and inputting the natural language prompt into the generative Al function to generate the synthetic representation.
Wherein ROTH’s method having wherein the generative Al function is configured to generate the synthetic representation based on a natural language prompt indicating the synthetic representation to be generated, and the generating includes obtaining the natural language prompt based on the parameterization, and inputting the natural language prompt into the generative Al function to generate the synthetic representation.
The motivation behind the modification would have been to obtain a method that improves the training and performance of learning systems using federated learning, since both ROTH and AZARIAN concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while AZARIAN’s systems and methods improves overall image output using synthesis neural networks and text prompts. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and AZARIAN et al. (US 20250166236 A1), Abstract and Paragraph [0028-0029].
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over ROTH et al. (US 20220366220 A1), hereinafter referenced as ROTH in view of CARETTE et al. (US 20260212206 A1), hereinafter referenced as CARETTE and in further view of ICKIN et al. (US 20230088561 A1), hereinafter referenced as ICKIN and in further view of FISHER et al. (US 20230126877 A1), hereinafter referenced as FISHER.
Regarding claim 6, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 5, although ROTH explicitly teaches generating a backward parameterization of the representation (Fig. 5. Paragraph [0108]-ROTH discloses code and/or data storage 705 stores weight parameters and/or input/output data of each layer of a neural network trained or used in conjunction with one or more embodiments during backward propagation of input/output data and/or weight parameters during training and/or inferencing);
comparing the backward parameterization with the parameterization (Fig. 5. Paragraph [0088]-ROTH discloses during a training phase, given a local data 304, 316, 328 sample x, a federated server 350 and/or one or more edge devices or clients 302, 314, 326 comprising a federated learning framework, federated training framework, or other software framework to train one or more local models 306, 318, 330 and/or a global model w 358 samples learnable aggregation weights a 352, 354, 356 from a Dirichlet distribution with concentration b as described above, approximates a gradient of b given a loss function L(b, x), using implicit re-parameterization or any other method of parameterization, and updates a concentration of b. During a training phase, given a local data 304, 316, 328 sample x, a federated server 350 and/or one or more edge devices or clients 302, 314, 326 comprising a federated learning framework, federated training framework, or other software framework to train one or more local models 306, 318, 330 and/or a global model w 358 learns parameterized aggregation weights 308, 310, 312, 320, 322, 324, 332, 334, 336, 352, 354, 356 in a differentiable way from data. Please also read paragraph [0089-0097, 0106 and 0115]).
ROTH fails to explicitly teaches wherein the determining a data quality of the synthetic representation comprises: generating a parameterization of the synthetic representation.
However, ICKIN explicitly teaches wherein the determining a data quality of the synthetic representation comprises:
generating a parameterization of the synthetic representation (Fig. 5. Paragraph [0093]-ICKIN discloses FIG. 5 illustrates an example exchange between a master 100 and a worker 200-N in a federated learning system. A worker 200-N may train a set of weights for a local ML model, such as a neural network, using a real training dataset S (block 502). The worker 200-N sends the trained weights to the master 100 in a message 503. The master 100 evaluates the trained weights at block 504, and in this example decides that the worker 200-N should use synthetic data for training, for example, to reduce a number of training iterations needed to stay below a threshold of training iterations. The master 100 indicates this to the worker 200-N vie a message 505 instructing the worker 200-N to generate synthetic training data. Further in paragraph [0094]-ICKIN discloses the master evaluates the quality metric and, in this example, decides that the worker 200-N should proceed with using the synthetic training dataset S′, which it indicates to the worker 200-N in a message 509. In paragraph [0095]-ICKIN discloses the worker 200-N then combines the real training dataset S with the synthetic training dataset S′ at block 510 and trains the ML model using the combined dataset (S, S′) at block 512. In paragraph [0096]-ICKIN discloses the worker 200-N then sends the re-trained weights to the master 100 in a message 513. The master 100 combines the re-trained weights with trained weights from other workers 200 at block 514, and transmits the combined weights to the workers 200 in a message 515. Please also see claim 28 and read paragraph [0005, 0090-0092 and 0097]);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of ICKIN of having wherein the determining a data quality of the synthetic representation comprises: generating a parameterization of the synthetic representation.
Wherein ROTH’s method having wherein the determining a data quality of the synthetic representation comprises: generating a backward parameterization of the synthetic representation; comparing the backward parameterization with the parameterization.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and ICKIN concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while ICKIN’s systems and methods improves model performance in federated learning using synthetic data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and ICKIN et al. (US 20230088561 A1), Abstract and Paragraph [0063].
ROTH in view of CARETTE and in further view of ICKIN fail to explicitly teach determining the data quality based on the comparing the backward parameterization with the parameterization.
However, FISHER explicitly teaches determining the data quality (Fig. 1. Paragraph [0050]-FISHER discloses the image adaptation model 106 is a machine learning model trained to convert model output images to synthetic scan images (wherein image adaptation model 106 is a generative adversarial network (GAN) and/or a diffusion network model using convolutional neural networks). In paragraph [0052]-FISHER discloses predictor model 108 is a machine learning model trained to identify a region of generated synthetic scan images predicted to show/have cancer (wherein predictor model 108 may comprise one or more similar neural network(s) such as a an encoder-decoder and/or the like). In paragraph [0065]-FISHER discloses at block 222, operations 204 proceed with receiving output from the first machine learning model (e.g., image adaptation model 106 illustrated in FIG. 1A) in response to the plurality features. The inference may be a predicted synthesized scan image meant to represent the first state of the simulated tumor. In paragraph [0066]-FISHER discloses at block 224, operations 204 proceed with evaluating a similarity of the output to the input scan image selected at block 212. Evaluating the similarity of the output to the input scan image is performed using a loss function) based on the comparing the backward parameterization with the parameterization (Fig. 1. Paragraph [0067]-FISCHER discloses at block 226, operations 204 proceed with determining whether to modify one or more parameters of the first machine learning model based on the similarity determined at block 224. The model training engine trains the first machine learning model by iteratively modifying parameters (e.g., through backpropagation) of the first machine learning model until its output in response to the training input matches the training output. The output may be compared to the training output (e.g., at block 224), and various parameters such as weights, connections, number of hidden nodes, weight decay, activation sparsity, nonlinearity, weight initialization, random seeds, model averaging, preprocessing input data, coordinate descent, grid search, random search, and model-based optimization methods, may be modified such until the output matches the training output).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of FISHER of having determining the data quality based on the comparing the backward parameterization with the parameterization.
Wherein ROTH’s method having determining the data quality based on the comparing the backward parameterization with the parameterization.
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and FISHER concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while ICKIN’s systems and methods improves federated learning and synthetic data generation. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and FISHER et al. (US 20230126877 A1), Abstract and Paragraph [0027 and 0115].
Claim 7 are rejected under 35 U.S.C. 103 as being unpatentable over ROTH et al. (US 20220366220 A1), hereinafter referenced as ROTH in view of CARETTE et al. (US 20260212206 A1), hereinafter referenced as CARETTE and in further view of ICKIN et al. (US 20230088561 A1), hereinafter referenced as ICKIN and in further view of ARGAWAL et al. (US 20240428468 A1), hereinafter referenced as ARGAWAL.
Regarding claim 7, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 5, ROTH in view of CARETTE and in further view of ICKIN fails to explicitly teaches wherein the determining a data quality of the synthetic representation comprises: generating a natural language summary based on the synthetic representation; comparing the natural language summary to a natural language prompt indicative of the synthetic representation; and determining the data quality based on the comparing the natural language summary to the natural language prompt wherein the determining a data quality of the synthetic representation.
However, AGARWAL explicitly teaches wherein the determining a data quality of the synthetic representation (Fig. 2. Paragraph [0047]-AGARWAL discloses FIG. 2 shows the attention image synthesis system 102 via the diffusion neural network 202 utilizing at least one of the attention retention loss 202a and the attention segregation loss 202b to generate the text-conditioned image 204 (wherein attention image synthesis system 102 may be implemented in a distributed computing environment). In paragraph [0042]-AGARWAL discloses the attention image synthesis system 102 utilizes machine learning to process the text query 200. In paragraph [0043]-AGARWAL discloses a neural network includes a convolutional neural network, a recurrent neural network (e.g., a long short-term memory neural network), a transformer neural network, a generative adversarial neural network, a graph neural network, a diffusion neural network, or a multi-layer perceptron) comprises:
generating a natural language summary based on the synthetic representation (Fig. 1. Paragraph [0104]-AGARWAL discloses experimenters generated captions for generated images with BLIP);
comparing the natural language summary to a natural language prompt indicative of the synthetic representation (Fig. 2. Paragraph [0104]-AGARWAL discloses experimenters compares the BLIP-generated captions with the text prompts (e.g., a ground truth)); and
determining the data quality based on the comparing the natural language summary to the natural language prompt (Fig. 2. Paragraph [0104]-AGARWAL discloses FIG. 10 shows a comparison between an example implementation of the attention image synthesis system 102 and prior methods. For each animal-animal, animal-object, and object-object, the attention image synthesis system 102 (indicated as A-STAR) shows improvement over prior methods in terms of text-text similarities. Namely, the attention image synthesis system 102 (A-STAR) shows a text-text similarity of 0.82, 0.84, and 0.82 which is at least 0.01 greater than the best results for prior methods).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of ARGAWAL of having wherein the determining a data quality of the synthetic representation comprises: generating a natural language summary based on the synthetic representation; comparing the natural language summary to a natural language prompt indicative of the synthetic representation; and determining the data quality based on the comparing the natural language summary to the natural language prompt wherein the determining a data quality of the synthetic representation.
Wherein ROTH’s method having wherein the determining a data quality of the synthetic representation comprises: generating a natural language summary based on the synthetic representation; comparing the natural language summary to a natural language prompt indicative of the synthetic representation; and determining the data quality based on the comparing the natural language summary to the natural language prompt wherein the determining a data quality of the synthetic representation.
The motivation behind the modification would have been to obtain a method that improves the training and performance of neural networks, since both ROTH and ARGAWAL concern image analysis and neural networks. Wherein ROTH’s systems and methods improve federated learning for neural networks, while ARGAWAL’s systems and methods improves operational flexibility and accuracy of an image synthesis system. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and ARGAWAL et al. (US 20230126877 A1), Abstract and Paragraph [0029].
Claim 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over ROTH et al. (US 20220366220 A1), hereinafter referenced as ROTH in view of CARETTE et al. (US 20260212206 A1), hereinafter referenced as CARETTE and in further view of ICKIN et al. (US 20230088561 A1), hereinafter referenced as ICKIN and in further view of AZARIAN et al. (US 20250166236 A1), hereinafter referenced as AZARIAN.
Regarding claim 18, ROTH in view of CARETTE and in further view of ICKIN explicitly teaches the method according to claim 2, ROTH in view of CARETTE and in further view of ICKIN fails to explicitly teaches wherein the generative AI function is configured to generate the synthetic representation based on a natural language prompt indicating the synthetic representation to be generated, and the generating includes obtaining the natural language prompt based on the parameterization, and inputting the natural language prompt into the generative Al function to generate the synthetic representation.
However, AZARIAN explicitly teaches wherein the generative AI function (Fig. 2. Paragraph [0035]-AZARIAN discloses FIG. 2 depicts additional details of an example image generation system 200 that implements diffusion processing. One or more components of the image generation system 200 may be implemented or run on one or more processors, which may be coupled to one or more memories. One or more processors may be configured to perform one or more functions of one or more components of the image generation system 200, such as by executing instructions stored in one or more memories, or without requiring instructions from memory (wherein the diffusion processor 212 comprises an encoder-decoder neural network architecture and the image generation system 200 may comprise one or more neural networks including transformers, artificial neural networks (ANN), generative adversarial models). Further in paragraph [0077]-AZARIAN discloses training and/or inference may be distributed amongst devices in a decentralized or federated fashion) is configured to generate the synthetic representation (Fig. 2. Paragraph [0036]-AZARIAN discloses the image generation system 200, which may be the same as image generation system 100, generates an output image 202 based on one or more conditionings 204) based on a natural language prompt indicating the synthetic representation to be generated (Fig. 2. Paragraph [0036]-AZARIAN discloses the one or more conditionings 204 may include a text prompt 206 that is the same as or similar to the text prompt 104 of FIG. 1. The text prompt 206 can be provided to a text encoder 208 configured to encode the text prompt 206 into a text embedding 210. The text encoder 208 encodes a text prompt 206 into a latent feature representation that captures the semantic meaning of the text (wherein text encoder 208 comprising a transformer, such as BERT, CLIP or another language model). Further in paragraph [0037]-AZARIAN discloses the text embedding 210 is then provided to a diffusion processor 212. The diffusion processor 212 comprises an encoder-decoder neural network architecture. Cross-attention aligns the text semantics of text embedding 210 with spatial image regions throughout the generation process performed by the diffusion processor 212. Please also read paragraph [0032]), and the generating includes obtaining the natural language prompt based on the parameterization, and inputting the natural language prompt into the generative Al function to generate the synthetic representation (Fig. 2. Paragraph [0036]-AZARIAN discloses the text prompt 206 can be provided to a text encoder 208 configured to encode the text prompt 206 into a text embedding 210. The text encoder 208 encodes a text prompt 206 into a latent feature representation that captures the semantic meaning of the text (wherein text encoder 208 comprising a transformer, such as BERT, CLIP or another language model). Further in paragraph [0037]-AZARIAN discloses the text embedding 210 is then provided to a diffusion processor 212. The diffusion processor 212 comprises an encoder-decoder neural network architecture. Cross-attention aligns the text semantics of text embedding 210 with spatial image regions throughout the generation process performed by the diffusion processor 212. In paragraph [0040]-AZARIAN discloses one or more conditioning parameters 304 can be utilized to tailor, refine, or direct an image generation process. Please also read paragraph [0048-0054 and 0061-0067]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further review of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of AZARIAN of having wherein the generative AI function is configured to generate the synthetic representation based on a natural language prompt indicating the synthetic representation to be generated, and the generating includes obtaining the natural language prompt based on the parameterization, and inputting the natural language prompt into the generative Al function to generate the synthetic representation.
Wherein ROTH’s method having wherein the generative AI function is configured to generate the synthetic representation based on a natural language prompt indicating the synthetic representation to be generated, and the generating includes obtaining the natural language prompt based on the parameterization, and inputting the natural language prompt into the generative Al function to generate the synthetic representation.
The motivation behind the modification would have been to obtain a method that improves the training and performance of learning systems using federated learning, since both ROTH and AZARIAN concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while AZARIAN’s systems and methods improves overall image output using synthesis neural networks and text prompts. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and AZARIAN et al. (US 20250166236 A1), Abstract and Paragraph [0028-0029].
Regarding claim 19, ROTH in view of CARETTE and in further view of ICKIN and in further view of AZARIAN explicitly teaches the method according to claim 18, ROTH fails to explicitly teaches further comprising: adding the synthetic representation to an existing test data set accessible for the model aggregator device for at least one of validating or testing the machine learned model to generate an extended test data set, and wherein the evaluating evaluates the local update based on the extended test data set.
However, ICKIN explicitly teaches further comprising: adding the synthetic representation to an existing test data set accessible for the model aggregator device for at least one of validating or testing the machine learned model to generate an extended test data set (Fig. 5. Paragraph [0090]-ICKIN discloses the synthetic dataset may then be appended to the original dataset to provide a larger combined training dataset. The larger combined training dataset may be used to train a NN model), and wherein the evaluating evaluates the local update based on the extended test data set (Fig. 5. Paragraph [0094]-ICKIN discloses the master evaluates the quality metric and, in this example, decides that the worker 200-N should proceed with using the synthetic training dataset S′, which it indicates to the worker 200-N in a message 509. In paragraph [0095]-ICKIN discloses the worker 200-N then combines the real training dataset S with the synthetic training dataset S′ at block 510 and trains the ML model using the combined dataset (S, S′) at block 512. In paragraph [0096]-ICKIN discloses the worker 200-N then sends the re-trained weights to the master 100 in a message 513. The master 100 combines the re-trained weights with trained weights from other workers 200 at block 514, and transmits the combined weights to the workers 200 in a message 515. In paragraph [0097]-ICKIN discloses the decision to require synthetic data may depend on the quality of weights provided by a given worker and/or based on other consideration. Please also read paragraph [0005 and 0090-0092]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of ICKIN of having evaluating, at the model aggregator device, the local update using the synthetic representation to obtain an evaluation result indicative of performance of the local update; updating, at the model aggregator device, the machine learned model based on the evaluation result.
Wherein ROTH’s method having updating, at the model aggregator device, the machine learned model based on the evaluation result and the local update
The motivation behind the modification would have been to obtain a method that improves federated learning using neural networks, since both ROTH and ICKIN concern federated learning. Wherein ROTH’s systems and methods improve federated learning for neural networks, while ICKIN’s systems and methods improves model performance in federated learning using synthetic data. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and ICKIN et al. (US 20230088561 A1), Abstract and Paragraph [0063].
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over ROTH et al. (US 20220366220 A1), hereinafter referenced as ROTH in view of CARETTE et al. (US 20260212206 A1), hereinafter referenced as CARETTE and in further view of ICKIN et al. (US 20230088561 A1), hereinafter referenced as ICKIN and in further view of FISHER et al. (US 20230126877 A1), hereinafter referenced as FISHER and in further view of ARGAWAL et al. (US 20240428468 A1), hereinafter referenced as ARGAWAL.
Regarding claim 20, ROTH in view of CARETTE and in further view of ICKIN and in further view of FISHER explicitly teaches he method according to claim 6, ROTH in view of CARETTE and in further view of ICKIN and in further view of FISHER fail to explicitly teaches wherein the determining a data quality of the synthetic representation comprises: generating a natural language summary based on the synthetic representation; comparing the natural language summary to a natural language prompt indicating the synthetic representation; and determining the data quality based on the comparing the natural language summary to the natural language prompt.
However, AGARWAL explicitly teaches wherein the determining a data quality of the synthetic representation (Fig. 2. Paragraph [0047]-AGARWAL discloses FIG. 2 shows the attention image synthesis system 102 via the diffusion neural network 202 utilizing at least one of the attention retention loss 202a and the attention segregation loss 202b to generate the text-conditioned image 204 (wherein attention image synthesis system 102 may be implemented in a distributed computing environment). In paragraph [0042]-AGARWAL discloses the attention image synthesis system 102 utilizes machine learning to process the text query 200. In paragraph [0043]-AGARWAL discloses a neural network includes a convolutional neural network, a recurrent neural network (e.g., a long short-term memory neural network), a transformer neural network, a generative adversarial neural network, a graph neural network, a diffusion neural network, or a multi-layer perceptron) comprises:
generating a natural language summary based on the synthetic representation (Fig. 1. Paragraph [0104]-AGARWAL discloses experimenters generated captions for generated images with BLIP);
comparing the natural language summary to a natural language prompt indicating the synthetic representation (Fig. 2. Paragraph [0104]-AGARWAL discloses experimenters compares the BLIP-generated captions with the text prompts (e.g., a ground truth)); and
determining the data quality based on the comparing the natural language summary to the natural language prompt (Fig. 2. Paragraph [0104]-AGARWAL discloses FIG. 10 shows a comparison between an example implementation of the attention image synthesis system 102 and prior methods. For each animal-animal, animal-object, and object-object, the attention image synthesis system 102 (indicated as A-STAR) shows improvement over prior methods in terms of text-text similarities. Namely, the attention image synthesis system 102 (A-STAR) shows a text-text similarity of 0.82, 0.84, and 0.82 which is at least 0.01 greater than the best results for prior methods).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ROTH in view of CARETTE and in further view of ICKIN of having a computer-implemented method for federated learning of a machine learned model in a model aggregator device, with the teachings of ARGAWAL of having wherein the determining a data quality of the synthetic representation comprises: generating a natural language summary based on the synthetic representation; comparing the natural language summary to a natural language prompt indicating the synthetic representation; and determining the data quality based on the comparing the natural language summary to the natural language prompt.
Wherein ROTH’s method having wherein the determining a data quality of the synthetic representation comprises: generating a natural language summary based on the synthetic representation; comparing the natural language summary to a natural language prompt indicating the synthetic representation; and determining the data quality based on the comparing the natural language summary to the natural language prompt.
The motivation behind the modification would have been to obtain a method that improves the training and performance of neural networks, since both ROTH and ARGAWAL concern image analysis and neural networks. Wherein ROTH’s systems and methods improve federated learning for neural networks, while ARGAWAL’s systems and methods improves operational flexibility and accuracy of an image synthesis system. Please see ROTH et al. (US 20220366220 A1), Abstract and Paragraph [0116] and ARGAWAL et al. (US 20230126877 A1), Abstract and Paragraph [0029].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant`s disclosure.
HALL et al. (US 20220344049 A1)- A decentralized training platform is described for training an Artificial Intelligence (AI) model where training data (e.g., medical images) is distributed across multiple sites (nodes) and due to confidentiality, legal, or other reasons the data at each site is unable to be shared or leave the site and so cannot be copied to a central location for training. The method comprises training a teacher model locally at each node and then moving each of the teacher models to a central node and using these to train a student model using a transfer dataset. This may be facilitated by setting up the cloud service using inter-region peering connections between the nodes to make the nodes appear as a single cluster. In one variation the student module may be trained at each node using the multiple trained teacher models. In another variation we train multiple student models where each student model is trained by each teacher model at the node the teacher model was trained on, and once the plurality of student models are trained, an ensemble model is generated from the plurality of trained student models. Loss function weighting and node under sampling to enable load balancing may be used to improve accuracy and time/cost efficiency....…....................... Please see Fig. 1-3. Abstract.
MARTINS et al. (US 20230058223 A1)- A method for enabling a machine learning model to be aggregated from local machine learning models comprised in at least two local nodes in a wireless communications network is provided. The method comprises receiving, from each of the at least two local nodes, a parametrized function of a local machine learning model, a generator function of a local generative model, and a discriminator function of a local discriminative model, wherein the generator function and the discriminator function are trained on the same data as the parametrized function. The method also comprises determining, for each pair of the at least two local nodes, a first cross-discrimination values by applying the received discriminator function from a first local node of the pair on samples generated using the received generator function from the second local node of the pair, and a second cross-discrimination value by applying the....…....................... Please see Fig. 2-5. Abstract.
VERMA et al. (US 20220051146 A1)- Techniques for non-iterative federated learning include receiving local models from agents, generating synthetic datasets for the local models, and producing outputs using the local models and the synthetic datasets. A global model is trained based on the synthetic datasets and the outputs.....…....................... Please see Fig. 1-5. Abstract.
Choudhary et al. (US 20240394407 A1)- The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements a secure distributed data collaboration architecture for generating synthetic datasets. For example, the disclosed system sends a request to perform a data collaboration with a first dataset of a first local node and a second dataset of a second local node. The disclosed system receives intermediate feature maps from the local nodes that correspond with the datasets and generates a combined feature map. Further, the disclosed system generates a synthetic dataset from the combined feature map by utilizing a central generative model. Moreover, the synthetic dataset generated by the disclosed system is statistically representative of the first dataset and the second dataset.......…....................... Please see Fig. 1-5. Abstract.
Natarajan et al. (US 20240370708 A1)- An example assistant system can use a multimodal multitask medical machine-learned model to perform image processing to answer natural language queries. A device can process speech data or other natural language inputs to obtain a query. The query can be processed alongside image data that provides context for the query. The example system can receive a query associated with a particular task domain; generate, based on the query, a query input that comprises query instruction data from a first modality and query context data from a second modality; generate a combined input comprising the query input and an exemplar input, wherein the exemplar input comprises exemplar instruction data from the first modality and an exemplar context placeholder in lieu of exemplar context data from the second modality; process the combined input with a multimodal machine-learned model to generate output data; and output a query response based on the output data......…....................... Please see Fig. 2-8. Abstract.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to Aaron Bonansinga whose telephone number is (703) 756-5380 The examiner can normally be reached on Monday-Friday, 9:00 a.m. - 6:00 p.m. ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Chineyere Wills-Burns can be reached by phone at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON TIMOTHY BONANSINGA/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673