DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/10/2023. The submission is in compliance with the provisions of 37 CFR1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 14-15 , are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As to Claim 14
The limitation “ computationally more expensive than training the proxy model.” makes the claim unclear on what more expensive means.
Claims 15 is further rejected on virtue of dependency to claim 14.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103, which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 1-3, 11-13 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sagawa et. al “Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization”(Sagawa) in view of Kalra et. al “ Shared Model Training Privacy Protection “US-20230153461-A1” ( “Kalra”)
As to Claim 1:
Sagawa teaches “A computer-implemented method for calibrating a training distribution for training a machine-learned model (Sagawa “[pg -1] Distributionally robust optimization (DRO) allows us to learn models that instead minimize the worst-case training loss over a set of pre-defined groups. Machine learning models [a machine-learned model ] are typically trained [training] to minimize the average loss on a training set, with the goal of achieving high accuracy on an independent and identically distributed (i.i.d.) test set. [calibrating a training distribution]
2. accessing a training dataset characterized by a plurality of data domains (Sagawa “[pg -2] This training procedure is an instance of distributionally robust optimization (DRO),
PNG
media_image1.png
324
773
media_image1.png
Greyscale
Figure 1: Representative training and test examples for the datasets we consider. The correlation between the label y and the spurious attribute a at training time does not hold at test time.
Under the BRI, Considering a different datasets as represented by Figure 1 is interpreted as accessing a training dataset characterized by a plurality of data domains corresponding to the claim.
3. training a reference model using a first batch of training examples, the first batch of training examples sampled from the training dataset according to an initial probability distribution over the plurality of data domains” (Sagwa pg -2)
PNG
media_image2.png
179
1021
media_image2.png
Greyscale
Under BRI , empiricial distribution p̂ is interpreted as initial probability distribution over the plurality of data domain and training data drawn from some distribution P is interpreted as first batch of training examples sampled from training dataset. And the goal is to find the model … standard training procedure is interpreted as training a reference model.
4. and outputting a calibrated training distribution over the plurality of data domains based on the plurality of learned distribution parameters. (Sagawa , pg-3 The uncertainty set Q encodes the possible test distributions that we want our model to perform well on. Choosing a general family Q, such as a divergence ball around the training distribution, confers robustness to a wide set of distributional shifts, but can also lead to overly pessimistic models which optimize for implausible worst-case distributions To construct a realistic set of possible test distributions without being overly conservative, we leverage prior knowledge of spurious correlations to define groups over the training data and then define the uncertainty set Q in terms of these groups. Concretely, we adopt the group DRO setting where the training distribution P is assumed to be a mixture of m groups Pg indexed by G = {1 ,2 …. , m) (plurality of data domains). We define the uncertainty set Q as any mixture of these groups, i.e. Q: = {
∑
g
=
1
m
q
Pg :q ∈ Δm } where Δm is (m-1) dimensional probability simplex, this choice Q allows us to learn models that are robust to group shifts.
PNG
media_image3.png
89
922
media_image3.png
Greyscale
Examiner notes : Under BRI Q is interpreted as Outputting calibrated training and Pg is interpreted as plurality of Learned distribution parameters in equation i.e corresponding to the claim.
Sagawa does not explicitly teach
training a proxy model using a second batch of training examples, the second batch of training examples sampled from the training dataset according to a proxy probability distribution over the plurality of data domains,
wherein:the proxy model is trained by evaluating, for a respective training iteration, a comparison between the reference model and the proxy model, wherein the comparison is evaluated using a plurality of learned distribution parameters;
learning, jointly during training of the proxy model, the plurality of learned distribution parameters;
Kalra teaches “training a proxy model using a second batch of training examples, the second batch of training examples sampled from the training dataset according to a proxy probability distribution over the plurality of data domains, (Kalra, paragraph 0048, 0049 “ [0048] A first experiment was performed to compare the accuracy of the ProxyFL [ a proxy model] embodiment with other federated models. [0049]Fa/MNIST has 60k training images of size 28×28, while CIFAR-10 has 50k RGB training images of size 32×32. [using a second batch of training examples]Each dataset has 10k test images, which are used to evaluate the model performance. Experiments were conducted on a server with 8 V100 GPUs, which correspond to 8 clients. In each run [training], every client had 1k (Fa/MNIST) or 3k (CIFAR-10) nonoverlapping private images sampled from the training set. [the second batch of training examples sampled from the training dataset] To test robustness on non-IID data (i.e., data with a different distribution than the client’s private training data), clients were given a skewed private data distribution. For each client, a randomly chosen class was assigned and a fraction pmajor (0.8 for Fa/MNIST; 0.3 for CIFAR-10) [a proxy probability distribution over the plurality of data domains,] of that client’s private data was drawn from that class.
Examiner notes: Under the BRI the proxyfFL is interpreted as poxy model corresponding to the claim, and each clients trains locally on the sampled training dataset.
2. :the proxy model is trained by evaluating, for a respective training iteration, a comparison between the reference model and the proxy model, wherein the comparison is evaluated using a plurality of learned distribution parameters; (Kalra paragraph 0025,0057 “[0025],The training module 130 trains [is trained by evaluating ] parameters of the proxy model 150 [proxy model] and private model 160 based on the data in the training data store 170 and parameters of the models. In general, the models may be trained in one or more training iterations [for a respective training iteration] based on batches of training data from the training data store 170. Each training data instance may be processed by the current parameters of the respective models to determine a prediction from that model [using a plurality of learned distribution parameters;]. The prediction by the model may be compared [comparasion] with the output labels associated with the training data instance to determine [is evaluated] a predictive loss based on a difference of the model prediction with the desired prediction (i.e., the labeled outcome.) [0057] For ProxyFL and FML the private models are used to compute the accuracy values whereas the central model [reference model] is used in the case of FedAvg.”).
PNG
media_image4.png
518
618
media_image4.png
Greyscale
3. learning, jointly during training of the proxy model, the plurality of learned distribution parameters; (Kalra “[0007] In the training step, the proxy model is jointly trained [learning, jointly during training of the proxy model] with the private model, such that the parameters of each model may be trained with a batch of training data. In addition to training with respect to a training batch, the models may also be trained with an objective (e.g., a training loss to be minimized) [ the plurality of learned distribution parameters;] based on the other model’s predictions.”).
Kalra and Sagawa are related to the same field of endeavor (i.e. Machine Learning ). In view of the teachings of Sagawa it would have been obvious for a person of ordinary skill in the art to apply the teachings of Kalra to Sagawa before the effective filing date of the claimed invention in order make the model more scalable while reducing the computational cost. ( Kalra, “[pg – 15] FIG. 6 shows the communication time for exchanging parameters for one embodiment in experiments. ProxyFL has a much lower communication cost compared to FML, as shown in FIG. 6. The exponential protocol has a constant time complexity per round regardless of the number of clients, which makes ProxyFL much more scalable.”).
As to claim 2:
Sagawa in view of Kalra teaches the method of claim 1.
Sagawa further teaches “jointly learning the plurality of learned distribution parameters using distributionally robust optimization (DRO) over the plurality of data domains” (Sagawa pg-1,3 “[Pg-1] Distributionally robust optimization (DRO) allows [using distributionally robust optimization (DRO)] us to learn that instead minimize the worst-case training loss over a set of pre-defined groups. ( jointly learning the plurality of learned distribution parameters) [ Pg-3] Concretely, we adopt the group DRO setting where the training distribution P is assumed to be a mixture of m groups Pg indexed by G = {1 ,2 …. , m) (plurality of data domains). We define the uncertainty set Q as any mixture of these groups, i.e. Q: = {
∑
g
=
1
m
q
g Pg :q ∈ Δm } where Δm is (m-1) dimensional probability simplex, this choice Q allows us to learn models that are robust to group shifts.
Sagawa and Kalra are combinable for the same rationale as set forth above with respect to Claim 1
As to claim 3:
Sagawa in view of Kalra teaches the method of claim 1.
Kalra further teaches “wherein the plurality of learned distribution parameters are used for weighting the comparison between the reference model and the proxy model.” (Kalra, paragraph “[0057] The experiments on WSI data were conducted using four V100 GPUs. Three FL methods were compared: [comparison] ProxyFL, FML, and FedAvg. In each scenario, training was conducted for 50 rounds with a mini-batch size of 16. All methods were tested with two DP settings, one with strong privacy σ = 1.4, and the other with comparatively weak privacy σ = 0.7, both with C = 0.7.The client-level privacy guarantees for the two DP settings are provided in FIG. 8C. FedAvg and the proxy models [and the proxy model] used the DP-SGD optimizer, whereas the private models used the Adam optimizer, both with a learning rate of 0.001. For ProxyFL and FML the private models are used to compute the accuracy values, [plurality of learned distribution parameters are used for weighting] whereas the central model [reference model] is used in the case of FedAvg.”).
Examiner notes: Under the BRI, ProxyFL and private model parameters together is interpreted as plurality of learned distribution parameters corresponding to the claim and (σ = 1.4 or 0.7, C = 0.7) are learned updates.
Sagawa and Kalra are combinable for the same rationale as set forth above with respect to Claim 1.
As to claim 11:
Sagawa in view of Kalra teaches the method of claim 1.
Sagawa Further teaches” generating a training trajectory that accumulates a plurality of states for the plurality of learned distribution parameters; and determining the calibrated training distribution based on the plurality of states.”(Sagawa “[pg-3] Concretely, we adopt the group DRO setting where the training distribution P is assumed to be a mixture of m groups Pg indexed by G = {1 ,2 …. , m) We define the uncertainty set Q as any mixture of these groups, i.e. Q: = {
∑
g
=
1
m
q
Pg :q ∈ Δm } where Δm is (m-1) dimensional probability simplex, this choice Q allows us to learn models that are robust to group shifts.
PNG
media_image3.png
89
922
media_image3.png
Greyscale
Examiner notes : The entire expression {
∑
g
=
1
m
q
Pg :q ∈ Δm } is interpreted as generating a trajectory and Pg is interpreted as plurality of learned distribution parameters and { P1, p2 .. pm} are interpreted as plurality of states that is accumulated using the expression {
∑
g
=
1
m
q
Pg :q ∈ Δm }, and Q is interpreted as the determining the calibrated training distribution based on plurality of states.”).
Sagawa and Kalra are combinable for the same rationale as set forth above with respect to Claim 1.
As to claim 12:
Sagawa in view of Kalra teaches the method of claim 11.
Sagawa Further teaches “determining the calibrated training distribution based on the plurality of states by at least one of; or determining a representative value of a cluster of the plurality of states.” (Sagawa “[pg-3]Concretely, we adopt the group DRO setting where the training distribution P is assumed to be a mixture of m groups Pg indexed by G = {1 ,2 …. , m) We define the uncertainty set Q as any mixture of these groups, i.e. Q: = {
∑
g
=
1
m
q
Pg :q ∈ Δm } where Δm is (m-1) dimensional probability simplex, this choice Q allows us to learn models that are robust to group shifts.
PNG
media_image3.png
89
922
media_image3.png
Greyscale
Examiner notes: Q is interpreted as the determining the calibrated training distribution based on plurality of states i.e. p1,p2… pm which as a group is interpreted as cluster and is
determined using the eq Q: = {
∑
g
=
1
m
q
Pg :q ∈ Δm } which is interpreted as determining a representative value of a cluster of plurality of states.”).
Sagawa and Kalra are combinable for the same rationale as set forth above with respect to Claim 1.
As to claim 13:
Sagawa teaches “A computer-implemented method for training a machine-learned model using a calibrated training mixture, (Sagawa “[pg -1] Distributionally robust optimization (DRO) allows us to learn models [A computer-implemented method ] that instead minimize the worst-case training loss over a set of pre-defined groups. Machine learning models [a machine-learned model ] are typically trained [training] to minimize the average loss on a training set,[ calibrated training mixture] with the goal of achieving high accuracy on an independent and identically distributed (i.i.d.) test set.
2. the method comprising: sampling a training example from a training dataset according to a calibrated training distribution; Sagawa “[pg -1] Machine learning models are typically trained to minimize the average loss on a training set, with the goal of achieving high accuracy on an independent and identically distributed (i.i.d.) test set. [according to a calibrating a training distribution] ( “[pg -2] This training procedure is an instance of distributionally robust optimization (DRO),
PNG
media_image1.png
324
773
media_image1.png
Greyscale
Figure 1: Representative training and test examples for the datasets we consider.[ sampling a training example from a training dataset] The correlation between the label y and the spurious attribute a at training time does not hold at test time.
3. and training the machine-learned model using the training example; wherein the calibrated training distribution was calibrated by: (“Sagawa “[pg -1] Machine learning models are typically trained to minimize the average loss on a training set, with the goal of achieving high accuracy on an independent and identically distributed (i.i.d.) test set. [the calibrating a training distribution is Calibrated by]. “[pg- 18] We use the Pytorch torchvision implementation of the ResNet50 model starting from pretrained weights…. For the standard training experiments in Section 3.1, we use a l2 penalty of λ = 0:0001 for both Waterbirds and CelebA, with a learning rate of 0:001 for Waterbirds and 0:0001 for CelebA. We train the CelebA models [training the machine-learned model using the training example] for 50 epochs and the Waterbirds models for 300 epochs.
Examiner notes: Under the BRI CelebA model is interpreted as machine learned model consisting of training examples shown in fig 1.
PNG
media_image5.png
291
748
media_image5.png
Greyscale
And for all the other limitation of Claim 19 it is rejected under the same basis as claim 1. As the Claim are analogous.
Sagawa and Kalra are combinable for the same rationale as set forth above with respect to Claim 1.
As to claim 19:
Kalra teaches “A computing system, comprising: one or more processors; and one or more non-transitory computer-readable media storing:” (Kalra, Paragraph, “[0063],Embodiments of the invention may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device [A computing system, comprising ] selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, tangible computer readable storage medium, [and one or more non-transitory computer-readable media storing] or any type of media suitable for storing electronic instructions, which may be coupled to a computer system bus.”).
2. and instructions that are executable by the one or more processors to cause the computing system to perform one or more operations, the operations comprising: obtaining input data; and generating output data using the machine-learned model and based on the input data. (Kalra Paragraph 0023, 0063 “[0063] Such a computer program may be stored in a non-transitory, tangible computer readable storage medium, or any type of media suitable for storing electronic instructions, [and instructions] which may be coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor [are executable by the one or more processors to cause the computing system to perform one or more operations, ]or may be architectures employing multiple processor designs for increased computing capability. [0023] The proxy model 150 and private model 160 are machine-learned models [using the machine-learned model] that may have a number of layers for processing an input [obtaining input data]to generate predicted outputs. [and generating output data based on the input data]”).
Kalra does not explicitly teach:
a machine-learned model that was trained using a calibrated training distribution
wherein the calibrated training distribution was calibrated by
Sagawa teaches “a machine-learned model that was trained using a calibrated training distribution wherein the calibrated training distribution was calibrated by “ (Sagawa, pg-1 , pg -18 ,“[pg -1] Machine learning models are typically trained to minimize the average loss on a training set, with the goal of achieving high accuracy on an independent and identically distributed (i.i.d.) test set. [using a calibrating a training distribution Calibrated by]. “[pg- 18] We use the Pytorch torchvision implementation of the ResNet50 model starting from pretrained weights…. For the standard training experiments in Section 3.1, we use a l2 penalty of λ = 0:0001 for both Waterbirds and CelebA, with a learning rate of 0:001 for Waterbirds and 0:0001 for CelebA. We train the CelebA models [a machine-learned model that was trained using a calibrated training distribution] for 50 epochs and the Waterbirds models for 300 epochs.”).
And for all the other limitation of Claim 19 it is rejected under the same basis as claim 1. As the Claim are analogous.
Sagawa and Kalra are combinable for the same rationale as set forth above with respect to Claim 1.
As to claim 20:
Sagawa in view of Kalra teaches the System of claim 19.
Sagawa Further teaches “calibrated training distribution was calibrated by: jointly learning the plurality of learned distribution parameters using distributionally robust optimization (DRO) over the plurality of data domains.” (Sagawa pg-1, pg-3, “[pg-1] Distributionally robust optimization (DRO) allows[using distributionally robust optimization (DRO)] us to learn that instead minimize the worst-case training loss over a set of pre-defined groups. ( jointly learning the plurality of learned distribution parameters) [ Pg-3] Concretely, we adopt the group DRO setting where the training distribution P is assumed to be a mixture of m groups Pg indexed by G = {1 ,2 …. , m) (plurality of data domains). We define the uncertainty set Q as any mixture of these groups, i.e. Q: = {
∑
g
=
1
m
q
g Pg :q ∈ Δm } where Δm is (m-1) dimensional probability simplex, this choice Q allows us to learn models that are robust to group shifts.
Sagawa and Kalra are combinable for the same rationale as set forth above with respect to Claim 19.
Claims 4-10 are rejected under 35 U.S.C. 103 as being unpatentable over (“Sagawa”) in view (“Kalra”) and in view of Oren et. al “Training Compute-Optimal Large Language Models”(“Oren”).
As to claim 4:
Sagawa in view of Kalra teaches the method of claim 1.
Sagawa in view of Kalra does not teach
wherein the plurality of learned distribution parameters are learned based on the comparison between the reference model and the proxy model.
Oren teaches “wherein the plurality of learned distribution parameters are learned based on the comparison between the reference model and the proxy model.”( Oren, pg-4, pg -6 “[pg-4]The baselined loss function interpreted as l (x; z; θ) = log px|z (x | z) - log pθ (x). evaluates the performance of the model relative to the best possible model for the topic, [pg- 6] We also keep track of the historical average of losses incurred for each topic so far, up to the current round t, which we call Î (t) (z; θ(1:t) ).Concretely, Î (t) (z; θ(1:t) ) is computed as an average of
{ ℓ (x (tt); θ(t’ )) : t’ ∈ [t], z(t’)= z}.At each iteration t, pz is updated by selecting an optimal value with respect to historical losses up to the current iteration, loosely inspired by the “Be The Leader” algorithm. This results in the following update rule to pz,
PNG
media_image6.png
81
550
media_image6.png
Greyscale
Examiner notes : PZ t is interpreted as plurality of learned distribution parameters learned based on argmax operation in eq 11, and (x; z; θ) = log px|z (x | z) - log pθ (x). is interpreted as comparison between proxy model and reference model .
Oren and Sagawa are related to the same field of endeavor (i.e. Machine Learning ). In view of the teachings of Sagawa it would have been obvious for a person of ordinary skill in the art to apply the teachings of Oren to Sagawa before the effective filing date of the claimed invention in order to minimize the loss of the model and optimize the performance while training different data domains. (Oren, “[pg-1]To remedy this without the knowledge of the test distribution, we propose an approach which trains a model that performs well over a wide range of potential test distributions). Furthermore, person of ordinary skill in the art will be motivated to apply the teachings of Oren to Sagawa in order to select the optimal parameter during the training of ML model. [Oren “Pg-6] “At each iteration t, pz is updated by selecting
an optimal value with respect to historical losses up to the current iteration.).
As to claim 5:
Sagawa in view of Kalra and in view of Oren teaches the method of claim 4
Sagawa further teaches “wherein: the proxy model is updated to change the comparison in a first direction;and the plurality of learned distribution parameters are updated to change the comparison in a second, different direction. (Sagawa “[pg- 9]
PNG
media_image7.png
108
903
media_image7.png
Greyscale
Extending existing minimax algorithms for DRO (Namkoong & Duchi, 2016; Oren et al., 2019), we interleave gradient-based updates on θ [plurality of learned distribution parameters are updated ] and q. Intuitively, we maintain a distribution q over groups, with high masses on high-loss groups, and update on each example proportionally to the mass on its group. Concretely, we interleave SGD on θ and exponentiated gradient ascent on q [change the comparasion on first direction and change the comparasion on second , different direction ] (Algorithm 1).
PNG
media_image8.png
326
1059
media_image8.png
Greyscale
Examiner notes: Interleave SGD is interpreted as changing the direction as SGD deals with gradient descending vs gradient ascent and q in algorithm is interpreted as proxy model corresponding to the claim.”).
Sagawa , Kalra and Oren are combinable for the same rationale as set forth above with respect to Claim 4.
As to claim 6:
Sagawa in view of Kalra and in view of Oren teaches the method of claim 5.
Oren Further teaches “jointly learning the learned distribution parameters comprises updating the plurality of learned distribution parameters to amplify a difference metric computed between the reference model and the proxy model, and training the proxy model comprises updating the proxy model to decrease the difference metric computed between the reference model and the proxy model.”(Oren, pg -4 , pg-6 “[ pg-4],The baselined loss function f(x, z; θ) = log px|z (x | z) − log pθ(x) evaluates the performance of the model relative to the best possible model for the topic, log px|z (x | z). [difference metric computed between the reference model and the proxy model].H(X | Z = z) by fitting a baseline model pβ_for each topic, and computing Hβ (X | Z = z) :=Epx|z [-log pβ (x | z)]. [Pg -6 ] At each iteration t, pz is updated [updating the plurality of learned distribution parameters] by selecting an optimal value with respect to historical losses up to the current iteration, loosely inspired by the “Be The Leader” algorithm. This results in the following update rule to pz,
PNG
media_image9.png
53
255
media_image9.png
Greyscale
The above argmax can computed efficiently by ordering topics in the order of decreasing average loss, and assigning each topic either
PNG
media_image10.png
42
84
media_image10.png
Greyscale
or the probability left to be assigned, whichever is lower.3 We update θ with online gradient descent, [training the proxy model]
PNG
media_image11.png
105
599
media_image11.png
Greyscale
PNG
media_image12.png
250
535
media_image12.png
Greyscale
Examiner notes: Here the expression log px|z (x | z) − log pθ(x) is interpreted as difference metric computed between the models and log pθ CVaR model consists θ of as a parameter vector interpreted as proxy model updated via a gradient decent. Pz is interpreted as jointly learning the learned distribution parameters as pzt is computed in each round in eq 11(detailed by pg-5). Furthermore, baseline model px|z is interpreted as a reference model which is substituted by px|z during computation .The eq of θt is interpreted as updating the proxy model to decrease the difference metric, corresponding to the claim.”).
Sagawa , Kalra and Oren are combinable for the same rationale as set forth above with respect to Claim 4.
As to claim 7:
Sagawa in view of Kalra and in view of Oren teaches the method of claim 6.
Oren Further teaches “evaluating the comparison by:determining a plurality of difference metrics respectively for the plurality of data domains,
wherein a respective difference metric for a respective domain comprises a comparison between: values generated by the reference model for training inputs from the respective domain,
and values generated by the proxy model for the training inputs from the respective domain;
weighting the plurality of difference metrics respectively using the plurality of learned distribution parameters;
and aggregating the weighted plurality of difference metrics. “
(Oren, “[pg 4] we will show later in section 4.2 that we can estimate sufficient statistics of log px|z (x | z) that allow us to compute the baselined loss. By using baselined loss, we intuitively encourage models to perform as well as it can on each topic while making optimal trade-offs among topics.
Plugging the baselined loss into the robust objective (7), the optimization problem is
PNG
media_image13.png
246
573
media_image13.png
Greyscale
Examiner notes: log pX|z is value generated by reference model , for training inputs x from respective domain z and p θ(x) is value generated by the proxy model The difference metric is the interpreted as [ log p|x ( x|z) – log p θ(x)] computed between two values of model. The outer pz is interpreted as leaned distribution parameters for domain z. Eq 8 as a whole is interpreted as weighting and aggregating the weighted plurality of difference metrics as
PNG
media_image14.png
59
116
media_image14.png
Greyscale
performs both weighting and then aggregation of plurarity of difference metric i.e. corresponding to the claim.”).
As to claim 8:
Sagawa in view of Kalra and in view of Oren teaches the method of claim 7.
Kalra Further teaches “wherein the respective difference metric comprises an excess loss of the proxy model as compared to the reference model.” (Kalra “paragraph [0032] In one embodiment in which the distillation loss for the private model 220 is aKL-divergence loss L.sub.KL with respect to proxy parameters h.sub.θ.sub.k of the proxy model [proxy model] 210A, the distillation loss L.sub.KL may be described by Equation 2:
PNG
media_image15.png
174
903
media_image15.png
Greyscale
Examiner notes:- private model is interpreted as reference model corresponding to the claim and KL divergence loss is interpreted as respective difference metric with excess loss comparasion.”).
Sagawa , Kalra and Oren are combinable for the same rationale as set forth above with respect to Claim 4.
As to claim 9:
Sagawa in view of Kalra and in view of Oren teaches the method of claim 7.
Kalra further teaches “the values generated by the proxy model for the training inputs from the respective domain correspond to predictions associated with designated outputs.” (Kalra paragraph [0023] [0024] “[0023] The proxy model 150 [the proxy model] and private model 160 are machine-learned models that may have a number of layers for processing an input to generate predicted outputs. ,[0024] A particular input example may be referred to as a data instance or data record, which may represent a “set” of input data that may be input to a model for which the model generates one or more output [ the values generated] predictions. In these examples, the training data in the training data store 170 may include input data instances [training inputs] along with labeled outputs for the data for which the models may be trained to learn parameters that accurately predict outputs matching the labels for a given input data instance. [respective domain corresponds to predictions associated with designated outputs.]”).
Sagawa , Kalra and Oren are combinable for the same rationale as set forth above with respect to Claim 4.
As to claim 10:
Sagawa in view of Kalra and in view of Oren teaches the method of claim 9
Kalra further teaches “a designated output corresponds to a ground truth token” (Kalra “paragraph [0024] A particular input example may be referred to as a data instance or data record, which may represent a “set” of input data that may be input to a model for which the model generates one or more output [ the values generated] predictions. In these examples, the training data in the training data store 170 may include input data instances [training inputs] along with labeled outputs for the data for which the models may be trained to learn parameters that accurately predict outputs designated output ]matching [corresponds]the labels for a given input data instance. [to a ground truth token]
Sagawa , Kalra and Oren are combinable for the same rationale as set forth above with respect to Claim 4.
Claims 14 is rejected rejected under 35 U.S.C. 103 as being unpatentable over (“Sagawa”) in view of (“Kalra”) and in view of Mindermann et.al “Prioritized Training on Points that are learnable, Worth Learning, and Not Yet Learnt.”( “Mindermann”).
As to claim 14:
Sagawa in view of Kalra teach the method of Claim 13.
Sagawa in view of Kalra does not teach .
wherein training the machine-learned model is computationally more expensive than training the proxy model.
Mindermann teaches wherein training the machine-learned model is computationally more expensive than training the proxy model. ( Mindermann, “[Pg 4] Counterintuitively, the reducible holdout loss can therefore be negative. Additionally, one IL model can be reused for many target model runs, amortizing its cost (Section 4.2). For example, we trained all 40 seeds of 5 target architectures in Fig. 1 using a single ResNet18 IL model. Further, this model trained [wherein training the proxy model ] for 37x fewer steps than each target model (reaching only [wherein training the machine-learned model] 62% accuracy).”).
Examiner notes:
Under the BRI, training the target model with 37x fewer steps is interpreted as machine- learned model taking 37x more steps then proxy model and is interpreted as computationally more expensive than proxy model.
Mindermann and Sagawa are related to the same field of endeavor (i.e. Machine Learning ). In view of the teachings of Sagawa it would have been obvious for a person of ordinary skill in the art to apply the teachings of Mindermann to Sagawa before the effective filing date of the claimed invention in order to reduce wasted computation or redundant data in while increasing speed , accuracy in model training of plurality of data domains. (Mindermann, “[pg -1],RHO-LOSS trains in far fewer steps than prior art, improves accuracy, and speeds up training on a wide range of datasets, hyperparameters, and architectures (MLPs, CNNs, and BERT). On the large web-scraped image dataset Clothing-1M, RHO-LOSS trains in 18x fewer steps and reaches 2% higher final accuracy than uniform data shuffling.”).
Claims 15-18 are rejected rejected under 35 U.S.C. 103 as being unpatentable over (“Sagawa”) in view of (“Kalra”) and in view of (“Mindermann”) and in further view of Hoffmann et.al. “ Training Compute-Optimal Large Language Models”( “Hoffmann”).
As to claim 15:
Sagwa in view of Kalra and in view of Mindermann teach the method of Claim 14.
Mindermann further teaches, wherein: the machine-learned model is characterized by a first number of parameters; the proxy model is characterized by a second number of parameters;( Mindermann “[Pg 4] Counterintuitively, the reducible holdout loss can therefore be negative. Additionally, one IL model can be reused for many target model runs, amortizing its cost (Section 4.2). For example, we trained all 40 seeds of 5 target architectures in Fig. 1 using a single ResNet18 IL model. Further, this model trained [the proxy model ] for 37x fewer steps than each target model (reaching only [machine-learned model] 62% accuracy).”).
Examiner notes: Under the BRI the machine learned model consists of parameter during training which is interpreted as first number of parameters and target model is interpreted as poxy model which consists second number of parameters.
Mindermann does not explicitly teach
and the first number of parameters is at least ten times the second number of parameters.
Hoffmann teaches and the first number of parameters is at least ten times the second number of parameters. ( Hoffmann, “ [Pg -1], In our first approach we vary the number of training steps for a fixed family of models (ranging from 70M to over 10B parameters), training each model for 4 different number of training sequences.
Examiner notes: Under the BRI the first number of parameter and second number of parameters are interpreted to be in the family of models, that ranges from 70M to 10B Parameters which is interpreted as first number of parameters is at least ten times the second number of parameters i.e. corresponding to the claim.”).
Hoffmann and Sagawa are related to the same field of endeavor (i.e. Machine Learning ). In view of the teachings of Hoffmann it would have been obvious for a person of ordinary skill in the art to apply the teachings of Hoffmann to Sagawa before the effective filing date of the claimed invention in order to train the model to its full potential while reducing the computational cost. ( Hoffmann “Pg-1 By training over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens, we find that for compute-optimal training, the model size and the number of training tokens should be scaled equally.)
As to claim 16:
Sagawa in view of Kalra and in view of Mindermann and in further view of Hoffmann teach the method of Claim 15.
Hoffmann further teaches “the first number of parameters is at least thirty times the second number of parameters. ( Hoffmann, “ [Pg -1] In our first approach we vary the number of training steps for a fixed family of models (ranging from 70M to over 10B parameters), training each model for 4 different number of training sequences.”).
Examiner notes: Under the BRI Under the BRI the first number of parameter and second number of parameters are interpreted to be in the family of models, that ranges from 70M to 10B Parameters which is interpreted as first number of parameters is at thirty times the second number of parameters i.e. corresponding to the claim.
Sagawa , Kalra , Mindermann and Hoffmann are combinable for the same rationale as set forth above with respect to Claim 15.
As to claim 17:
Sagawa in view of Kalra teach the method of Claim 13.
Sagawa in view of Kalra does not teach
the machine-learned model is trained using a first training task; and the proxy machine-learned model is trained using a second training task different from the first training task.
Hoffmann teaches “ the machine-learned model is trained using a first training task; and the proxy machine-learned model is trained using a second training task different from the first training task.”(Hoffmann “[pg-8] FlOPS and training tokens for various model sizes. For various models sizes we show the projections from Approach 1 of how many FLOPs and training tokens would be needed to train compute-optimal models.
PNG
media_image16.png
254
704
media_image16.png
Greyscale
Examiner notes under the BRI, the parameter in the table are from the model interpreted as Machine learned model and Proxy model, where machine learned model consists of 10 trillion parameters and 1.30e +28 FLOPs in the training process and is interpreted as the machine-learned model is trained using a first training task and proxy model consists of 400 Million parameters and 1.92e+ 19 FLOPs in the training process and is interpreted as the and is interpreted as the proxy machine-learned model is trained using a second training task different from the first training task.”).
Sagawa , Kalra, and Hoffmann are combinable for the same rationale as set forth above with respect to Claim 15.
As to claim 18:
Sagawa in view of Kalra teach the method of Claim 13.
Sagawa in view of Kalra does not teach
a training iteration of the machine-learned model comprises executing a first number of floating point operations; and a training iteration of the proxy machine-learned model comprises executing a second number of floating point operations that is fewer than the first number of floating point operations.
Hoffmann teaches a training iteration of the machine-learned model comprises executing a first number of floating point operations; and a training iteration of the proxy machine-learned model comprises executing a second number of floating point operations that is fewer than the first number of floating point operations. (Hoffmann [pg-8] “FlOPS and training tokens for various model sizes. For various models sizes we show the projections from Approach 1 of how many FLOPs and training tokens would be needed to train compute-optimal models.
PNG
media_image16.png
254
704
media_image16.png
Greyscale
Examiner notes : under the BRI, the parameter in the table are from the model interpreted as Machine learned model and Proxy model, where machine learned model consists of 10 trillion parameters with 1.30e+28 FLOPs interpreted as first number of floating point operation. Furthermore, proxy model consists of 400 Million parameters with 1.92e+19 FLOPS interpreted as second number of floating operations which is fewer than first number of floating operation.
Sagawa , Kalra, and Hoffmann are combinable for the same rationale as set forth above with respect to Claim 15.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIROJ KOIRALA whose telephone number is (571)270-0748. The examiner can normally be reached Monday -Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MICHAEL HUNTLEY can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.K./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights