Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to submission of application on 03/15/2024.
Claims 1-8 are presented for examination.
The prior art used for rejections are provided below:
Tent: Fully Test-Time Adaptation By Entropy Minimization (March 18, 2021) to Wang et al. (hereinafter Wang)
Semi-supervised Domain Adaptation via Minimax Entropy (September 14, 2019) to Saito et al. (hereinafter Saito)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-8 are rejected under 35 USC 103 as being unpatentable over Wang in view of Saito.
Per claim 1, Wang discloses A non-transitory computer-readable recording medium storing a machine learning program for causing a computer to execute processing comprising [Wang, pg. 1, Abstract "Please see the project page at https://github.com/DequanWang/tent for the code and more”. (note: Wang provides its test-time adaptation method as a machine learning program implemented in software and executed by a computer, the code being distributed for others to run); pg. 4, Section 4 "Our implementation is in PyTorch (Paszke et al., 2019) with the pycls library"]:
generating a second machine learning model by updating, while fixing parameters of first one or more layers corresponding to a first position in a first machine learning model, parameters of second one or more layers corresponding to a second position in the first machine learning model [Wang, pg. 4, Section 3.3 "The optimizer collects the affine transformation parameters {γl, k, βl,k} for each normalization layer l and channel k in the source model. The remaining parameters θ ∖ {γl, k, βl,k} are fixed". (note: Wang generates an adapted (second) model from a trained source (first) model by updating the affine transformation parameters of the normalization layers (second one or more layers at a second position) while holding all remaining parameters - including the classification layers at the first position - fixed,)], based on a loss function including entropy of a first output that is outputted from the first machine learning model in response to an input of first data to the first machine learning model [Wang, pg. 3, Section 3.1 "Our test-time objective L(xt) is to minimize the entropy H(ŷ) of model predictions ŷ = fθ(xt)" . (note: the update is driven by a loss function that is the entropy of the model's output on the input data)], the first data being data that does not include correct labels [Wang, pg. 1, Section 1 "This fully test-time adaptation setting cannot rely on source data or supervision". (note: the input first data is unlabeled target data having no correct labels)].
Wang does not expressly disclose, but Saito does teach:
generating a third machine learning model by updating, while fixing parameters of third one or more layers corresponding to the second position in the second machine learning model, parameters of fourth one or more layers corresponding to the first position, based on a loss function including entropy of a second output that is outputted from the second machine learning model in response to the input of the first data to the second machine learning model. [Saito, pg. 1, Abstract "Adaptation is achieved by alternately maximizing the conditional entropy of unlabeled target data with respect to the classifier and minimizing it with respect to the feature encoder". (note: Saito teaches a second, complementary update step in which, after the feature-extractor layers (second position) are held, the classifier layers at the first position are updated based on the entropy of the model's output on the same unlabeled data, so that adaptation alternates between updating one layer group while the other is fixed); pg. 2, fig. 2 "Step 1: we update the estimated prototypes in the classifier to maximize the entropy on the unlabeled target domain. Step 2: we minimize the entropy with respect to the feature extractor"; pg. 4, Section 3.2 "the prototypes are parameterized by the weight vectors of the last linear layer". (note: the classifier updated based on entropy is the last linear (first-position) layer)].
Wang and Saito are analogous art because they are from the same field of endeavor, specifically adapting deep neural network classifiers to a shifted target domain by entropy-based objectives computed on unlabeled target data. They are further reasonably pertinent to the same problem of improving classification accuracy on unlabeled target data without access to correct labels.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art (PHOSITA) to incorporate the teaching of Saito - alternately updating a second, complementary group of layers (the classifier layers at the first position) based on the entropy of the model's output on unlabeled data while holding the previously-updated layers fixed - into Wang's test-time entropy-minimization method that updates the normalization layers at the second position, so as to arrive at the two-stage generation of a second and then a third machine learning model.
The suggestion/motivation for doing so is provided by Wang itself, which teaches that adapting more of the model than the normalization affine parameters is desirable [Wang, pg. 9, Section 6 "Test-time adaptation could update more of the model, but the issue is to identify parameters that are both expressive and reliable"]. Saito supplies exactly such an additional, reliable adaptation stage by showing that the classifier (first-position) layers can be updated from the entropy of unlabeled target data in alternation with the feature layers to reduce error [Saito, pg. 1, Abstract].
Combining these known entropy-based adaptation steps according to their established methods would have yielded the predictable result of further reducing generalization error on the unlabeled target data, consistent with the KSR rationale of combining prior art elements according to known methods to yield predictable results (MPEP Section 2143.01(A)).
Per Claim 2, Wang-Saito discloses claim 1.
Wang further teaches the processing further comprising: inputting the first data to the generated third machine learning model; and performing prediction on the first data based on a result outputted from the third machine learning model [Wang, pg. 4 “For offline adaptation, the model is first updated and then inference is repeated.”. (note: Wang uses the adapted model for inference after adaptation. Model 3, which is doubly adapted, is used for prediction on the same unlabeled data.)].
Per Claim 3, Wang-Saito discloses claim 1.
Wang further teaches generating a first machine learning model by updating parameters of each layer in the first machine learning model so that a difference between an output result of the first machine learning model and the correct labels is reduced, the output result being a result outputted from the first machine learning model in response to an input of training data including the correct labels to the first machine learning model, wherein
the generating of the second machine learning model includes generating the second machine learning model by updating, based on the loss function according to the input of the first data to be predicted to the first machine learning model, the first machine learning model generated using the training data,
the generating of the third machine learning model includes generating the third machine learning model by updating the first machine learning model based on the loss function according to the input of the first data to be predicted to the second machine learning model [Wang, pg. 2 “A model f θ(x) with parameters θ trained on source data and labels xs; ys may not generalize when tested on shifted target data xt”. (note: Wang requires a pre-trained source model, model 1, which is trained on labeled source data xs, ys. The adaptation procedure generates model 2 and model 3 from this base without altering the training procedure.)].
Per Claim 4, Wang-Saito discloses claim 1.
Wang does not fully disclose, but with Saito does teach the first machine learning model is a machine learning model that includes at least a batch normalization layer and a fully connected layer [Wang, pg. 4
PNG
media_image1.png
170
1145
media_image1.png
Greyscale
(note: this shows that tent’s model architecture includes batch normalization layers (with affine parameters) and fully connected layers (which are among the remaining fixed parameters));
Saito, pg. 5, 4.1. Setup “We remove the last linear layer of these networks to build F, and add a K-way linear classification layer C with a randomly initialized weight matrix W”. (note: this shows a feature extractor F (BN layers) and a separate linear classification layer C (FC layers))],
the generating of the second machine learning model includes generating the second machine learning model by updating parameters of the batch normalization layer corresponding to the second position in the first machine learning model while fixing the parameter of the fully connected layer corresponding to the first position in the first machine learning model [Wang, pg. 4 “The optimizer collects the affine transformation parameters
PNG
media_image2.png
44
132
media_image2.png
Greyscale
for each normalization layer l and channel k in the source model. The remaining parameters
PNG
media_image3.png
105
549
media_image3.png
Greyscale
are fixed”. (note: this updates BN affine parameters (y, β, the parameter of the normalization/BN layers) while fixing all other parameters including the FC classifier)],
the updating of the parameters of the batch normalization layer being performed by performing machine learning that minimizes the entropy of the output of the batch normalization layer based on the loss function [Wang, pg. 3 “
PNG
media_image4.png
98
1140
media_image4.png
Greyscale
”. (note: the entropy minimized is the entropy of the model’s output, which passes through the BN layer as part of the forward pass)],
the loss function being a loss function including the entropy of the output of the batch normalization layer in response to the input of the first data to the first machine learning model [Wang, pg. 3 “
PNG
media_image5.png
37
432
media_image5.png
Greyscale
PNG
media_image6.png
38
290
media_image6.png
Greyscale
”; pg. 2 “Our fully test-time adaptation setting uniquely requires only the model fθ and unlabeled target data xt for adaptation during inference.”. (note: this shows that the loss function for phase one is the entropy of the output produced by feeding the first (unlabeled) data through the model, which is the output shaped by the BN layer’s transformation)], and
the generating of the third machine learning model includes generating the third machine learning model by updating parameters of the fully connected layer corresponding to the first position in the second machine learning model while fixing the parameter of the batch normalization layer corresponding to the second position in the second machine learning model [Saito, pg. 2, Figure 2 “Step 1: we update the estimated prototypes in the classifier to maximize the entropy on the unlabeled target domain. Step 2: we minimize the entropy with respect to the feature extractor to cluster features around the estimated prototype.” (note: Saito's MME's step 1 shows updating the classifier C (FC layer/first position/fourth layer) using conditional entropy H on unlabeled target data Du (first data) while the feature extractor F (BN layers/second position/third layers) is not updated in that step, which is fixing BN while updating FC. The entropy formula H is computed over the classifier's output on the unlabeled data, which is the loss function including the entropy of the output of the fully connected layer in response to the input of the first data to the second machine learning model. Under BRI, the direction of entropy optimization (Saito's MME maximizes with respect to C, the claims minimize) is a design parameter within the same entropy objective, not a structural distinction that removes MME from the combination. Wang's Tent establishes that minimization is the operative direction for test-time adaptation, and a POSITA would apply the same entropy objective in the minimization direction for the FC update step as they do for the BN update step.)],
the updating of the parameters of the fully connected layer being performed by performing machine learning that minimizes the entropy of the output of the fully connected layer based on the loss function including the entropy of the output of the fully connected layer in response to the input of the first data to the second machine learning model [Saito, pg. 4, 3.2. Training Objectives, Equation 2 “
PNG
media_image7.png
105
645
media_image7.png
Greyscale
”. (note: the entropy formula H is computed over the classifier’s output on the unlabeled data)].
Wang and Saito are analogous art because they are from the same field of endeavor, specifically adapting deep neural network classifiers to a shifted target domain by entropy-based objectives computed on unlabeled target data. They are further reasonably pertinent to the same problem of improving classification accuracy on unlabeled target data without access to correct labels.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art (PHOSITA) to incorporate the teaching of Saito - alternately updating a second, complementary group of layers (the classifier layers at the first position) based on the entropy of the model's output on unlabeled data while holding the previously-updated layers fixed - into Wang's test-time entropy-minimization method that updates the normalization layers at the second position, so as to arrive at the two-stage generation of a second and then a third machine learning model.
The suggestion/motivation for doing so is provided by Wang itself, which teaches that adapting more of the model than the normalization affine parameters is desirable [Wang, pg. 9, Section 6 "Test-time adaptation could update more of the model, but the issue is to identify parameters that are both expressive and reliable"]. Saito supplies exactly such an additional, reliable adaptation stage by showing that the classifier (first-position) layers can be updated from the entropy of unlabeled target data in alternation with the feature layers to reduce error [Saito, pg. 1, Abstract].
Combining these known entropy-based adaptation steps according to their established methods would have yielded the predictable result of further reducing generalization error on the unlabeled target data, consistent with the KSR rationale of combining prior art elements according to known methods to yield predictable results (MPEP Section 2143.01(A)).
Per Claim 5, Wang-Saito discloses claim 1.
Wang further teaches each layer of the first one or more layers corresponding to the first position is a layer in which a number of parameters updated by machine learning is equal to or greater than a predetermined value, and each of the second one or more layers corresponding to the second position is a layer in which a number of parameters updated is less than the thresholds [Wang, pg. 4 “In practice, adapting y, β is efficient because they make up <1% of model parameters”. (note: these parameters are the low-parameter second position layers, while the remaining fixed parameters (>99%, including FC) are the high-parameter first position layers.)].
Per Claim 6, Wang-Saito discloses claim 1.
Wang further teaches the loss function is a loss function for updating a weight of a layer to be updated so as to minimize, as conditional entropy, entropy of an output of the layer which is updated by machine learning [Wang, pg. 3 “
PNG
media_image5.png
37
432
media_image5.png
Greyscale
PNG
media_image6.png
38
290
media_image6.png
Greyscale
”. (note: Wang minimizes Shannon conditional entropy of softmax outputs over unlabeled data)].
Claim 7 and 8 are substantially similar in scope and spirit to claim 1. Therefore, the rejection of claim 1 is applied accordingly. Saito further shows the article of manufacture being implemented by a method [Saito, pg. 1, Abstract “We empirically demonstrate the superiority of our method over many baselines, including conventional feature alignment and few-shot methods, setting a new state of the art for SSDA.”]. Saito further shows the article of manufacture being implemented by an apparatus [Saito, pg. 5, 4.1 Setup “All experiments are implemented in Pytorch [23]. We employ AlexNet [16] and VGG16 [34] pre-trained on ImageNet. To investigate the effect of deeper architectures, we use ResNet34 [14] in experiments on DomainNet.”].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sayed M Shah whose telephone number is (571)272-9406. The examiner can normally be reached Monday-Friday 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAYED MUNEER SHAH/Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights