DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is in response to the instant application filed on 05/17/2023. Claims 1-20 are pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/03/2024 s being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 6, 13, 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Sallee (US20210295105A1).
Regarding claim 1, Sallee teaches forming an autoencoder comprising a neural network as an encoder and a decoder (Title, Par 0020, An autoencoder can be trained to encode data into a latent representation and decode back to the original data space); for each training image of multiple training images, generating a latent representation of the training image by the encoder (Par 0025, The autoencoder 102 includes encoders 112, 114 that generate a latent feature vector for the content 104, 106); transforming each training image to produce transformed training images (Par 0023, The content 106 can include a transformed version of the content 104. The transform can include a rotation, scaling, viewpoint change, word change (without changing semantic meaning), color change, or the like); supplying information about the transformation and at least a part of the latent representation to the decoder to generate a decoder output for the training image (Par 0025, The latent feature vector for the content 104 includes static features 116 and dynamic features 118….The dynamic features 118 represent the transform of the object or other features of the content around the object, Par 0024, The autoencoder 102 includes a decoder 124, 126 that maps the feature vectors 116, 118 and 120, 122 to a reconstruction of the content 104, 106, respectively. The reconstruction of the content 104, 106 is illustrated as constructed content 108, 110 in FIG. 1); and adjusting the encoder and the decoder to reduce a loss between the transformed training images and the decoder outputs (Par 0029, A loss function used to train the autoencoder 102 can include one or more of a term representing the reconstruction loss between the content 104 and the constructed content 108, a term representing the reconstruction loss between the content 106 and the reconstructed content 110).
Regarding claim 6, Sallee teaches wherein the loss between the transformed training images and the decoder outputs further comprises a mean-square-error loss, a cosine distance, or a Kullback-Leibler divergence of the transformed training images and the decoder outputs (Par 0029, A loss function used to train the autoencoder 102 can include one or more of a term representing the reconstruction loss between the content 104 and the constructed content 108, a term representing the reconstruction loss between the content 106 and the reconstructed content 110, or a term representing a difference between the static features 116 and the static features 120. The loss can include a mean square error (MSE)).
Regarding claims 13 and 18, the claims cite essentially the same limitations as claim 1 but with the addition of a non-transitory computer-readable media and/or a system with computers and storage devices, which are taught by Sallee (Par 0065). Therefore, the claims are rejected for the same reasons.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2-5, 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Sallee as applied to claims 1 and 13 above, and further in view of He (Masked Autoencoders Are Scalable Vision Learners).
Regarding claim 2, Sallee teaches the method of claim 1 but fails to teach further comprising: masking the latent representation and supplying the masked latent representation to the decoder to generate the decoder output.
He teaches further comprising: masking the latent representation and supplying the masked latent representation to the decoder to generate the decoder output (Figure 1, During pre-training, a large random subset of image patches (e.g., 75%) is masked out. The encoder is applied to the small subset of visible patches. Mask tokens are introduced after the encoder, and the full set of encoded patches and mask tokens is processed by a small decoder that reconstructs the original image in pixels).
He and Sallee are analogous to the claimed invention because they are both in the field of autoencoders for visual learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the masked autoencoder system in He along with Sallee to “accelerate training (by 3× or more) and improve accuracy” (He Abs).
Regarding claim 3, Sallee in view of He teaches the method of claim 2, but Sallee fails to teach further comprising: subdividing each training image into a plurality of training image patches, wherein the latent representation comprises an encoding for each training image patch, wherein masking the latent representation further comprising replacing at least one of the plurality of training image patches by mask tokens.
He teaches further comprising: subdividing each training image into a plurality of training image patches, wherein the latent representation comprises an encoding for each training image patch, wherein masking the latent representation further comprising replacing at least one of the plurality of training image patches by mask tokens (Figure 1, During pre-training, a large random subset of image patches (e.g., 75%) is masked out. The encoder is applied to the small subset of visible patches. Mask tokens are introduced after the encoder, and the full set of encoded patches and mask tokens is processed by a small decoder that reconstructs the original image in pixels.)
Regarding claim 4, Sallee in view of He teaches the method of claim 3, but Sallee fails to teach further comprising randomly selecting the mask tokens. He teaches further comprising randomly selecting the mask tokens (Figure 1, a large random subset of image patches (e.g., 75%) is masked out).
Regarding claim 5, Sallee in view of He teaches the method of claim 4, but Sallee fails to teach further comprising adjusting the mask tokens, the encoder, and the decoder to reduce the loss between the transformed training images and the decoder outputs. He teaches further comprising adjusting the mask tokens, the encoder, and the decoder to reduce the loss between the transformed training images and the decoder outputs (Section 3, Our loss function computes the mean squared error (MSE) between the recon structed and original images in the pixel space. We compute the loss only on masked patches, similar to BERT [14]), Section 3, Each mask token [14] is a shared, learned vector, Section 4, Then we do supervised training to evaluate the representations with (i) end-to-end fine-tuning, end-to-end fine-tuning with learnable mask tokens implies both the autoencoder and mask tokens are fine-tunned).
Regarding claims 14-17, the claims cite essentially the same limitations as claims 2-5 and are rejected for the same reasons.
Claim(s) 7, 8, 19, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sallee as applied to claims 1 and 18 above, and further in view of DeTone (US20200097819A1).
Regarding claim 7, Sallee teaches the method of claim 1 but fails to teach wherein the transformation comprises a feature extraction of the training image followed by a homography transformation.
DeTone teaches wherein the transformation comprises a feature extraction of the training image followed by a homography transformation (Par 0003, A conventional approach to homography estimation includes two stages: corner estimation and robust homography estimation…. Since corners are not as reliable as man-made linear structures, the research community has put considerable effort into adding line features and more complicated geometries into the feature detection step, Par 0004, The method also includes perturbing at least one of the first set of corners to form a second set of corners. The second set of corners defines a modified subset of the image…... The method also includes generating a transformed image by applying the homography to the image).
DeTone and Sallee are analogous to the claimed invention because they both deal with image transformation using neural networks. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the system described in DeTone along with Sallee to help with disentangling two images by using “a single robust algorithm that, given a pair of images, returns the homography relating that pair” (DeTone Par 0003).
Regarding claim 8, Sallee teaches the method of claim 1 but fails to teach wherein the transformation is a homography transformation of the training image. DeTone teaches wherein the transformation is a homography transformation of the training image (Par 0004, The method also includes generating a transformed image by applying the homography to the image).
Regarding claims 19 and 20, the claims recite essentially the same limitations as claims 7 and 8 and are rejected for the same reasons.
Claim(s) 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Sallee as applied to claim 1 above, and further in view of Matsuo (Transform Invariant Auto-encoder).
Regarding claim 9, Sallee teaches the method of claim 1 but fails to teach wherein the information about the transformation is an encoding of hyper parameters of the transformation. Matsuo teaches wherein the information about the transformation is an encoding of hyper parameters of the transformation (Section IV, We define a transform parameter of an input I as a parameter representing a transform from the input I to the restored input D(E(I)).)
Matsuo is analogous to the claimed invention because it is in the field of using auto-encoders for visual learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have added the inference model of transform parameters in Matsuo to Sallee to achieve a model “that can infer shift parameters required to restore the input from the typical sub pattern” (Matsuo Abs).
Regarding claim 10, Sallee in view of Matsuo teaches the method of claim 9, further Matsuo teaches, which Sallee is silent on, further comprising generating the encoding of hyper parameters of the transformation by a further neural network (Section 4, Therefore, a pair of a transform invariant auto-encoder and the corresponding inference model of a transform parameter).
Regarding claim 11, Sallee in view of Matsuo teaches the method of claim 10, further Matsuo teaches, which Sallee is silent on, further comprising adjusting the further neural network, the encoder, and the decoder to reduce the loss between the transformed training images and the decoder outputs (Section 4, We propose the following cost function to train an inference model R of a transform parameter. We can achieve an inference model R of a transform parameter by minimizing Cpar(R), cost includes encoder and decoder).
Regarding claim 12, Sallee in view of Matsuo teaches the method of claim 11, further Matsuo teaches, which Sallee is silent on, wherein the neural network is a convolutional neural network, a vision transformer network, or a multi-layer perceptron-based neural network (Section 5, The inference model consisted of a three-layer fully connected NN).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NATNAEL A ASEGDEW whose telephone number is (571)270-0407. The examiner can normally be reached 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATNAEL A ASEGDEW/Examiner, Art Unit 2122
/MICHAEL H HOANG/PRIMARY EXAMINER, Art Unit 2122