DETAILED ACTION
This action is in response to the application filed on 09/22/2023. Claims 1-20 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7, 13; 14, 19, 20 are rejected under 35 U.S.C. 103 as being anticipated by Gao et al. (“Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup” [2021], hereinafter “Gao”) in view of Savchenko et al. (“Favorable Random Gradients for Optimization of Deep Neural Networks” [2018], hereinafter “Savchenko”).
Regarding Claim 1,
Gao discloses A method for determining update gradient for a contrastive learning model, comprising: (Gao [Abstract]; “This paper intro duces a gradient caching technique that decouples backpropagation between contrastive loss and the encoder, removing encoder backward pass data dependency along the batch dimension. As a result, gradients can be computed for one subset of the batch at a time, leading to almost constant memory usage.”)
determining a gradient factor of a first type for the contrastive learning model based on a first group of training data and a second group of training data for training the contrastive learning model …; (Gao [Section 3.3];
PNG
media_image1.png
616
299
media_image1.png
Greyscale
PNG
media_image2.png
351
309
media_image2.png
Greyscale
wherein each of the sub-batches having gradient vectors calculated from their representations thus reads on such gradient vectors being gradient factors associated with a plurality of sub-batches (different types of gradient factors); wherein the sets S and T representative of two classes of training data read on a first and second group of training data for training the contrastive learning mode; wherein the step 2 of determining the gradient factors for each of the sub-batches comprising data S and T thus reads on gradient factors computed for the contrastive learning model based on a first and second group of training data)
determining, in a first stage of the training process, a gradient factor of a second type associated with the first group of training data based on the contrastive learning model, the gradient factor of the second type associated with the first group of training data being used for backpropagation during the training process;
(Gao [Section 3.3];
PNG
media_image1.png
616
299
media_image1.png
Greyscale
PNG
media_image2.png
351
309
media_image2.png
Greyscale
wherein each of the sub-batches having gradient vectors calculated from their representations thus reads on such gradient vectors being gradient factors associated with a plurality of sub-batches (different types of gradient factors) thus a gradient factor associated with a second sub-batch iteration type; wherein the sets S and T representative of two classes of training data read on a first and second group of training data for training the contrastive learning mode; wherein the step 2 of determining the gradient factors for each of the sub-batches comprising data S and T thus reads on gradient factors computed for the contrastive learning model based on a first and second group of training data; wherein step 3’s gradient accumulation comprising back propagation through an encoder using the determined gradient factors across the sub-batches thus reads on such determined gradient factors of the second type being used in backpropagation)
and obtaining gradient for updating the contrastive learning model based on the gradient factor of the first type and the gradient factor of the second type associated with the first group of training data
(Gao [Section 3.3];
PNG
media_image1.png
616
299
media_image1.png
Greyscale
PNG
media_image2.png
351
309
media_image2.png
Greyscale
wherein the final accumulated gradient based on the accumulation of gradients associated with the sub-batches (plurality of gradient factor types) thus reads an obtained gradient for updating the contrastive learning model based on the gradient factors of the first and second types associated with the first group of training data (sample data set S))
Gao fails to explicitly disclose but Savchenko discloses the gradient factor of the first type being not used for backpropagation during a training process of the contrastive learning model (Savchenko [Section I Page 352 Line 12]; “Moreover, only the favorable random gradients are considered for the additional optimization. To decide whether the random gradients are favorable or not, the effect of introducing them to a given mini-batch at the current sub-step has to be estimated on the network results. If by using these random gradients for the additional optimization sub-step, the training loss is decreasing (or at least not increasing) on the current mini-batch, the random gradients are considered as favorable, and this optimization sub-step is viewed as having a positive effect on the current state of the model. Otherwise, the random gradients have a negative impact on the optimization process, and the additional optimization sub-step will be ignored “
Savchenko [Abstract]; “Otherwise, if the random gradients affect poorly, only the standard training step with backpropagation is performed on the given mini-batch.” wherein the gradients that downgrade model output (gradient factor of a first type of a plurality of types) are not selected for backpropagation training of the contrastive learning model)
It would have been obvious to modify Gao’s method of contrastive learning comprising gradient factors determined for a plurality of sub-batches to use Savchenko’s method of selectively excluding certain gradient factors associated with certain sub-batch types during backpropagation training of its contrastive learning model. One would have been motivated to do so in order to “not downgrade the network result on a training mini-batch” (Savchenko [Abstract]).
Regarding Claim 7,
Gao/Savchenko teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Gao/Savchenko already discloses determining, in a second stage after the first stage of the training process, a gradient factor of the second type associated with the second group of training data based on the contrastive learning model and the second group of training data; and wherein obtaining the gradient further comprises: updating the gradient based on the gradient factor of the first type and the gradient factor of the second type associated with the second group of training data (Gao [Section 3.3];
PNG
media_image1.png
616
299
media_image1.png
Greyscale
PNG
media_image2.png
351
309
media_image2.png
Greyscale
wherein step 2 being after the initial forward pass obtaining representations performed in step 1 thus reads on a second stage after the first stage of the training process; wherein step 2 comprising gradient factors of the plurality of sub-batches associated with the second group of training data (sub-batches comprising first and second groups S and T thus reads on an implicit association) thus reads on determining gradient factors of the second type based on the contrastive learning model and the second group of training data; wherein step 3 of gradient accumulation obtaining the final accumulated gradient through accumulation of the gradient factors of the plurality of sub-batches thus reads on updating the gradient based on the gradient factors of the first and second types associated with the second group of training data)
Regarding Claim 13,
Gao/Savchenko teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Gao/Savchenko already discloses training the contrastive learning model based on the gradient; and determining an association relationship between data in a sample that is to be processed using the trained contrastive learning model (Gao [Section 4; Figure 1; Table 1];
PNG
media_image3.png
233
614
media_image3.png
Greyscale
wherein figure 1 demonstrating the training speed of the gradient accumulation contrastive learning model thus implicitly reads on training the contrastive learning model based on the (accumulated) gradient; wherein table 1 demonstrating the tested accuracy results of the gradient accumulation contrastive learning model on the dense passage retriever (DPR) dataset comprising open-domain question answering representations thus reads on the contrastive learning model processing open-domain question answering queries to determine association relationships in the text)
Claims 14, 19 recite an electronic device comprising at least one processing unit and at least one memory coupled to the at least one processing unit storing instructions to cause the device to perform the exact method of Claims 1, 13 respectively. Thus, Claims 14, 19 are rejected for reasons set forth in the rejection of Claim 1, 13 respectively.
Claim 20 recites a non-transitory computer-readable storage medium storing a computer program to perform the method of Claim 1. Thus, Claim 20 is rejected for reasons set forth in the rejection of Claim 1.
Claims 2-6, 9-12; 15-18 are rejected under 35 U.S.C. 103 as being anticipated by Gao et al. (“Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup” [2021], hereinafter “Gao”) in view of Savchenko et al. (“Favorable Random Gradients for Optimization of Deep Neural Networks” [2018], hereinafter “Savchenko”) in view of Peng et al. (“Balanced Multimodal Learning via On-The-Fly Gradient Modulation” [2022], hereinafter “Peng”).
Regarding Claim 2,
Gao/Savchenko teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Gao/Savchenko only discloses wherein training data in the first group of training data and the second group of training data comprises: data of a first [class], data of a second [class] (Gao [Section 3.1];
PNG
media_image4.png
269
301
media_image4.png
Greyscale
wherein Gao/Savchenko only discloses the training data of the first group of training data S and the second training data T being of a first and second class, which is notably distinct from a first and second modality. Gao/Savchenko does not explicitly recite that such classes must be of different modalities)
Gao/Savchenko does not explicitly disclose but Peng discloses wherein training data in the first group of training data and the second group of training data comprises: data of a first modality, data of a second modality, and a label representing an association relationship between the data of the first modality and the data of the second modality (Peng [Section 3.1];
PNG
media_image5.png
192
329
media_image5.png
Greyscale
wherein xia and xib read on first and second groups of training data each of a separate audio and visual modality; wherein the yi label representative of a categorical association relationship between xia and xib thus reads on an association relationship between the data of the first modality and the data of the second modality).
It would have been obvious to modify Gao’s first and second groups of training data associated with first and second classes to be of separate first and second modalities alongside an association label in a similar fashion to Peng. One would have been motivated to do so because “Multimodal data usually provides more views compared with uni-modal one, accordingly learning with multimodal data should match or outperform the uni-modal case” (Peng [Introduction Paragraph 2]).
Regarding Claim 3,
Gao/Savchenko/Peng teaches the method of Claim 2 (and thus the rejection of Claim 2 is incorporated). Gao/Savchenko/Peng already discloses wherein determining the gradient factor of the first type comprises: determining a loss function associated with the training data; determining, using the contrastive learning model, a first feature for the data of the first modality and a second feature for the data of the second modality, respectively; and determining the gradient factor of the first type based on the loss function, the first feature and the second feature (Gao [Page 2 Column 2];
PNG
media_image6.png
190
306
media_image6.png
Greyscale
PNG
media_image7.png
631
298
media_image7.png
Greyscale
Gao [Section 3.1];
PNG
media_image8.png
262
310
media_image8.png
Greyscale
wherein the computation of the contrastive learning loss through partial derivatives comprising a denoted normalized similarity between encoded features representations of the sample and target thus reads on determining a loss function associated with the training data; wherein the encoded representations of the S and T data read on determined features of the data;
Gao [Section 3.3 Step 2];
PNG
media_image9.png
261
297
media_image9.png
Greyscale
wherein the computed contrastive loss being dependent on the first and second features thus reads on the overall determined gradient factor obtained through the contrastive loss function to be based on the loss function, the first feature, and the second feature)
Regarding Claim 4,
Gao/Savchenko/Peng teaches the method of Claim 3 (and thus the rejection of Claim 3 is incorporated). Gao/Savchenko/Peng already discloses wherein determining the loss function comprises: determining a predicted value associated with the training data using the contrastive learning model; and determining the loss function based on a difference between the predicted value and the label in the training data (Gao [Section 3.2];
PNG
media_image10.png
638
314
media_image10.png
Greyscale
wherein the backpropagation process’ backward pass comprises a computed loss function determined through normalized similarity of the predicted sample representations against target representations thus reads on determining the loss function through a difference between predicted data and labels in the training data (since determined similarity between the S and T data is interpretable as determined differences between the S anchor and T target data))
Regarding Claim 5,
Gao/Savchenko/Peng teaches the method of Claim 4 (and thus the rejection of Claim 4 is incorporated). Gao/Savchenko/Peng already discloses wherein determining the first feature and the second feature comprises: determining the first feature and the second feature based on a first encoder and a second encoder in the contrastive learning model, respectively, the first encoder describing an association relationship between the data of the first modality and the feature of the data of the first modality, and the second encoder describing an association relationship between the data of the second modality and the feature of the data of the second modality (Peng [Section 3.1 Paragraph 1];
PNG
media_image11.png
122
324
media_image11.png
Greyscale
wherein a first and second encoder associated with the first data of the first modality and second data of the second modality respectively are used to determine extracted first and second features of the first and second modalities; wherein the two encoders extracting audio and visual features of the audio and visual data implicitly reads on the two encoders describing respective association relationships between the data and its features (since mapping the raw data of different modalities into feature space naturally reads on mapped associations))
Regarding Claim 6,
Gao/Savchenko/Peng teaches the method of Claim 5 (and thus the rejection of Claim 5 is incorporated). Gao/Savchenko/Peng already discloses wherein the gradient factor of the second type associated with the first group of training data comprises a feature of the data of the first modality and a feature of the data of the second modality in the training data of the first group of training data, and the method further comprises: determining the feature of the data of the first modality and the feature of the data of the second modality in the training data based on the first encoder and the second encoder, respectively (Peng [Section 3.1 Paragraph 1];
PNG
media_image11.png
122
324
media_image11.png
Greyscale
wherein a first and second encoder associated with the first data of the first modality and second data of the second modality respectively are used to determine extracted first and second features of the first and second modalities
Gao [Section 3.3 Step 2];
PNG
media_image12.png
261
299
media_image12.png
Greyscale
wherein the gradient factors comprising ui and vi which are both computed as partial derivatives with respect to already encoded representations f(si) and g(ti) thus reads on gradient factors of the second sub-batch type comprising features of the data. Notably, the encoder not being included in this backward pass gradient computation does not meaningfully remove from the already encoded representations f(si) and g(ti) which are used; thus, such features are inherently based on the first and second encoders respectively)
Regarding Claim 9,
Gao/Savchenko/Peng teaches the method of Claim 2 (and thus the rejection of Claim 2 is incorporated). Gao/Savchenko/Peng already discloses obtaining the first group of training data by: obtaining a positive sample of training data from a training dataset for the contrastive learning model; determining a first data of the first modality and a second data of the second modality in the positive sample; (Gao [Section 3.1];
PNG
media_image13.png
267
303
media_image13.png
Greyscale
wherein the first group of training data S being positively related to elements tr in T thus reads on such related S and T elements within the group being positive samples for the contrastive learning model; wherein the positive sample comprising related elements S and T of first and second modalities (Peng disclosing S and T being of two classes of separate modalities) thus reads on a determined first data of the first modality and a second data of the second modality in the positive sample)
selecting a third data of the second modality from a data space of the second modality, the third data being different from the second data; and generating a negative sample in the first group of training data based on the first data of the first modality and the third data of the second modality (Gao [Section 3.1];
PNG
media_image13.png
267
303
media_image13.png
Greyscale
wherein the rest of the random samples in T reads on a selected third data of the second modality from a data space of the second modality different from the second data itself; wherein the zero or more specially sampled hard negatives generated for elements si thus read on generated negative samples in the first group of training data based on the first data of the first modality and the third data of the second modality since such negative samples must not be related to T samples (including the in-batch negatives) and are derived from the initial first data set S of the first modality)
Regarding Claim 10,
Gao/Savchenko/Peng teaches the method of Claim 2 (and thus the rejection of Claim 2 is incorporated). Gao/Savchenko/Peng already discloses wherein the contrastive learning model describes a forward association relationship from the data of the first modality to the data of the second modality (Gao [Section 3.1];
PNG
media_image13.png
267
303
media_image13.png
Greyscale
wherein the distance-based relationship between encoded representations of the data of the first and second modalities thus reads on such distance being equivalently computable either in forward association from f(s) to g(t) or backward association from g(t) to f(s), thus such distance-based relationships are admissible as describing both a forward and backward association)
Regarding Claim 11,
Gao/Savchenko/Peng teaches the method of Claim 10 (and thus the rejection of Claim 10 is incorporated). Gao/Savchenko/Peng already discloses wherein the contrastive learning model further describes a backward association relationship from the data of the second modality to the data of the first modality (Gao [Section 3.1];
PNG
media_image13.png
267
303
media_image13.png
Greyscale
wherein the distance-based relationship between encoded representations of the data of the first and second modalities thus reads on such distance being equivalently computable either in forward association from f(s) to g(t) or backward association from g(t) to f(s), thus such distance-based relationships are admissible as describing both a forward and backward association)
Regarding Claim 12,
Gao/Savchenko/Peng teaches the method of Claim 2 (and thus the rejection of Claim 2 is incorporated). Gao/Savchenko/Peng already discloses wherein the first modality comprises any of a plurality of modalities: image, text, video, audio, and the second modality comprises a further one of the plurality of modalities (Peng [Section 3.1];
PNG
media_image5.png
192
329
media_image5.png
Greyscale
wherein xia and xib read on first and second groups of training data each of a separate audio and visual modality)
Claims 15-18 recite an electronic device comprising at least one processing unit and at least one memory coupled to the at least one processing unit storing instructions to cause the device to perform the exact method of Claims 2-5 respectively. Thus, Claims 15-18 are rejected for reasons set forth in the rejection of Claim 2-5 respectively.
Claim 8 is rejected under 35 U.S.C. 103 as being anticipated by Gao et al. (“Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup” [2021], hereinafter “Gao”) in view of Savchenko et al. (“Favorable Random Gradients for Optimization of Deep Neural Networks” [2018], hereinafter “Savchenko”) in view of Knodt (“Structural Dropout for Model With Compression” [2022], hereinafter “Knodt”)
Regarding Claim 8,
Gao/Savchenko teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Gao/Savchenko already discloses and determining the gradient factor of the first type and the gradient factor of the second type based on a network node (Gao [Section 3.3];
PNG
media_image1.png
616
299
media_image1.png
Greyscale
wherein the determined gradient factors obtained in step 2 for each of the plurality of sub-batch iterations thus reads on a determined gradient factor of a first and second type based on a network node in the contrastive learning model)
Gao/Savchenko does not explicitly disclose but Knodt discloses determining a discard rule associated with the first group of training data, the discard rule specifying a group of network nodes in the contrastive learning model that should be discarded during the training process; a network node other than the group of network nodes in the contrastive learning model (Knodt [Figure 1];
PNG
media_image14.png
261
564
media_image14.png
Greyscale
).
It would have been obvious to modify the network architecture of Gao/Savchenko to introduce Knodt’s discard rule to discard certain network nodes during training and thus determine a pruned set of network nodes other than the initial network nodes in the contrastive learning model predictably resulting in performing Gao/Savchenko’s method of determining gradient factors of first and second types on Knodt’s network nodes other than the group of network nodes derived through Knodt’s discard rule thereby determining the gradient factor of the first type and the gradient factor of the second type based on a network node other than the group of network nodes in the contrastive learning model. One would have been motivated to do so because “to prevent networks from overfitting by preventing neurons from strongly entangling information, preventing memorization of training data to specific outputs” (Knodt page 3 paragraph 2).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“Systems and Methods for Contrastive Learning of Visual Representations” (US 20210319266 A1) which discloses a method of training a contrastive learning multi-modal model comprising, in part, gradient updates.
“Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning” (US 20230102544 A1) which discloses a method of computing a contrastive learning model loss function based on a plurality of gradient factors.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN J KIM whose telephone number is (571) 272-0523.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN J KIM/Examiner, Art Unit 2141
/ANDREW L TANK/Primary Examiner, Art Unit 2141
Prosecution Insights