DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Applicant’s submission filed on 03/13/2025 has been entered. Claims 13 and 14 were amended. Claims 7-12 were canceled. Claims 15-24 were added. Claims 1-6 and 13-24 are pending in the application.
Figure 1 should be designated by a legend such as --Prior Art-- because only that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claims 15-17 and 20-22 are objected to because of the following informalities: Claims 15 and 16 recite “wherein the processor being caused to separately determine … comprises being caused to” instead of “wherein separately determining … comprises;” “the processor being caused to separately determine … comprises being caused to” instead of “separately determining … comprises;” “determine, by each graphics processing unit” instead of “determining, by each graphics processing unit;” and “separately determine, according to the” instead of “separately determining, according to the.” Claim 17 recites “wherein the processor being caused to determine overall contrastive loss … comprises being caused to” instead of “wherein determining overall contrastive loss … comprises” and “determine the overall contrastive loss” instead of “determining the overall contrastive loss.” Claims 20 and 21 recite “wherein the computing device being caused to separately determine … comprises being caused to” instead of “wherein separately determining … comprises;” “the computing device being caused to separately determine … comprises being caused to” instead of “separately determining … comprises;” “determine, by each graphics processing unit” instead of “determining, by each graphics processing unit;” and “separately determine, according to the” instead of “separately determining, according to the.” Claim 22 recites “wherein the computing device being caused to determine overall contrastive loss … comprises being caused to” instead of “wherein determining overall contrastive loss … comprises” and “determine the overall contrastive loss” instead of “determining the overall contrastive loss.” Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract ideas without significantly more. The claim(s) recite(s) the step of “processing” in claims 1, 5-6, 13-14, 18-19, and 23-24, which amounts to a mental process; and the steps of “determining,” “determining,” and “determining” in claims 2-3, 15-16, and 20-21; and “determining” in claims 4, 17, and 22, which amount to a mathematical concept. See MPEP § 2106.04(a). These judicial exceptions are not integrated into a practical application because none is claimed. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exceptions. For example, “graphics processing units” and “a corresponding video memory of the graphics processing unit” are merely generic computer components.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 5-6, 13-15, 18-20, and 23-24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. ("Disco-clip: A distributed contrastive loss for memory efficient clip training." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) in view of Howson et al. (US 2022/0139022).
Regarding claim 1, Chen teaches/suggests: A method for calculating contrastive loss through multiple graphics processing units, comprising:
processing a feature of a target batch of samples through N graphics processing units, and each graphics processing unit separately processes a feature of at least one sample comprised in the target batch of samples (Chen §2.1 ¶1 “Suppose we have B text-image pairs as a batch sending to two encoders and the model is trained using N GPUs, with each GPU assigned b = B/N pairs. Here, we use TA, IA to denote all text and image features obtained from two encoders”);
Chen further teaches/suggests contrastive loss (Chen §2.1 ¶1 “The training is supervised by two contrastive losses, an image-to-text loss and a text-to-image loss”). Chen does not teach/suggests divided into M processing unit groups, wherein each processing unit group comprises one or more graphics processing units. Nor does Chen teach/suggest:
separately determining, by each processing unit group, a similarity matrix between features processed by a graphics processing unit comprised in the processing unit group, and storing the similarity matrix into a corresponding video memory of the graphics processing unit comprised in the processing unit group; and
separately determining, according to the similarity matrix stored in the corresponding video memory of the graphics processing unit comprised in each processing unit group, group contrastive loss corresponding to each processing unit group; and
determining overall contrastive loss according to the group contrastive loss corresponding to each processing unit group.
Howson, in view of Chen, teaches/suggests divided into M processing unit groups, wherein each processing unit group comprises one or more graphics processing units (Chen §2.1 ¶1 “the model is trained using N GPUs” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the GPUs of Chen to be arranged into groups as taught/suggested by Howson for performance. As such, Chen as modified by Howson teaches/suggests:
separately determining, by each processing unit group, a similarity matrix between features processed by a graphics processing unit comprised in the processing unit group, and storing the similarity matrix into a corresponding video memory of the graphics processing unit comprised in the processing unit group (Chen §4.1 ¶5 “the n-th GPU only needs to compute the following four terms … to compute two small similarity matrices of shape B/N × B instead of a full matrix of shape B × B which consumes a large amount of memory” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502”); and
separately determining, according to the similarity matrix stored in the corresponding video memory of the graphics processing unit comprised in each processing unit group, group contrastive loss corresponding to each processing unit group (Chen §4.1 ¶¶1-2 “we use In and Tn to denote the image and text features on the n-th GPU … the contrastive loss can be decomposed and rewritten as” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502”); and
determining overall contrastive loss according to the group contrastive loss corresponding to each processing unit group (Chen §7 ¶1 “By decomposing the loss computation into intra-GPU computation and inter-GPU computation, we can remove redundant memory consumption and use a all_reduce operator to collect inter-GPU gradients” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502” [In view of Chen and Howson, the overall contrastive loss would include intra-group computation and inter-group computation.]).
Regarding claim 2, Chen as modified by Howson teaches/suggests: The method according to claim 1, wherein separately determining, by each processing unit group, the similarity matrix between features processed by the graphics processing unit comprised in the processing unit group, and storing the similarity matrix into the corresponding video memory of the graphics processing unit comprised in the processing unit group comprises:
separately determining, by each graphics processing unit in each processing unit group, a first similarity matrix between features processed by the processing unit group, and storing the first similarity matrix into a corresponding video memory of the graphics processing unit (Chen §4.1 ¶5 “the n-th GPU only needs to compute the following four terms … to compute two small similarity matrices of shape B/N × B instead of a full matrix of shape B × B which consumes a large amount of memory” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502”); and
separately determining, according to the similarity matrix stored in the corresponding video memory of the graphics processing unit comprised in each processing unit group, group contrastive loss corresponding to each processing unit group comprises:
determining, by each graphics processing unit in each processing unit group according to the first similarity matrix stored in the corresponding video memory, first contrastive loss corresponding to the graphics processing unit (Chen §4.1 ¶¶1-2 “we use In and Tn to denote the image and text features on the n-th GPU … the contrastive loss can be decomposed and rewritten as” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502”); and
separately determining, according to the first contrastive loss corresponding to each graphics processing unit in each processing unit group, the group contrastive loss corresponding to each processing unit group (Chen §4.1 ¶¶1-2 “we use In and Tn to denote the image and text features on the n-th GPU … the contrastive loss can be decomposed and rewritten as” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502”).
The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Regarding claim 5, Chen as modified by Howson teaches/suggests: The method according to claim 1, wherein a quantity of graphics processing units comprised in each processing unit group is equal (Chen §2.1 ¶1 “the model is trained using N GPUs” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502”). The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Regarding claim 6, Chen as modified by Howson teaches/suggests: The method according to claim 1, wherein the target batch of samples comprises one or more of a text sample, a picture sample, a video sample, and an audio sample (Chen §2.1 ¶1 “Suppose we have B text-image pairs as a batch”).
Claims 13, 15, and 18-19 recite limitation(s) similar in scope to those of claims 1-2 and 5-6, respectively, and are rejected for the same reason(s). Chen as modified by Howson further teaches/suggests a non-transitory computer-readable storage medium (Chen §5.1 ¶2 “we use 64 NVIDIA A100 40GB GPUs for model training”).
Claims 14, 20, and 23-24 recite limitation(s) similar in scope to those of claims 1-2 and 5-6, respectively, and are rejected for the same reason(s). Chen as modified by Howson further teaches/suggests a memory and a processor (Chen §5.1 ¶2 “we use 64 NVIDIA A100 40GB GPUs for model training”).
Claim(s) 4, 17, and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. ("Disco-clip: A distributed contrastive loss for memory efficient clip training." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) in view of Howson et al. (US 2022/0139022) as applied to claims 1 and 13-14 above, and further in view of Zeng et al. (US 2024/0119725).
Regarding claim 4, Chen as modified by Howson does not teach/suggest: The method according to claim 1, wherein determining overall contrastive loss according to the group contrastive loss corresponding to each processing unit group comprises:
determining the overall contrastive loss according to a weighted average value of the group contrastive loss corresponding to each processing unit group.
Zeng, however, teaches/suggests a weighted average value (Zeng [0059] “find the weighted average to get the final contrast loss”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the overall contrastive loss of Chen as modified by Howson to be a weighted average as taught/suggested by Zeng to give weight to each group contrastive loss. As such, Chen as modified by Howson and Zeng teaches/suggests:
determining the overall contrastive loss according to a weighted average value of the group contrastive loss corresponding to each processing unit group (Chen §7 ¶1 “By decomposing the loss computation into intra-GPU computation and inter-GPU computation, we can remove redundant memory consumption and use a all_reduce operator to collect inter-GPU gradients” Howson [0103] “the processing logic 420 is taken to have nine processor cores arranged into three processor groups 502” Zeng [0059] “find the weighted average to get the final contrast loss”).
Claims 17 and 22 recites limitation(s) similar in scope to those of claim 4, and are rejected for the same reason(s).
Allowable Subject Matter
Claims 3, 16, and 21 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101 set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The limitations “separately determining, by each graphics processing unit in each processing unit group, a second similarity matrix between a feature processed by the graphics processing unit and a feature processed by the processing unit group” and “determining, by each graphics processing unit in each processing unit group according to the second similarity matrix stored in the corresponding video memory, second contrastive loss corresponding to the graphics processing unit,” taken as a whole, render the claims patentably distinct over the prior art.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2024/0331137 – contrastive loss
US 2024/0394592 – contrastive loss
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANH-TUAN V NGUYEN whose telephone number is 571-270-7513. The examiner can normally be reached on M-F 9AM-5PM ET. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JASON CHAN can be reached on 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANH-TUAN V NGUYEN/
Primary Examiner, Art Unit 2619
Prosecution Insights