Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Claim limitation “an image data augmenter”, “a sample checker”, and “a loss calculator” in claims 9 – 16 has/have been interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because it uses/they use a generic placeholder “augmenter”, “checker” and “calculator” coupled with functional language “generate”, “classify”, and “calculate” without reciting sufficient structure to achieve the function. Furthermore, the generic placeholder is not preceded by a structural modifier.
Since the claim limitation(s) invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, claim(s) 9 – 16 has/have been interpreted to cover the corresponding structure described in the specification that achieves the claimed function, and equivalents thereof.
A review of the specification shows that the following appears to be the corresponding structure described in the specification for the 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph limitation: the Examiner considered above units as part of processor and memory disclosed in Fig, 1 and 2.
If applicant wishes to provide further explanation or dispute the examiner’s interpretation of the corresponding structure, applicant must identify the corresponding structure with reference to the specification by page and line number, and to the drawing, if any, by reference characters in response to this Office action.
If applicant does not intend to have the claim limitation(s) treated under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may amend the claim(s) so that it/they will clearly not invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, or present a sufficient showing that the claim recites/recite sufficient structure, material, or acts for performing the claimed function to preclude application of 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
For more information, see MPEP § 2173 et seq. and Supplementary Examination Guidelines for Determining Compliance With 35 U.S.C. 112 and for Treatment of Related Issues in Patent Applications, 76 FR 7162, 7167 (Feb. 9, 2011).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 2, 9, 10 and 17 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by He et al. (“Momentum Contrast for Unsupervised Visual Representation Learning”, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)), hereinafter referred as He.
Regarding claim 17, He discloses an apparatus for self-supervised contrastive learning (SSCL) (page 9729, for inherency, implement PyTorch/GPU neural-network training necessarily using processing hardware and memory to execute instructions; abstract, section 3), comprising:
memory in which a non-index object image necessary for learning (page 9729, for inherency, implement PyTorch/GPU neural-network training necessarily using processing hardware and memory to execute instructions), an expression vector corresponding to the non-index object image (page 9728, queue of keys; section 4, augmentation on images loaded from datasets such as ImageNet and Instagram. The images are necessarily stored before loading; image class labels are not used in its unsupervised training), and
a program for SSCL based on the non-index object image are stored (page PyTorch implemented Algorithm 1); and
a processor configured to execute the program stored in the memory, wherein by executing the program (page 9729, for inherency, implement PyTorch/GPU neural-network training necessarily using processing hardware and memory to execute instructions), the processor
generates a plurality of different view images by applying at least one conversion scheme for SSCL to the non-index object image (page 9729, Algorithm 1: (generating)
x_q = aug (x) # a randomly augmented version
x_ k = aug (x ) # another randomly augmented version)
by applying at least one conversion scheme for SSCL (section 3.3, Technical details: “a randomly resized image, and then undergoes random color jittering, random horizontal flip, and random grayscale conversion”) to the non-index object image (section 4, augmentation on images loaded from datasets such as ImageNet and Instagram. The images are necessarily stored before loading; image class labels are not used in its unsupervised training);
generates expression vectors (section 3.1, last para., generating vector q and k; also section 3.3) by alternately inputting the plurality of view images to a backbone network (section 3.1, last para., input xq and xk to query encoder network (backbone network) and key encoder (momentum network)) and a momentum network initialized to have an identical network parameter values (page 9729, Algorithm 1: f_k.params = f_q.params # initialize);
classifies the plurality of view images into a positive sample and a negative sample (section 3.1, 3.3, Algorithm 1; positive, negative) based on an anchor vector selected in a batch of the expression vectors (section 3.1, 3.3, Algorithm 1; encoded query q as the anchor against which a positive key k+ and negative key k- are compared);
calculates a loss value of a loss function based on a distance value between the anchor vector and the expression vector (section 3.1, 3.3, Algorithm 1; equation (1), loss function, similarity measure by dot product represents the distance or proximity between the vectors in representation space), and
updates parameters of the backbone network and the momentum network by reversely propagating the loss value of the loss function to the backbone network (section 3.3, Algorithm 1, loss.backward(), update query network and key network; Fig. 2(a), “The encoders for computing the query and key representations are updated end-to-end by back-propagation (the two encoders can be different)”).
Regarding claims 1 and 9, they are corresponding to claim 17, thus, they are interpreted and rejected for a same reason set forth for claim 17.
Regarding claim 2 (depends on claim 1), He discloses the method wherein the classifying of the plurality of view images into the positive sample and the negative sample based on the anchor vector selected in the batch of the expression vectors comprises
selecting, as the anchor vector, an expression vector that is first generated by the backbone network, among the view images generated by the identical original image (section 3.1, 3.3, Algorithm 1; select query q generated by the backbone network as the anchor against which a positive key k+ and negative key k- are compared).
Regarding claim 10 (depends on claim 9), He discloses the apparatus wherein the sample checker selects, as the anchor vector, an expression vector that is first generated by the backbone network, among the view images generated by the identical original image (section 3.1, 3.3, Algorithm 1; select query q generated by the backbone network as the anchor against which a positive key k+ and negative key k- are compared).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 7, 11 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over He in view of Liu et al. (US Patent Application Publication 2024/0177320), hereinafter referred as Liu.
Regarding claim 3 (depends on claim 1), He discloses the method wherein the classifying of the plurality of view images into the positive sample and the negative sample based on the anchor vector selected in the batch of the expression vectors comprises:
calculating a distance between the expression vector and the anchor vector corresponding to a group of the view images generated from an original image identical with an anchor of the anchor vector (section 3.1, 3.3, Algorithm 1; encoded query q as the anchor against which a positive key k+ and negative key k- are compared);
comparing the calculated distance (section 3.1, 3.3, Algorithm 1; equation (1), loss function, similarity measure by dot product represents the distance or proximity between the vectors in representation space); and
classifying the view images within the group into the positive sample and the negative sample based on results of the comparison (section 3.1, 3.3, Algorithm 1; equation (1), loss function, similarity measure by dot product represents the distance or proximity between the vectors in representation space).
However, He fails to explicitly disclose the method wherein comparing the calculated distance with a preset positive sample reference.
However, in a similar field of endeavor Liu discloses a method for training neural network for image analysis (abstract). In addition, Liu discloses the method wherein comparing the calculated distance with a preset positive sample reference, and classifying the images within the group into the positive sample and the negative sample based on results of the comparison ([0033]).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to apply Liu’s threshold-based similarity screening to He’s learned query and key representation. The modification would have predictably prevented highly dissimilar or semantically inconsistent augmentations from being assigned as positive samples thereby improving the quality of the positive and negative training sets.
Regarding claim 7 (depends on claim 1), He discloses the method wherein the classifying of the plurality of view images into the positive sample and the negative sample based on the anchor vector selected in the batch of the expression vectors comprises:
calculating a distance between the expression vector and the anchor vector corresponding to view images converted from an original image different from the anchor vector (section 3.1, 3.3, Algorithm 1; encoded query q as the anchor against which a positive key k+ and negative key k- are compared);
comparing the calculated distance (section 3.1, 3.3, Algorithm 1; equation (1), loss function, similarity measure by dot product represents the distance or proximity between the vectors in representation space); and
classifying the view images into the positive sample and the negative sample based on results of the comparison (section 3.1, 3.3, Algorithm 1; equation (1), loss function, similarity measure by dot product represents the distance or proximity between the vectors in representation space).
However, He fails to explicitly disclose the method wherein comparing the calculated distance with a preset positive sample reference.
However, in a similar field of endeavor Liu discloses a method for training neural network for image analysis (abstract). In addition, Liu discloses the method wherein comparing the calculated distance with a preset positive sample reference, and classifying the images within the group into the positive sample and the negative sample based on results of the comparison ([0033]).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to apply Liu’s threshold-based similarity screening to He’s learned query and key representation. The modification would have predictably prevented highly dissimilar or semantically inconsistent augmentations from being assigned as positive samples thereby improving the quality of the positive and negative training sets.
Regarding claim 11, it is corresponding to claim 3, thus, it is interpreted and rejected for a same reason set forth for claim 3.
Regarding claim 15 (depends on claim 9), He discloses the apparatus wherein the sample checker
calculates a distance between the expression vector and the anchor vector corresponding to view images converted from an original image different from the anchor vector (section 3.1, 3.3, Algorithm 1; encoded query q as the anchor against which a positive key k+ and negative key k- are compared),
compares the calculated distance (section 3.1, 3.3, Algorithm 1; equation (1), loss function, similarity measure by dot product represents the distance or proximity between the vectors in representation space),
classifies the view image as the positive sample (section 3.1, 3.3, Algorithm 1; equation (1), loss function, similarity measure by dot product represents the distance or proximity between the vectors in representation space), and
classifies the view image as the negative sample (section 3.1, 3.3, Algorithm 1; equation (1), loss function, the rest are classified as non-similar class (negative samples)
However, He fails to explicitly disclose the method wherein comparing the calculated distance with a preset positive sample reference; classifies the view image as the positive sample when the view image is within the positive sample reference; classifies the view image as the negative sample when the view image is not within the positive sample reference.
However, in a similar field of endeavor Liu discloses a method for training neural network for image analysis (abstract). In addition, Liu discloses the method wherein comparing the calculated distance with a preset positive sample reference, and classifies the view image as the positive sample when the view image is within the positive sample reference; classifies the view image as the negative sample when the view image is not within the positive sample reference ([0033]).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to apply Liu’s threshold-based similarity screening to He’s learned query and key representation. The modification would have predictably prevented highly dissimilar or semantically inconsistent augmentations from being assigned as positive samples thereby improving the quality of the positive and negative training sets.
Claim(s) 8 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over He in view of Liu, and Lin et al. (“SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling”, arXiv:2108:09412v1 [cs.LG], 21 Aug 2021), hereinafter referred as Lin.
Regarding claim 8 (depends on claim 7), He discloses the method wherein the classifying of the plurality of view images into the positive sample and the negative sample based on the anchor vector selected in the batch of the expression vectors comprises:
selecting a group of view images having an original image different from the anchor vector in the batch of the expression vectors (section 3.1, 3.3, Algorithm 1; equation (1), measure and classify negative samples by comparing with the anchor vector); and
updating types of samples for the group of the view images with the negative sample (section 3.1, 3.3, Algorithm 1; equation (1), classifies negative samples).
However, He fails to explicitly disclose the method wherein updating all types of samples for the group of the view images with the negative sample when at least one negative sample is present in a sample type of an expression vector corresponding to the selected group of the view images.
However, in a similar field of endeavor Lin discloses a new framework dubbed SemiFed that unifies two dominant approaches for semi-supervised learning: consistency regularization and pseudo-labeling (abstract). In addition, Lin discloses wherein accepting a pseudo-label for an image only when multiple evaluations/models provide high-confidence predictions and agree on the label. Thus, disagreement by even one evaluation prevents the group-level positive designation (abstract, page 2, col. 1).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to apply the agreement-based reliability criterion of Lin to He’s multiple augmented views so that views originating from one image receive a consistent sample designation, thereby preventing an image group containing a dissimilar or unreliable view from being erroneously used as positive group.
Regarding claim 16, it is corresponding to claim 8, thus, it is interpreted and rejected for a same reason set forth for claim 8.
Allowable Subject Matter
Claims 4 – 6 and 12 – 14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 4 (similar as to claim 12) further discloses limitations of “counting numbers of the positive samples and the negative samples when sample classification for the view images within the group, which are converted from the identical original image is completed; and selecting an anchor vector that is currently selected as the anchor vector that is the reference when the number of positive samples is greater than the number of negative samples as a result of the counting.”
The closest reference Li et al. (“Multi-Augmentation Contrastive Learning as Multi-Objective Optimization for Graph Neural Networks”, Pacific-Asia Conference on Knowledge Discovery and Data Mining, May 23, 2023) discloses “
To investigate the effect of the number of objectives, we change the size of T. For the case where the number is 2, PGCL degenerates, and we report the GraphCL accuracy since it elaborately handpicks the best augmentation pair for every dataset. For more augmentations cases, we randomly select augmentations from IPDSM as T and report the accuracy. Table 3 shows the advantages of combining more objectives. As the number of objectives increases within a certain range, PGCL's performance improves. It is rational that combining more augmentation makes the model more robust, thus generating a better representation.” (page 504)
However, neither He, nor Liu and Li, alone or in a combination, teaches claimed limitations in claim 4 (similar as to claim 12) in combination with all the claims it depended upon.
Claims 5 – 6 and 13 – 14 are dependent claims for claims 4 and 12, respectively, thus, they are objected accordingly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QIAN YANG whose telephone number is (571)270-7239. The examiner can normally be reached on Monday-Thursday 8am-6pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QIAN YANG/
Primary Examiner, Art Unit 2677
