DETAILED ACTION
This second non-final office action is responsive to application 17/708,214 as filed 06 Nov 2025.
Claim status is currently pending for claims 1, 4, 6-7, 11, 14, 16, 18 and 21; independent claims are 1, 11 and 21; newly presenting claim 21 and all other pending claims being currently amended, canceled claims are 2-3, 5, 8-10, 12-13, 15, 17 and 19-20.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
As required by MPEP 609(c), the applicant’s submissions of the Information Disclosure Statement dated 09/03/22 is acknowledged by the examiner and the cited references have been considered in the examination of the claims now pending. As required by MPEP 609 C(2), a copy of the PTOL-1449 initialed and dated by the examiner is attached to the instant office action.
Response to Remarks
This application has been transferred within the office, prior case history has been reviewed along with the remarks as filed 11/06/25, and the office action has been made non-final. In as much as any prior rejection or objection is not maintained herein, it is considered to be withdrawn. Updated search and consideration identifies new prior arts, and new issues are identified regarding §112 and §101 as detailed below.
Claim Objections
Claim 21 is objected to because of following informality: final limitation “(h3)” should read “(f3)” since there is no earlier limitation of (h) or preceding (g). Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1, 11, 18 and 21 and their dependents are rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention. Particularly, insufficient antecedent basis is provided for the following limitations:
Claim 1 limitation (d) “the probability” occurs twice without introducing ‘a probability’
Claim 11 limitations “a training network” and “wherein said training network” which recite “the deep neural network.” However, the claim introduces first and second neural networks in the first two limitations. It is unclear if “the deep neural network” is referring to the first, the second or some other third neural network of the plurality of neural networks. For purposes of examination, “the deep neural networks” is interpreted to comprise any of the deep neural networks.
Claim 18 preamble depends from cancelled claim 16 and should depend from claim 11 or claim 14 similar to dependency of first grouping.
Claim 21 limitation (c) “the probability” occurs twice without introducing ‘a probability’
Claims depending from the above identified claims include the issue as noted and thus are rejected under 35 U.S.C. 112(b) as indefinite for lacking sufficient antecedent basis.
Claim Rejection – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
10. Claim 11 and its dependents are rejected under 35 U.S.C. 101 for being directed to non-statutory subject matter. Claim 11 is drawn to a computer-implemented system that can be a computer-software system as software per-se. It fails to assert that the software system is executed by processor or recorded on a non-transitory computer-readable medium so as to be structurally and functionally interrelated to the system and permit the function of the descriptive material to be realized. A computer program is merely a set of instructions capable of being executed by a computer. Without the computer hardware to realize the computer program's functionality, the computer program constitutes non-statutory functional descriptive material. Therefore, claim 11 recites non-statutory subject matter and its dependent claims include this deficiency without further remedy. Thus, claim 11 and its dependents are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter. See MPEP 2106.01.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over:
Li et al., US PG Pub No 2022/0156591A1 hereinafter LiJ (CoMatch) as evidenced by Li “CoMatch” arXiv:2011.11183v2 hereinafter LiJ183 (same author), in view of
Lienen et Hullermeier, “Credal Self-Supervised Learning” hereinafter Lienen (arXiv: 2106.11853v1), in view of
Li et al., “Contrastive Clustering” hereinafter LiY (arXiv: 2009.09687v1).
With respect to claim 1, LiJ teaches:
A computer implemented method for training a deep convolutional neural network for recognizing images using unlabeled images {LiJ proposes “CoMatch” Alg.1 – Fig 7, and further shows CNN Fig 1, [0019] “co-training framework includes a CNN” e.g. ResNet encoder [0068] and inputs unlabeled images Figs 2-3:202, [0078] “CoMatch provides a better backbone for object detection”}, comprising:
(a) electronically generating a weak augmentation version and a strong augmentation version of an input image, wherein the input image is unlabeled {LiJ Fig 6:604 “Generate a weakly augmented sample, a first strongly augmented sample…augmented sample from an unlabeled sample” samples of unlabeled images shown Fig 3:202 with 204,06 Augw & Augs augmented weak and strong versions, and described e.g. [022-23], [0041] “weakly augmented unlabeled sample 201, and strongly-augmented unlabeled samples 206” and/or per Fig 7 – Alg.1 Lines 5,10-11 Augw and Augs subscripts denote w-weak and s-strong};
(b) electronically, using a deep convolutional neural network, predicting a class of the weak augmentation version of the input image {LiJ [0016] “CNN…predicts class probabilities” particularly [0022] “weakly augmented sample Augw(xb) is sent to encoder 205 and the classification head 106, …classification head 106 outputs a predicted probability” classification head may use softmax classifier as evidenced per LiJ183 at [P.2 ¶1]};
(c) electronically, using a deep convolutional neural network, predicting a class of the strong augmentation version of the input image {LiJ [0016] “CNN…predicts class probabilities” particularly [0023] “strongly augmented sample Augs(ub) is sent to the encoder 205 and the classification head 106, …computes the unsupervised classification” classification head may use softmax classifier as evidenced per LiJ183 at [P.2 ¶1]};
(d) electronically determining the probability of the predicted classes of the weak augmentation version of the input image and the probability of the predicted class of the strong augmentation version of the input image {LiJ [0031,30] “class probability is generated by the encoder f() and the classification head h() and defined by the model’s prediction on its weak-augmentation: pw=h○f(Augw(u))” pw is probability weak class, strong is [0030] “strongly augmented sample Augs(ub) 206a is used to generate classification probabilities p”, similar at [0022-23], Alg.1 Lines 3-4, 21-22, see Fig 3:204,06 Augw & Augs versions of input images 202};
However, LiJ does not appear to disclose the following limitations which are met by Lienen:
(e) electronically determining whether the predicted class of the weak augmentation version of the input image is confident {Lienen Fig 6 illustrates weakly-augmented image input into model for precisiation, [P.6 Last¶] “With increasing confidence, the target sets are becoming more precise …consistency and hence the confidence in a particular class prediction” confidence is interpreted comprising precisiation to entail ratio/proportion of class prior and predictions as per Eq.6};
(f) electronically using the predicted class of the weak augmentation version of the input image, upon determining the predicted class of the weak augmentation version of the input image is confident, as a target to compute a loss between the predicted class of the weak augmentation version of the input image and the predicted class of the strong augmentation version of the input image {Lienen Fig 6 illustrates target set Q, Q to compute loss is Eq.7 [P.7 ¶1] “target sets are then used within the unlabeled loss Lu” second term of Eq.7 combined loss, and introduced [P.5] Eqs.1-4 credal labels for self-supervised learning (CSSL) which compares favorably to LiJ’s CoMatch Table 1 [P.8]. Additionally see Alg.1 [P.14 App. A.1]};
(g) electronically using computed loss to train the deep neural network upon determining the predicted class of the weak augmentation version of the input image is confident {Lienen [P.7 ¶3,1] “we train a Wide ResNet” known CNN, the training/learning is CSSL credal self-supervised learning Fig 1 where learning comprises loss function Eq.7 using target sets having been determined confident Eq.6. The CSSL technique is introduced [P.5] Eqs.1-4 and implemented Alg.1 [P.14 App. A.1]};
Lienen is directed to trained neural networks with unlabeled images thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to include the techniques of Lienen in combination for a motivation [P.7 Last¶] “CSSL is especially competitive when the number of labels is small… credal self-supervision improves the performance in almost all cases over hard pseudo-labeling” emphasis small/few-label approach, and/or stated Motivations [Sect. 3.1 ¶5] “to account for possible uncertainty about a true soft label… enables the learner to model its belief about the ground-truth” as well as [P.5 Last2¶] “optimistic generalization of the original loss… precise probabilistic target inside every credal set.”
However, the combination LiJ and Lienen does not appear to disclose the following limitations which are met by LiY:
(h) electronically clustering features of a layer of the deep neural network wherein the electronically clustering includes {LiY [P.3] Fig 2 shows clustering over image features with layered architecture, e.g. [P.4 Sect. 3.3] describes feature vector/matrix and 2-layer MLP multi-layer perceptron is a deep neural network, further [P.3 Sect. 3.1] “ResNet” is a known CNN. See also Alg.1, Fig 1}
(h1) assigning the weak augmentation version of the input image to a cluster {LiY [P.4] Alg.1 “cluster assignment by c = argmax gC(h)” upon “sample two augmentations Ta, Tb […] hai = f(Ta(xi))” illustrated Fig 2 shows gC(∙) in cluster head receiving input image x, and where ha represents Ta corresponding to a first/weak augmentation version of a contrastive learning. Example augmentations are described [P.3 Sect. 3.1]}, and
(h2) assigning the strong augmentation version of the input image to a cluster {LiY [P.4] Alg.1 “cluster assignment by c = argmax gC(h)” upon “sample two augmentations Ta, Tb […] hbi = f(Tb(xi))” illustrated Fig 2 shows gC(∙) in cluster head receiving input image x, and where hbi represents Tb corresponding to a second/strong augmentation version of a contrastive learning. Example augmentations are described [P.3 Sect. 3.1]};
(i) electronically determining a first cluster assignment prediction for the weak augmentation version of the input image {LiY [P.3 Sect.3 ¶1] “cluster assignments can easily be obtained through the soft labels predicted by CCH” cluster contrastive head, emphasis predicted. Notably, [P.4 Sect. 3.3 ¶2-3] “probability of sample n being assigned to cluster m …cluster assignment probabilities” the prediction may comprise loss function Eq.6, Alg.1, and weak augmentation may comprise “first augmentation” and described e.g. [P.3 Sect. 3.1]};
(i) electronically determining a second cluster assignment prediction for the strong augmentation version of the input image {LiY [P.3 Sect.3 ¶1] “cluster assignments can easily be obtained through the soft labels predicted by CCH” cluster contrastive head, emphasis predicted. Notably, [P.4 Sect. 3.3 ¶2-3] “probability of sample n being assigned to cluster m …cluster assignment probabilities” the prediction may comprise loss function Eq.6, Alg.1, and strong augmentation may comprise “second augmentation” and described e.g. [P.3 Sect. 3.1]}; and
(k) electronically using cluster labels as targets to train the deep neural network upon determining the predicted class of the weak augmentation version of the input image is confident {LiY Alg.1 //training epochs for clustering where [P.4 Sect. 3.3] “ỹia denotes the soft label” soft label as a target subject to “minimize L“ loss Eqs.6-7 is at least as confident as the minimized loss and/or similarity of augmented images to be assigned clusters such that “Following the idea of ‘label as representation’ …feature can be interpreted as its probability of belonging to the i-th cluster” considers augmentation of image x Fig 2 deep neural network of gC(∙) cluster head where predicted class is by “ ~ denotes the Softmax operation to produce soft labels” See also Fig 1}.
LiY is directed to trained neural networks with image augmentation thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to perform cluster assignments per LiY in combination to arrive at the invention as claimed for a motivation [P.2 ¶1-2] “simultaneously learn discriminative features and perform online clustering in a one-stage and end-to-end manner, e.g. by [P.4 ¶2] “Following the idea of ‘label as representation’ …feature can be interpreted as its probability of belonging to the i-th cluster” and/or because [P.3 ¶2] “clustering performance could be improved by decoupling the instance- and cluster-level contrastive learning into two independent subspaces.”
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over: LiJ, Lienen and LiY in view of Zhao et Han, “Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation” hereinafter Zhao (arXiv: 2107.03358v1).
With respect to claim 4, the combination of LiJ, Lienen and LiY teaches the computer implemented method as claimed in claim 1. Zhao teaches wherein
said electronically computing a loss between the predicted class of the weak augmentation version of the input image and the predicted class of the strong augmentation version of the input image is determined by coalescing self-training and consistency-regularization in a single training loss {Zhao [P.6 Sect. 3.3] Eq.10 overall training loss, loss term LMSE is “consistency regularization loss” of Eq.8 and “self-supervised learning” is used at Eq.7 for LCE term, further coalescing is distillation [P.5 Sect. 3.2] with loss term LJSD, and further discloses “images under different augmentations” [P.6 Sect. 3.3]}.
Zhao is directed to…trained neural networks for augmentation images thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to combine overall loss with the terms of Zhao in combination to arrive at the invention as claimed as applying known techniques to known methods ready for improvement to yield predictable results and/or a motivation [P.6 Sect. 3.3 ¶2] “to enforce the predictions of the same data point under different transformation to be the same” hence consistency, and coalescing/distillation helps two models learn from each other [P.5 Sect. 3.2 ¶2].
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over: LiJ, Lienen and LiY in view of Shiran et Weinshall, “Multi-Modal Deep Clustering: Unsupervised Partitioning of Images” hereinafter Shiran (arXiv: 1912.02678v3).
With respect to claim 6, the combination of LiJ, Lienen and LiY teaches the computer implemented method as claimed in claim 1. Shiran teaches wherein
said electronically using cluster labels as targets to train the deep convolutional neural network is realized by using a self-supervised loss based on deep clustering {Shiran [P.4 Sect. III.D ¶1] “we employ RotNet [14], which is a self-supervised learning algorithm that learns image features by training a ConvNet… optimizing the cross-entropy loss” implemented Alg.1 “compute ∇θLr(θ) // Lr is cross-entropy loss” the algorithm for deep clustering hence title. See Fig 1, [P.2-3]}.
Shiran is directed to deep clustering with trained convnets thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to use self-supervision learning with loss per Shiran in combination to arrive at the invention as claimed for a motivation [P.4 Sect. III.D] “self-supervision methods can significantly improve the quality of representations” and where [P.2 Sect.III ¶1] “Our goal is to partition a set of images into k clusters …self-supervised task that helps the training.”
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over: LiJ, Lienen, LiY and Zhao in view of Shiran.
With respect to claim 7, the combination of LiJ, Lienen, LiY and Zhao teaches the computer implemented method as claimed in claim 4, and further combination with Shiran teaches the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 7.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over:
Nassar et al., “All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training” hereinafter Nassar (arXiv: 2104.05248v1).
Gong “AlphaMatch: Improving Consistency for Semi-Supervised Learning with Alpha-divergence” hereinafter Gong (arXiv: 2011.11779v1), and further in view of
LiY (as above).
With respect to claim 11, Nassar teaches:
A computer implemented system configured for to train, using unlabeled images, a deep convolutional neural network for recognizing images {Nassar [P.12 Last2¶] “our system” provides github code [P.1 Abst] for requisite computer to implement Fig 3 [P.5] image model learning, the images being unlabeled [P.4 ¶2] and the model comprising ResNet [P.6 ¶6] as a known CNN}, comprising:
a first deep convolutional neural network configured to receive a weak augmentation image version of an unlabeled image and electronically determine a weak augmentation image class prediction {Nassar [P.6 ¶6] “We used WideResnet” known CNN model for [P.5] Fig 3-top model receiving weak augmentation image, [P.4 Sect. 4.1 ¶2] “unlabeled image uj, we obtain the predicted embedding for a weakly augmented version of the image… Then we calculate class scores”};
a first model configured to receive said weak augmentation image class prediction and electronically assigns a confidence level to said weak augmentation image class prediction that is above a first predetermined threshold {Nassar [P.5] Fig 3-top model inputs weak augmentation image for fSC semantic classifier yielding predictions for K – classes with class score subject to τe which is a confidence threshold described [P.4 Sect. 4.1-4.2] Eq.4 “ ≥ τe” and/or “ ≥ τo” below Eq.7, as introduced [P.3 Sect.3 ¶1] “classifier confidence exceeds the threshold” see Table 5 [P.13] confidence thresholds};
However, Nassar does not appear to disclose the following limitations which are met by Gong:
a second deep convolutional neural network configured to receive for receiving a strong augmentation image version of an unlabeled image and electronically determine a strong augmentation image class prediction {Gong Fig 1-top model receiving strong augmentation image x’ of unlabeled image x and for subsequent model predictions, the model being CNN is e.g. ResNet and/or DG-CNN known CNN classification models [P.5 Sect.4.1 ¶3], [P.7 Sect. 4.3 ¶1,3]};
a second model configured to receive said strong augmentation image class prediction and electronically assign a confidence level to said strong augmentation image class prediction that is above a second predetermined threshold {Gong Fig 1-top model receiving strong augmentation x’ for subsequent prediction and confidence above second threshold is “high confidence” e.g. Eq.7 operand ‘≥‘ greater than or equal to, e.g .[P.1 ¶3] “confidence higher than a threshold”};
a confidence evaluator for configured to electronically evaluate said confidence level of said weak augmentation image class prediction {Gong [P.3 Sect. 3.1] “high confidence…lower confidence” evaluates confidence as high or low, Fig 1 shows weak augmentation instances x, e.g. “instances x with high confidence”};
a loss determinator configured to electronically use said weak augmentation image class prediction, upon said confidence evaluator determining that said confidence level of said weak augmentation image class prediction is confident, as a target to compute a loss between said weak augmentation image class prediction and said strong augmentation image class prediction {Gong see [P.4 Sect. 3.2] “Our loss function” Eqs. 8-9 illustratively Fig 1 “Alpha divergence based consistency loss, see Eqn (9)” arrows coming from both strong and weak augmentation models which comprise the confidence described [P.3 Sect. 3.1], and discloses [P.1 ¶3] “label as the target” which more particularly the target may enforce consistency on the labels, hence consistency loss};
a training network configured to use said computed loss to train the deep neural network upon determining the predicted class of the weak augmentation version of the inputted image is confident {Gong [P.6 ¶2,5] “ResNet… train it using SGD…models trained with AlphaMatch” detailed Alg.1 Lines9,7 gradient descent on (11) (i.e. Eq.11) plugging Line7 with (9) (i.e. Eq.9) being loss function detailed [P.4 Sect. 3.2] Eqs. 9,8. Weak augmentation shown Fig 1, confidence described [P.3 Sect. 3.1]};
Gong is directed to trained neural networks with unlabeled image augmentations thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to employ second model with confidence and loss-based training per Gong in combination for a motivation [P.8 Sect.6 ¶1] “The proposed AlphaMatch is simple yet powerful. With only a few lines of extra code to implement alpha-divergence and the EM-like update, it achieves state-of-the-art performance” and includes [P.1 ¶4] “two key algorithmic advances to improve the objective and algorithm for consistency matching in SSL: 1) we propose to use alpha-divergence to measure the label consistency. We show that, by using a large value of a in alpha-divergence, we can focus more on high confidence instances in a way similar to the hard-thresholded regularization of FixMatch, but in a more ‘soft’ and flexible fashion. 2) We propose an optimization-based framework for consistency matching, which yields an EM-like algorithm with better convergence” similar at [P.3 ¶1].
However, the combination of LiJ and Gong does not appear to disclose the following limitations which are met by LiY:
a first cluster assignment prediction network configured to electronically assign said weak augmentation image version of the unlabeled image to a first cluster label {LiY Fig 2 cluster head gC(∙) neural network (MLP), configured by Alg.1 for [P.4 Alg.1] “cluster assignment by c = argmax gC(h)” implements first label [P.4 Sect. 3.3] “ỹia denotes the soft label of sample xia …cluster i under the first data augmentation” again at Fig 2, the sampled images includes [P.5 ¶2] “100,000 unlabeled samples” to “sample two augmentations Ta, Tb […] hai = f(Ta(xi))” where ha represents Ta corresponding to a first/weak augmentation version of a contrastive learning. Example augmentations are described per [P.3 Sect. 3.1]. See also Fig 1}; and
a second cluster assignment prediction network to electronically assign said strong augmentation image version of the unlabeled image to a second cluster label {LiY Fig 2 cluster head gC(∙) neural network (MLP), configured by Alg.1 for [P.4 Alg.1] “cluster assignment by c = argmax gC(h)” second assigned cluster label is ỹib (superscript-b), in contrast to ỹia (superscript-a), conveying second augmented sample xb shown Fig 2, the sampled image being from [P.5 ¶2] “100,000 unlabeled samples” for “sample two augmentations Ta, Tb […] hbi = f(Tb(xi))” where hb represents Tb corresponding to a second/strong augmentation version of a contrastive learning. See [P.4 Sect. 3.3] and example augmentations described [P.3 Sect. 3.1] and Fig 1};
wherein said training network, upon determining the predicted class of the weak augmentation version of the received image is not confident, is configured to electronically use said first and second cluster labels to train the deep convolutional neural network {LiY Alg.1 training epochs for clustering comprises first and second cluster labels respectively ỹia and ỹib from augmented samples Ta and Tb represented by ha and hb, the clustering detailed [P.4 Sect. 3.3], and trained models being convolutional comprises ResNet [P.5 ¶3] as a known CNN. A class prediction is by softmax Fig 2 denoted ‘ ~ ’, and the aspect of not confident is interpreted in that training epochs are epochs iterated until model convergence with minimized loss and may comprise cluster-level parameter τC tunable as input and applicable to similarity between cluster pairs [P.4 Sect. 3.3], see Fig 2 cluster head gC(∙) of deep neural network where predicted class is by “ ~ denotes the Softmax operation to produce soft labels” See also Fig 1}.
LiY is directed to trained neural networks with image augmentation thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to perform cluster assignments per LiY in combination to arrive at the invention as claimed for a motivation [P.2 ¶1-2] “simultaneously learn discriminative features and perform online clustering in a one-stage and end-to-end manner, e.g. by [P.4 ¶2] “Following the idea of ‘label as representation’ …feature can be interpreted as its probability of belonging to the i-th cluster” and/or because [P.3 ¶2] “clustering performance could be improved by decoupling the instance- and cluster-level contrastive learning into two independent subspaces.”
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over: Nassar, Gong and LiY in view of Zhao.
With respect to claim 14, the combination of Nassar, Gong and LiY teaches the computer implemented system as claimed in claim 11, and further combination with Zhao teaches the limitation of claim 4. Therefore, the rejection of claim 4 with equal motivation is applied to claim 14.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over: Nassar, Gong and LiY in view of Shiran.
With respect to claim 16, the combination of Nassar, Gong and LiY teaches the computer implemented system as claimed in claim 11 [sic], further combination with Shiran teaches the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 14.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over: Nassar, Gong , LiY and Zhao in view of Shiran.
With respect to claim 18, the combination of Nassar, Gong, LiY and Zhao teaches the computer implemented system as claimed in claim 16, and further combination with Shiran teaches the limitation of claim 7. Therefore, the rejection of claim 7 with equal motivation is applied to claim 18.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over: LiJ in view of Nassar and further in view of LiY.
With respect to claim 21, LiJ teaches
A computer implemented method for training a deep convolutional neural network for recognizing images using unlabeled images {LiJ proposes “CoMatch” Alg.1 – Fig 7, and further shows CNN Fig 1, [0019] “co-training framework includes a CNN” e.g. ResNet encoder [0068] and inputs unlabeled images Figs 2-3:202, [0078] “CoMatch provides a better backbone for object detection”}, comprising:
(a) generating a weak augmentation version and a strong augmentation version of an input image {LiJ Fig 6:604 “Generate a weakly augmented sample, a first strongly augmented sample …augmented sample from an unlabeled sample” samples of unlabeled images shown Fig 3:202 with 204,06 Augw & Augs augmented weak and strong versions, described e.g. [022-23], [0041] “weakly augmented unlabeled sample 201, and strongly-augmented unlabeled samples 206” and/or per Fig 7 – Alg.1 Lines 5,10-11 Augw and Augs subscripts denote w-weak and s-strong };
(b) using a convolutional neural network to electronically predict a class of the weak augmentation version of the input image and a class of the strong augmentation version of the input image {LiJ [0016] “CNN…predicts class probabilities” particularly [0022] “weakly augmented sample Augw(xb) is sent to encoder 205 and the classification head 106, …classification head 106 outputs a predicted probability” and [0023] “strongly augmented sample Augs(ub) is sent to the encoder 205 and the classification head 106, …computes the unsupervised classification” classification head may use softmax classifier as evidenced per LiJ183 at [P.2 ¶1]. LiJ further shows input images Fig 3:202};
(c) electronically determining the probability of the predicted class of the weak augmentation version of the input image and the probability of the predicted class of the strong augmentation version of the input image {LiJ [0031,30] “class probability is generated by the encoder f() and the classification head h() and defined by the model’s prediction on its weak-augmentation: pw=h○f(Augw(u))” pw is probability weak class, strong is [0030] “strongly augmented sample Augs(ub) 206a is used to generate classification probabilities p”, similar at [0022-23], Alg.1 Lines 3-4, 21-22, and Fig 3:204,06 Augw & Augs versions of input images 202};
However, LiJ does not appear to disclose the following limitations which are met by Nassar:
(d) electronically determining if the predicted class of the weak augmentation version of the inputted image is confident {Nassar [P.5] Fig 3-top half illustrates a weak augmentation version of an inputted image and K – classes with class score subject to τe which is a confidence threshold described [P.4 Sect. 4.1-4.2] Eq.4 “ ≥ τe” and/or “ ≥ τo” below Eq.7, introduced [P.3 Sect.3 ¶1] “classifier confidence exceeds the threshold” see Table 5 [P.13] confidence thresholds};
(e) upon determining the predicted class of the weak augmentation version of the inputted image is confident: {Nassar [P.5] Fig 3-top half illustrates weak augmentation version of an inputted image and K – classes with class score subject to τe which is a confidence threshold [P.4 Sect. 4.1-4.2] Eq.4 “ ≥ τe” and/or “ ≥ τo” below Eq.7, see [P.3 Sect.3 ¶1] “classifier confidence exceeds the threshold” Table 5 [P.13] confidence thresholds}
(e1) electronically using the predicted class of the weak augmentation version of the input image as a target to compute a loss between the predicted class of the weak augmentation version of the input image and the predicted class of the strong augmentation version of the input image {Nassar [P.5] Fig 3 “Unsupervised training loss” arrows from both the weak and strong augmentation model predictions over input image, the loss LuSC is Eq.5 which is then added to total loss Eq.9 [P.4 Sect. 4.1,4.3], and discloses “training targets” [P.7 Last¶] e.g. [P.6 Last¶] “use the obtained class attributes matrix as targets for our Semantic Classifier”}, and
(e2) electronically using computed loss to train the deep neural network {Nassar [P.5] Fig 4 “training loss” implemented [P.4] Eq.9 totaling Eqs. 3 and 5-8, further disclosing [P.3 ¶4] “train two classifiers sharing the same backbone network (see Fig. 1)” e.g. per [P.6 ¶6] “We use WideResnet… We train our model end-to-end with the backbone network”}; and
(f) upon determining the predicted class of the weak augmentation version of the input image is determined to be not confident: {Nassar Fig 3 K-classes of weakly augmented model predictions as [P.3 ¶2] “low-confidence predictions” e.g. Fig 6 confidence chart}
Nassar is directed to trained neural networks for weak and strong augmented images thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to employ the confidence-based classification for augmented images per Nassar in combination for a motivation [P.3 ¶3] “We aim to address the issues demonstrated for visually similar classes… we enhance the model by incorporating knowledge about potential confusions based on semantic and visual similarities” similarly as per [P.4 ¶2] “our main goal is to obtain label embeddings which capture visual similarity” thereby [P.2 ¶4] “improve pseudo-labeling quality by addressing the confusion events” and [P.4 Rt.Col] “ensure that the learners are sufficiently diverse so that they learn better based on the their different views… enable both classifiers to learn from each other. The intuition is that due to each classifier’s different view of the labels, they will each be confident about different samples of the unlabeled data. We exploit that by retaining a sample for pseudo-labeling if either of the classifiers is confident about its prediction.”
Nassar further nominally suggests [P.4 Ft.Nt] “cluster assignment” for Eq.4. However, the combination of LiJ and Nassar does not appear to detail the following limitations which are met by LiY:
(f1) electronically clustering features of a layer of the deep convolutional neural network to assign the weak augmentation version of the input image to a cluster label and assign the strong augmentation version of the input image to a cluster label {LiY Fig 1 shows feature matrix for cluster representation e.g. columnar feature vector corresponding to a dog, Fig 2 shows layered network which may use “ResNet” [P.3 ¶4] as a known CNN. The clustering per [P.4 Sect. 3.3] “feature can be interpreted as its probability of belonging to the i-th cluster, and the feature vector denotes its soft label” and assignment based on augmentations is implemented Alg.1. Example augmentations are described [P.3 Sect. 3.1]},
(f2) electronically determining a cluster assignment prediction for the weak augmentation version of the input image and a cluster assignment prediction for the strong augmentation version of the input image {LiY [P.4] Alg.1 outputs cluster assignment which is predicted by cluster head gC(∙) for argmax gC(h) where h represents augmentations Ta and Tb, the input images shown Figs 1-2 and example augmentations described [P.3 Sect. 3.1]}, and
(h3) electronically using cluster labels as targets to train the deep convolutional neural network {LiY [P.4] Alg.1 //training epochs for cluster assignment, target labels are soft labels denoted ỹia and ỹib, the training to train a CNN includes ResNet [P.3 ¶4], [P.5 ¶3]}.
LiY is directed to trained neural networks with image augmentation thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to perform cluster assignments per LiY in combination to arrive at the invention as claimed for a motivation [P.2 ¶1-2] “simultaneously learn discriminative features and perform online clustering in a one-stage and end-to-end manner, e.g. by [P.4 ¶2] “Following the idea of ‘label as representation’ …feature can be interpreted as its probability of belonging to the i-th cluster” and/or because [P.3 ¶2] “clustering performance could be improved by decoupling the instance- and cluster-level contrastive learning into two independent subspaces.”
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Li et al., US PG Pub No 2021/0374553A1 (same author LiJ) see cover/Fig 2 multi-cnn
Kim et al., US PG Pub No 2022/0129705A1 Samsung, see Fig 6 contrastive clustering, relates Kim et al., “SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learning” arXiv: 2101.06480v1 Fig 2
Saito et al., “OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers” arXiv: 2105.14148v1 see Figs 1-2, Alg.1 discloses OOD out-of-domain and OVA one-vs-all classifier
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chase P Hinckley whose telephone number is (571)272-7935. The examiner can normally be reached M-F 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda M. Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHASE P. HINCKLEY/Examiner, Art Unit 2124
Prosecution Insights