Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
2. The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
3. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
4. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“a hallucination hard anchor generation circuit, configured to generate a plurality of hallucination hard anchors according to a plurality of easy samples ...”, “a classification circuit, coupled to the hallucination hard anchor generation circuit, configured to classify a plurality of hard samples as the plurality of types ...”, and “a training circuit, coupled to the classification circuit, configured to ...” in claim 1; and
“a feature extraction circuit, coupled to the classification circuit, configured to extract a plurality of feature vectors of the plurality of original samples” in claim 2.
Further limitations found in dependent claims 3-6 which further clarify what the same aforementioned circuit instances are further configured to do in terms of additional limitations/steps.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Upon a review of Applicants’ specification, the Examiner has found examples of the corresponding structure to perform the claimed functions in [0018] (describing processor and memory elements in relation to the circuit instances noted here by the Examiner).
If Applicants do not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, Applicants may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office Action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
7. Claims 1-2, 5, 7-8, 10, and 12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent Application Publication No. 2019/0228267 (“Singh”).
Regarding claim 1, SINGH teaches A machine learning training device ([0002]: “The present disclosure relates generally to computer vision and machine learning. More particularly, the present disclosure relates to computer vision systems and methods for machine learning using image hallucinations.”), comprising:
a hallucination hard anchor generation circuit, configured to generate a plurality of hallucination hard anchors according to a plurality of easy samples ([0020] discussing the generation of a synthesized whole training set from a smaller number of images, where the images are used as a basis for the generation of the further perturbed images (see [0023]) that all together constitute the training set, e.g. the further images being the result of taking a particular image and, after perturbing it per [0023], then generating different views of it by way of the perturbation, and where the original images that have not been perturbed are understood to be without noise and hence easy, within the context of [0028]-[0029]), wherein the plurality of easy samples are classified as a plurality of types, wherein each of the plurality of hallucination hard anchors corresponds to one of the plurality of types ([0023] clarifying that each image and each resulting generating image obtained therefrom via perturbation is associated with a particular keypoint as identified by a feature detector, where the association of a keypoint to the relevant/pertinent original and generated images can be understood to be a type corresponding to a hard anchor for those images, e.g. corresponding to a particular detectable/identifiable thing/object in the context of a computer vision system capable of performing detection (where the Examiner equates the detectable/identifiable thing/object as embedded as a hard anchor for that particular type and for which the multiple views resulting from its perturbing/deforming are understood to be variants/views relating to the original image and hence the thing/object and hence the anchor element as embedded));
a classification circuit, coupled to the hallucination hard anchor generation circuit, configured to classify a plurality of hard samples as the plurality of types according to the plurality of hallucination hard anchors ([0025]-[0028] discussing the use of the images, e.g. original and generated, to train what is essentially a classifier (e.g., [0025]: “build a neural model to learn a feature embedding for those synthesized matches”), and then further clarified in [0028]-[0029] within the broader context of computer vision, object detection, and so forth per [0003], such that per [0028]-[0029] the point of this training improvement is so that the classifier can learn to perform with harder and more noisier image inputs), wherein parts of the plurality of hard samples which are classified as the plurality of types are a plurality of clean hard samples, wherein another parts of the plurality of hard samples which are not classified as the plurality of types are a plurality of noisy hard samples ([0030] discussing the use of positive and negative samples that correspond to a particular anchor, which the Examiner understands to be perturbations of the anchor/easy/original image that when processed result in a positive or negative classification result, and per [0029] the hard/difficulty level of a sample is adjustable by deformation/perturbation (i.e., made more noisy) such that some samples would be understood to be more noisy than others (e.g., to read on the distinction between clean and hard verses noisy and hard, as recited), and per [0030] there is an example of a hard negative sample which is understood to delineate the edge of the classifier’s ability to perform for that particular difficulty); and
a training circuit, coupled to the classification circuit, configured to perform a machine learning training according to the plurality of easy samples and the plurality of clean hard samples ([0025]-[0028] discussing the use of the images, e.g. original and generated, to train what is essentially a classifier (e.g., [0025]: “build a neural model to learn a feature embedding for those synthesized matches”), and then further clarified in [0028]-[0029] within the broader context of computer vision, object detection, and so forth per [0003], such that per [0028]-[0029] the point of this training improvement is so that the classifier can learn to perform with harder and more noisier image inputs, and hence the aspects of the taught invention surrounding the model that modulate its difficulty in view of loss could be understood to read on the recited training circuit).
Regarding claim 2, Singh teaches The machine learning training device of claim 1, wherein the classification circuit is further configured to classify a plurality of original samples as the plurality of easy samples and the plurality of hard samples according to a plurality of loss and a loss threshold of the plurality of original samples (Singh’s [0028]-[0030] clearly teach an aspect where hardness/difficulty correlates with loss, thereby encompassing a spectrum of difficulty including easy and harder and so forth as discussed therein, and there is a loss threshold for which the performance relating thereto is understood to be negative), wherein the machine learning training device further comprises:
a feature extraction circuit, coupled to the classification circuit, configured to extract a plurality of feature vectors of the plurality of original samples (in building the model / training, [0026]-[0027] describe feature extraction and the embedding of features via a feature vector).
Regarding claim 5, Singh teaches The machine learning training device of claim 1, wherein the hallucination hard anchor generation circuit is further configured to update the hallucination hard anchor generation circuit according to a plurality of loss functions of the plurality of hallucination hard anchors ([0028]-[0030] discussing the modulation of difficulty for the model based on loss, for example [0029] explicitly: “The difficulty can be automatically adjusted by either increasing or decreasing Vpix and Vrot, thus allowing the neural network to fall back to easier examples if it starts making mistakes on harder examples. FIG. 5 is an example of pairs of patches with increased difficulty.” and where the mistakes as mentioned would be indicated by the loss and whether the loss is presumably acceptable for the model to perform as expected), wherein the plurality of easy samples are classified as a plurality of batches (training is performed in terms of processing batches explicitly, per [0030] (and also [0032]): “During each training iteration, the system can load a batch of anchors and positives, and a pool of negatives. The pool of negatives can be used to choose the hardest example for each pair of anchors and positives. The hard examples can be determined through their loss values. The system can retry mining for a fixed number of times, and if no hard negatives are found, the system can proceed with the batch in hand.”), wherein the hallucination hard anchor generation circuit is further configured to generate the plurality of hallucination hard anchors and to update the hallucination hard anchor generation circuit according to the plurality of batches ([0028]-[0030] discussing the adjustment of the model based on training and the model’s performance in view of loss and difficulty).
Regarding claim 7, the claim includes the same or similar limitations as claim 1 discussed above, and is therefore rejected under the same rationale.
Regarding claim 8, the claim includes the same or similar limitations as claim 2 discussed above, and is therefore rejected under the same rationale.
Regarding claim 10, the claim includes the same or similar limitations as claim 5 discussed above, and is therefore rejected under the same rationale.
Regarding claim 12, the claim includes the same or similar limitations as claim 1 discussed above, and is therefore rejected under the same rationale.
Claim Rejections - 35 USC § 103
8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
9. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
10. Claim 3-4, 9, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Singh in view of Non-Patent Literature “Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation” (“Zhang 1”).
Regarding claim 3, Singh teaches the machine learning training device of claim 1, as discussed above. The aforementioned reference teaches the further limitation wherein the hallucination hard anchor generation circuit is further configured to select at least two of the plurality of easy samples, and to mix at least two of the plurality of easy samples ... so as to generate one of the plurality of hallucination hard anchors, wherein the at least two of the plurality of easy samples are different types of the plurality of types (Singh’s [0022]-[0025] discussing perturbing/deforming of obtained images to generate further images, and specifically [0025]’s discussion of extracting two patches to create an image pair appears to teach a version of mixing based on two or more images similar to what Applicants have recited) but not a mixing as taught according to at least one ratio value. Rather, the Examiner relies upon ZHANG 1 to teach what Singh otherwise lacks, see e.g., Zhang 1’s page 2, 2nd full paragraph, discussing the generation of virtual instances based on a weighted combination of what is essentially training data to arrive at further augmented data. The Examiner reasons that a weighted combination of elements is akin to a mixing of elements to be combined in accordance to a ratio, much like Applicants’ recitation.
The references both relate to the improvement of model training for tasks relating to detection/classification by use of anchors in the embedded space. Hence, they are similarly directed and therefore analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a weighted combination aspect per Zhang 1 into Singh’s mixing/combination feature as cited, with a reasonable expectation of success, for purposes of enabling the combination with a greater specificity/precision by way of the capability to weight the affected elements.
Regarding claim 4, Singh in view of Zhang 1 teach the machine learning training device of claim 3, as discussed above. As discussed per claim 3, Zhang 1 teaches a weighted combination of elements to arrive at a further element, in a manner where the further generated elements are understood to improve the model’s training and performance. As with claim 3, the Examiner reasons that such a weighted combination as taught is akin to combining elements in accordance with a ratio, for example. Hence, under that same reasoning, the Examiner believes the aforementioned references Singh and Zhang 1, as discussed above per claim 3, teach the present claim’s additional limitation wherein a first hallucination hard anchor of the plurality of hallucination hard anchors comprises a first ratio value of a first easy sample and a second ratio value of a second easy sample, wherein the first easy sample is classified as a first type of the plurality of types, the second easy sample is classified as a second type of the plurality of types, wherein when the first ratio value is higher than the second ratio value, the first hallucination hard anchor is set to be the first type.
The motivation for combining the references is as discussed above in relation to claim 3.
Regarding claim 9, the claim includes the same or similar limitations as claim 4 discussed above, and is therefore rejected under the same rationale.
Regarding claim 13, the claim includes the same or similar limitations as claim 4 discussed above, and is therefore rejected under the same rationale.
11. Claims 6, 11, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Singh in view of Non-Patent Literature “DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection” (“Zhang 2”).
Regarding claim 6, Singh teaches the machine learning training device of claim 1, as discussed above. The aforementioned reference teaches wherein the classification circuit is further configured to obtain at least one hallucination hard anchor of the plurality of hallucination hard anchors (as discussed per claim 1, there is an anchoring aspect as discussed per Singh’s [0028]-[0030] used to improve training and performance for the model as applied to object detection type tasks), wherein at least one distance between the at least one hallucination hard anchor and a first hard sample of the plurality of hard samples is smaller than a distance threshold, and the first hard sample is classified as one of the plurality of types according to the at least one hallucination hard anchor (see Singh’s [0027] for the mention of a distance function to consider loss of embedded information in relation to its general detection/classification task, which the Examiner reasons is similar to the distance aspect recited here, and hence the further discussions of loss in view of difficulty found in subsequent paragraphs [0029]-[0030] use loss to indicate whether the performance is sufficient or whether the difficulty is too great and needs adjusting, of which there would be understood to be the search of a threshold for which loss and difficulty are in balance). Based on the latter, the Examiner reasons that Singh’s model management per [0027]-[0030] appears to involve a threshold for which loss, distance, and difficulty are balanced, and if not then the model parameters are subject to adjustment. However, to the extent that Singh’s balancing of loss and difficulty in tuning/adjusting its model is not sufficiently grounded in a teaching of a definite threshold that would satisfy the claim’s limitation as written, the Examiner further relies upon ZHANG 2 to teach what Singh might otherwise lack, see e.g., Zhang 2’s FIG. 3, and specifically its “GT box” which teaches a distance based threshold for each of positive and negative spaces in relation to an embedded anchor, and the Examiner reasons that such a thresholding as realized per Zhang 2 is in accordance with realizing the sort of difficulty adjustment that is loss-aware as described per Singh’s [0027]-[0030].
The references both relate to the improvement of model training for tasks relating to detection/classification by use of anchors in the embedded space. Hence, they are similarly directed and therefore analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a threshold-based distance and loss association per Zhang 2 to more concretely realize Singh’s comparable loss-aware model parameter adjustment aspect as discussed per its [0027]-[0030], with a reasonable expectation of success, for purposes of exposing the model adjustment more finely in terms of that distance and hence loss, as defined per Zhang 2, with the understanding that a better definition for distance and hence loss in this way per Zhang 2 can improve the model adjustment’s precision over how Singh already does it.
Regarding claim 11, the claim includes the same or similar limitations as claim 6 discussed above, and is therefore rejected under the same rationale.
Regarding claim 14, the claim includes the same or similar limitations as claim 6 discussed above, and is therefore rejected under the same rationale.
Conclusion
12. The prior art made of record and not relied upon is considered pertinent to Applicants’ disclosure:
US 2025/0068847
US 2024/0356967
WO 2023/118317 A1
Non-Patent Literature “A Survey on Hallucination in Large Visual Language Models” (Lan)
Non-Patent Literature “Aligning Large Multi-Modal Model with Robust Instruction Tuning” (Liu)
Non-Patent Literature “Learning Transferable Visual Models From Natural Language Supervision” (Radford)
Non-Patent Literature “Full Guide to Contrastive Learning” (Buhl)
Non-Patent Literature “Advancements in Scientific Controllable Text Generation Methods” (Goel)
Non-Patent Literature “Learning Patch-Based Anchors for Face Hallucination” (Ko)
Non-Patent Literature “A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by ...” (Varshney)
Non-Patent Literature “Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision” (Wang)
13. Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHOURJO DASGUPTA whose telephone number is (571)272-7207. The examiner can normally be reached M-F 8am-5pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 571 272 4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHOURJO DASGUPTA/Primary Examiner, Art Unit 2144
Prosecution Insights