CTNF 18/475,758 CTNF 83271 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. The following action is in response to the original filing of 09/27/2023. Claims 1-18 are pending and have been considered blow. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract ideas without significantly more. Regarding claims 1, 7 and 13: Step 1, MPEP 2106.03: These limitations have been determined, under Step 1, to be statutory categories of invention: A method [..] (claim 1) A system comprising at least one computer including a processor and a memory [..] (claim 7) A non-transitory computer readable storage medium, including instructions stored thereon, which instructions [..] (claim 13) Step 2A Prong One MPEP 2106.04, 2106.04(a): These limitations represent, under Step 2A Prong One, mathematical concepts such as mathematical relationships, mathematical formulas or equations, or mathematical calculations, MPEP 2106.04(a)(2)(I): [..] determining a first cross-entropy loss [..] [..] wherein a second cross-entropy loss and an outlier regularization loss are computed by the classifier head based on the set of feature vectors and the outlier samples [..] Step 2A Prong Two, MPEP 2106.04(d): These limitations represent, under Step 2A Prong Two, mere instructions to implement the abstract idea using generic computing tools, MPEP 2106.05(f): [..] wherein the at least one computer is configured to: [..] (claim 7) [..] when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising [..] (claim 13) [..] a classifier head of a machine learning model [..] These limitations represent, under Step 2A Prong Two, mere instructions to apply the abstract idea at a high level of generality, MPEP 2106.05: [..] wherein the first cross-entropy loss is determined based on a set of predictions [..] [..] wherein the set of predictions are based on .. generating the set of predictions based on a set of feature vectors [..] [..] updating the classifier head and a prompt of the machine learning model with the first cross-entropy loss [..] [..] generating outlier samples based on the set of feature vectors [..] [..] updating the classifier head with the second cross-entropy loss and the outlier regularization loss [..] These limitations represent, under Step 2A Prong Two, mere data gathering, MPEP 2106.05: [..] providing, as input to the classifier head, the set of feature vectors and the outlier samples [..] Step 2B, MPEP 2106.05: These limitations are considered, under Step 2B, insignificant extra-solution activity as being recited at a high level of generality, MPEP 2106.05(d): [..] wherein the at least one computer is configured to: [..] (claim 7) [..] when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising [..] (claim 13) [..] a classifier head of a machine learning model [..] These limitations are considered, under Step 2B, mere instructions to apply to obtain a solution/outcome, MPEP 2106.05(f): [..] wherein the first cross-entropy loss is determined based on a set of predictions [..] [..] wherein the set of predictions are based on .. generating the set of predictions based on a set of feature vectors [..] [..] updating the classifier head and a prompt of the machine learning model with the first cross-entropy loss [..] [..] generating outlier samples based on the set of feature vectors [..] [..] updating the classifier head with the second cross-entropy loss and the outlier regularization loss [..] These limitations are considered, under Step 2B, insignificant extra-solution activity of data gathering/selecting a particular type of data, MPEP 2106.05(g): [..] providing, as input to the classifier head, the set of feature vectors and the outlier samples [..] Regarding claims 2, 8 and 14: Step 1, MPEP 2106.03: Analysis of respective parent is incorporated. Step 2A Prong One MPEP 2106.04, 2106.04(a): Analysis of respective parent is incorporated. Step 2A Prong Two, MPEP 2106.04(d): These limitations represent, under Step 2A Prong Two, mere instructions to apply the abstract idea at a high level of generality, MPEP 2106.05: [..] wherein the prompt of the machine learning model is fixed after updating the classifier head and the prompt of the machine learning model with the first cross-entropy loss; [..] Step 2B, MPEP 2106.05: These limitations are considered, under Step 2B, mere instructions to apply to obtain a solution/outcome, MPEP 2106.05(f): [..] wherein the prompt of the machine learning model is fixed after updating the classifier head and the prompt of the machine learning model with the first cross-entropy loss; [..] Regarding claims 3, 9 and 15: Step 1, MPEP 2106.03: Analysis of respective parent is incorporated. Step 2A Prong One MPEP 2106.04, 2106.04(a): Analysis of respective parent is incorporated. These limitations represent, under Step 2A Prong One, mathematical concepts such as mathematical relationships, mathematical formulas or equations, or mathematical calculations, MPEP 2106.04(a)(2)(I): [..]wherein Huber loss is a component in computing the outlier regularization loss[..] Step 2A Prong Two, MPEP 2106.04(d): All limitations are part of the abstract idea. Step 2B, MPEP 2106.05: All limitations are part of the abstract idea. Regarding claims 4, 10 and 16: Step 1, MPEP 2106.03: Analysis of respective parent is incorporated. Step 2A Prong One MPEP 2106.04, 2106.04(a): Analysis of respective parent is incorporated. These limitations represent, under Step 2A Prong One, mathematical concepts such as mathematical relationships, mathematical formulas or equations, or mathematical calculations, MPEP 2106.04(a)(2)(I): [..]wherein generating the outlier samples includes applying Gaussian noise to samples at a boundary of a cluster [..] Step 2A Prong Two, MPEP 2106.04(d): These limitations represent, under Step 2A Prong Two, mere instructions to apply the abstract idea at a high level of generality, MPEP 2106.05: [..] wherein the cluster is formed by samples from a same training session [..] Step 2B, MPEP 2106.05: These limitations are considered, under Step 2B, mere instructions to apply to obtain a solution/outcome, MPEP 2106.05(f): [..] wherein the cluster is formed by samples from a same training session [..] Regarding claims 5, 11 and 17: Step 1, MPEP 2106.03: Analysis of respective parent is incorporated. Step 2A Prong One MPEP 2106.04, 2106.04(a): Analysis of respective parent is incorporated. These limitations represent, under Step 2A Prong One, mathematical concepts such as mathematical relationships, mathematical formulas or equations, or mathematical calculations, MPEP 2106.04(a)(2)(I): [..]wherein the outlier sample generation is performed in a feature vector space RD[..] Step 2A Prong Two, MPEP 2106.04(d): All limitations are part of the abstract idea. Step 2B, MPEP 2106.05: All limitations are part of the abstract idea. Regarding claims 6, 12 and 18: Step 1, MPEP 2106.03: Analysis of respective parent is incorporated. Step 2A Prong One MPEP 2106.04, 2106.04(a): Analysis of respective parent is incorporated. Step 2A Prong Two, MPEP 2106.04(d): These limitations represent, under Step 2A Prong Two, mere instructions to implement the abstract idea using generic computing tools, MPEP 2106.05(f): [..] wherein the machine learning model includes a pre-trained encoder [..] Step 2B, MPEP 2106.05: These limitations are considered, under Step 2B, insignificant extra-solution activity as being recited at a high level of generality, MPEP 2106.05(d): [..] wherein the machine learning model includes a pre-trained encoder [..] Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15 AIA Claim s 1-2, 5-8, 11-14 and 17-18 are rejected under 35 U.S.C. 102( a)(1 ) as being anticipated by Wu, Junda, et al. "Infoprompt: Information-theoretic soft prompt tuning for natural language understanding." Advances in neural information processing systems 36 (8 June 2023): 61060-61084 [“WU”] . Regarding claim 1, WU discloses a method comprising: determining a first cross-entropy loss, wherein the first cross-entropy loss is determined based on a set of predictions (page 2-3, 2.1 Prompt Training, “The whole model is generally trained via minimizing the following loss function, PNG media_image1.png 36 354 media_image1.png Greyscale ”) and wherein the set of predictions are based on a classifier head of a machine learning model generating the set of predictions based on a set of feature vectors (page 2-3, 2.1 Prompt Training, “The model prediction for X is made on top of Z with a trainable language modeling head hθ (parameterized by θ), s.t ., the output hθ( Z ) is the distribution over all possible labels for classification.”); updating the classifier head and a prompt of the machine learning model with the first cross-entropy loss (page 3, 3 Our Method: InfoPrompt, “In encouraging the task-relevancy of the learnt prompts, we consider maximizing the mutual information between the prompt and the parameters of he language model head, denoted as θ. By maximizing such mutual information, the learnt prompt will be more correspondent with the training data with which the language model head is trained, thus captures more task-relevant information from training.”); generating outlier samples based on the set of feature vectors (pages 3-4, 3.1 The Head Loss, “ PNG media_image2.png 36 89 media_image2.png Greyscale are the negative prompt samples for contrastive learning”); providing, as input to the classifier head, the set of feature vectors and the outlier samples, wherein a second cross-entropy loss and an outlier regularization loss are computed by the classifier head based on the set of feature vectors and the outlier samples (page 4, 3.2 The Head Loss, “the head loss is the negative mutual information between the prompt P and parameters θ, i.e., PNG media_image3.png 25 126 media_image3.png Greyscale . In maximizing PNG media_image4.png 26 107 media_image4.png Greyscale , we follow [MC18] that approximate it with the following lower bound, PNG media_image5.png 32 354 media_image5.png Greyscale ”); and updating the classifier head with the second cross-entropy loss and the outlier regularization loss (page 5, 3.3 Overall Objective, “We minimize the following objective in prompt tuning: PNG media_image6.png 37 463 media_image6.png Greyscale . We denote PNG media_image7.png 25 57 media_image7.png Greyscale as the task loss. β and γ are balancing parameters for the proposed representation loss and head loss, respectively. We denote our approach as InfoPrompt.”, page 3, 3 Our Method: InfoPrompt, “In encouraging the task-relevancy of the learnt prompts, we consider maximizing the mutual information between the prompt and the parameters of he language model head, denoted as θ. By maximizing such mutual information, the learnt prompt will be more correspondent with the training data with which the language model head is trained, thus captures more task-relevant information from training.”). Regarding claim 2, WU discloses the method of claim 1, wherein the prompt of the machine learning model is fixed after updating the classifier head and the prompt of the machine learning model with the first cross-entropy loss (page 3, 3 Our Method: InfoPrompt, “By maximizing such mutual information, the learnt prompt will be more correspondent with the training data with which the language model head is trained, thus captures more task-relevant information from training.”). Regarding claim 5, WU discloses method of claim 1, wherein the outlier sample generation is performed in a feature vector space RD (page 2, 2.1 Prompt Tuning, “ PNG media_image8.png 34 274 media_image8.png Greyscale is a embedding vector with dimension D and D is the embedding dimension of the pretrained language model.”). Regarding claim 6, WU discloses the method of claim 1, wherein the machine learning model includes a pre-trained encoder (page 2, 2.1 Prompt Tuning, “further encoded by the pretrained encoder Φ”). Regarding claims 7-8 and 11-12, claims 7-8 and 11-12 recite limitations similar to claims 1-2 and 5-6, respectively, and are similarly rejected. Regarding claims 13-14 and 17-18, claims 13-14 and 17-18 recite limitations similar to claims 1-2 and 5-6, respectively, and are similarly rejected . Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claim s 3-4, 9-10 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over WU in view of Li, Alexander C., et al. "Your diffusion model is secretly a zero-shot classifier." Proceedings of the IEEE/CVF International Conference on Computer Vision . (13 September 2023). [“LI”] Regarding claim 3, WU discloses method of claim 1. WU fails to disclose wherein Huber loss is a component in computing the outlier regularization loss. LI discloses methods for training classifier models using prompts for inference (page 1, Abstract, “The recent wave of large-scale text-to-image diffusion models has dramatically increased our text-based image generation abilities. These models can generate realistic images for a staggering variety of prompts and exhibit impressive compositional generalization abilities .. Our generative approach to classification, which we call Diffusion Classifier, attains strong results on a variety of benchmarks and outperforms alternative methods of extracting knowledge from diffusion models.”). In particular, LI discloses using Huber loss for a regularization loss of the inference model (page 14, C. Inference Objective Function, “We hope followup work can explain the empirical success of the ℓ1 loss. Combining these two losses does not get the “best of both worlds.” The Huber loss, which is the squared ℓ2 loss for values less than1andis theℓ1 loss for values greater than1, roughly achieves the same performance as the theoretically-justified squaredℓ2 loss.”) . Therefore it would have been obvious to one having ordinary skill in the art and the teachings of WU and LI before them before the effective filing of the claimed invention to utilizing Huber loss in computing a regularization loss, as taught by LI, when computing the outlier regularization loss of WU. One would have ben motivated to make this utilization when investigating results between different inference accuracy and time, as suggested by LI (page 14, Table 7, Table 8, Table 9). Regarding claim 4, WU discloses the method of claim 1, wherein generating the outlier samples includes applying random noise to samples from a same training session (pages 3-4, 3.1 The Head Loss, “ PNG media_image2.png 36 89 media_image2.png Greyscale are the negative prompt samples for contrastive learning. In practice, we randomly sample K − 1 tokens from the context as the negative samples, i.e., PNG media_image9.png 30 237 media_image9.png Greyscale .”). WU fails to explicitly disclose wherein the random noise is Gaussian noise at a boundary of a cluster formed from the samples. LI discloses methods for training classifier models using prompts for inference (page 1, Abstract, “The recent wave of large-scale text-to-image diffusion models has dramatically increased our text-based image generation abilities. These models can generate realistic images for a staggering variety of prompts and exhibit impressive compositional generalization abilities .. Our generative approach to classification, which we call Diffusion Classifier, attains strong results on a variety of benchmarks and outperforms alternative methods of extracting knowledge from diffusion models.”). In particular, LI discloses adding Gaussian noise of when generating additional samples from clusters (page 3, 3.1 Diffusion Model Preliminaries “Starting at a clean sample x0, the fixed forward process PNG media_image10.png 18 91 media_image10.png Greyscale adds Gaussian noise, whereas the learned reverse process PNG media_image11.png 18 109 media_image11.png Greyscale tries to denoise its input, optionally conditioning on a variable c.”) . Therefore it would have been obvious to one having ordinary skill in the art and the teachings of WU and LI before them before the effective filing of the claimed invention to utilizing Gaussian noise of a cluster when generating samples, as taught by LI, when generating the samples using randomness of WU. One would have been motivated to make this increase accuracy of the models, as suggested by LI (page 4, Figure 3). Regarding claims 9-10, claims 9-10 recite limitations similar to claims 3-4, respectively, and are similarly rejected. Regarding claims 15-16, claims 15-16 recite limitations similar to claims 3-4, respectively, and are similarly rejected . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Batlle López; Marta et al. US 20250200342 A1 SYNTHETIC TIME-SERIES DATA GENERATION AND ITS USE IN SURVIVAL ANALYSIS AND SELECTION OF DRUG FOR FURTHER DEVELOPMENT Bao; Zhipeng et al. US 20250078327 A1 UTILIZING INDIVIDUAL-CONCEPT TEXT-IMAGE ALIGNMENT TO ENHANCE COMPOSITIONAL CAPACITY OF TEXT-TO-IMAGE MODELS YIN; Hongxu et al. US 20240119361 A1 TECHNIQUES FOR HETEROGENEOUS CONTINUAL LEARNING WITH MACHINE LEARNING MODEL ARCHITECTURE PROGRESSION Mrini; Khalil et al. US 20230419164 A1 MULTITASK MACHINE-LEARNING MODEL TRAINING AND TRAINING DATA AUGMENTATION BAI; Song et al. US 20230334834 A1 MODEL TRAINING BASED ON SYNTHETIC DATA Zhang; Zizhao et al. US 20230274143 A1 COMPLEMENTARY PROMPTING FOR REHEARSAL-FREE CONTINUAL LEARNING Detone; Daniel et al. US 20220028110 A1 SYSTEMS AND METHODS FOR PERFORMING SELF-IMPROVING VISUAL ODOMETRY Kolouri; Soheil et al. US 20210192363 A1 SYSTEMS AND METHODS FOR UNSUPERVISED CONTINUAL LEARNING Katuwal; Gajendra Jung et al. US 20200160201 A1 CLINICAL CASE SEARCH AND GENERATION SYSTEM AND METHOD BASED ON A PROBABILISTIC ENCODER-GENERATOR FRAMEWORK Gianelle; Thomas Francis et al. US 11704540 B1 SYSTEMS AND METHODS FOR RESPONDING TO PREDICTED EVENTS IN TIME-SERIES DATA USING SYNTHETIC PROFILES CREATED BY ARTIFICIAL INTELLIGENCE MODELS TRAINED ON NON-HOMOGENOUS TIME SERIES-DATA Puscas; Mihai et al. US 11544532 B2 GENERATIVE ADVERSARIAL NETWORK WITH DYNAMIC CAPACITY EXPANSION FOR CONTINUAL LEARNING Grigorescu Sorin Mihai EP 3627403 A1 TRAINING OF A ONE-SHOT LEARNING CLASSIFIER Shin, Hanul, et al. "Continual learning with deep generative replay." Advances in neural information processing systems 30 (2017). Li, Zhizhong, and Derek Hoiem. "Learning without forgetting." IEEE transactions on pattern analysis and machine intelligence 40.12 (2017): 2935-2947. Serra, Joan, et al. "Overcoming catastrophic forgetting with hard attention to the task." International conference on machine learning . PMLR, 2018. Dinh, Tuan, et al. "Lift: Language-interfaced fine-tuning for non-language machine learning tasks." Advances in Neural Information Processing Systems 35 (2022): 11763-11784. Kim, Gyuhak, Bing Liu, and Zixuan Ke. "A multi-head model for continual learning via out-of-distribution replay." Conference on lifelong learning agents . PMLR, 2022. Liu, Lingbo, et al. "Prompt-matched semantic segmentation." arXiv preprint arXiv:2208.10159 (2022). Wang, Zifeng, et al. "Learning to prompt for continual learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 2022. Wang, Yabin, Zhiwu Huang, and Xiaopeng Hong. "S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning." Advances in Neural Information Processing Systems 35 (2022): 5682-5695. Lin, Haowei, et al. "Class incremental learning via likelihood ratio based task prediction." arXiv preprint arXiv:2309.15048 (2023). Wang, Zhen, et al. "Multitask prompt tuning enables parameter-efficient transfer learning." arXiv preprint arXiv:2303.02861 (2023). Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW L TANK whose telephone number is (571)270-1692. The examiner can normally be reached Monday-Thursday 9a-6p. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /ANDREW L TANK/ Primary Examiner, Art Unit 2141 Application/Control Number: 18/475,758 Page 2 Art Unit: 2141 Application/Control Number: 18/475,758 Page 3 Art Unit: 2141 Application/Control Number: 18/475,758 Page 4 Art Unit: 2141 Application/Control Number: 18/475,758 Page 5 Art Unit: 2141 Application/Control Number: 18/475,758 Page 6 Art Unit: 2141 Application/Control Number: 18/475,758 Page 7 Art Unit: 2141 Application/Control Number: 18/475,758 Page 8 Art Unit: 2141 Application/Control Number: 18/475,758 Page 10 Art Unit: 2141 Application/Control Number: 18/475,758 Page 12 Art Unit: 2141 Application/Control Number: 18/475,758 Page 13 Art Unit: 2141 Application/Control Number: 18/475,758 Page 14 Art Unit: 2141 Application/Control Number: 18/475,758 Page 15 Art Unit: 2141 Application/Control Number: 18/475,758 Page 16 Art Unit: 2141