DETAILED ACTION
This communication is in response to the application filed 4/23/24 in which claims 1-20 were presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 4/23/24 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
An autoencoder computer program product for optimizing machine learning by soft prompting a machine learning language model (“MLLM”) trained on a corpus of unlabeled sample data for natural language classification, the autoencoder computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein an execution of the program instructions by a computer processor causes a computing device to: [Step 1: YES. Computer program product stored on a computer readable storage medium is a statutory category. Step 2A Prong 2/Step 2B: NO. Recitation of generic computer components is mere instruction to apply exception. Recitation of “for optimizing machine learning by soft prompting a machine learning language model trained on a corpus of unlabeled same data for natural language classification” generally links the exception to a technological environment or field of use.]
encode a query into a soft computer prompt corresponding to a target response from the MLLM; [Step 2A Prong 1: YES. Mental process.]
embed the soft computer prompt into a multidimensional prompt vector in a representation space of the unlabeled sample data; [Step 2A Prong 1: YES. Mental process.]
embed a minibatch of the unlabeled sample data into a plurality of multidimensional data vectors in the representation space; [Step 2A Prong 1: YES. Mental Process.]
determine a contrastive loss from the plurality of multidimensional data vectors in relation to the multidimensional prompt vector in the representation space; and [Step 2A Prong 1: YES. Mathematical Concept.]
in response to determining that the contrastive loss is nonzero, then train the MLLM to reduce the contrastive loss [Step 2A Prong 2/Step 2B: Instruction to apply exception.].
Step 1: YES. A computer program product comprising a computer readable storage medium having instructions embodied therewith falls in a statutory category.
Step 2A Prong 1: YES. Encoding a query into a prompt corresponding to a target response, embedding the prompt into a multidimensional prompt vector in a representation space, embedding a minibatch of the unlabeled sample data into multidimensional data vectors in the representation space, are steps capable of being performed mentally or with the aid of pen and paper. Determining a contrastive loss from the multidimensional data vectors in relation to the prompt vector in the representation space corresponds to a mathematical calculation.
Step 2A Prong 2: NO. Recitation of generic computer components in the preamble is mere instruction to apply the judicial exception. The last limitation of training the MLLM to reduce the contrastive loss in response to determining that the calculated contrastive loss is non-zero describes an outcome at a high level and, therefore, is mere instruction to apply the exception.
Accordingly, claim 1 is ineligible.
Claim 2
The autoencoder computer program product of claim 1, wherein the execution of the program instructions further causes the computing device to:
inference a response to the soft computer prompt; [Step 2A Prong 1: YES. Mental process.]
determine an MLLM loss by comparing the response to the target response; and [Step 2A Prong 1: YES. Mental Process and/or Mathematical Concept]
in response to determining the MLLM loss is non-zero, train the MLLM to minimize the MLLM loss [Step 2A Prong 2/Step 2B: NO. Mere instruction to apply exception.].
Step 2A Prong 1: YES. Inferencing a response to a prompt, under a broadest reasonable interpretation, encompasses a mental process. Determining an MLLM loss by comparing the response to a target response encompasses a mental process and/or a mathematical calculation.
Step 2A Prong 2/Step 2B: NO. Training the MLLM to minimize the MLLM loss in response to determining the loss is non-zero is mere instruction to apply the exception.
Accordingly, claim 2 is ineligible.
Claim 3
The autoencoder computer program product of claim 1, wherein the encode the query further comprises:
allocate a computer programmable memory to a plurality of tokens; [Step 2A Prong 1: YES. Mental process. Step 2A Prong 2: NO. Recitation of generic computer components is mere instruction to apply exception.]
divide the query into a plurality of query segments; [Step 2A Prong 1: YES. Mental Process.]
store each query segment to one of the tokens in the plurality of tokens; and [Step 2A Prong 2: NO. Mere instruction to apply judicial exception. Step 2B: NO. Storing information is well understood routine and conventional.]
mask one of the tokens to withhold the corresponding query segment from the soft computer prompt [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Allocating tokens, dividing the query into segments and masking one of the tokens is capable of being performed manually by a user with the aid of pen and paper and, therefore, encompasses a mental process.
Step 2A Prong 2/Step 2B: Recitation of “computer programmable memory” and “storing each query segment to one of the tokens…” is mere instruction to apply the exception. In 2B, storing information is well understood, routine and conventional. Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93.
Accordingly, claim 3 is ineligible.
Claim 4
The autoencoder computer program product of claim 3, wherein the determine the contrastive loss further comprises:
embed a copy of the prompt vector into a positive sample vector in the representation space; and [Step 2A Prong 1: YES. Mental Process.]
embed a sample data outside the minibatch into a negative sample vector in the representation space [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Embedding a vector and sample data into vectors is capable of being performed mentally or with the aid of pen and paper and, therefore, encompasses a Mental Process.
Accordingly, claim 4 is ineligible.
Claim 5
The autoencoder computer program product of claim 4, wherein the positive sample vector comprises an augmented copy of the prompt vector [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. An augmented copy of the prompt vector can be generated manually by a user and encompasses a mental process.
Accordingly, claim 5 is ineligible.
Claim 6
The autoencoder computer program product of claim 3, wherein the encode the query further comprises:
mask one of the tokens of the plurality of tokens to form a first encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
embed a first soft computer prompt with the first encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
set the first soft computer prompt to the prompt vector; [Step 2A Prong 1: YES. Mental Process.]
mask a different one of the tokens of the plurality of tokens to form a second encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
embed a second soft computer prompt with the second encoder configuration; and [Step 2A Prong 1: YES. Mental Process.]
set the second soft computer prompt to a positive sample vector [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Masking tokens may be performed manually by a user with the aid of pen and paper. Embedding the prompt in a first or second configuration can also be performed manually or with the aid of pen and paper. Setting the prompt to a prompt vector encompasses a mental process.
Step 2A Prong 2/Step 2B: NO. Recitation of a generic computer component (e.g., soft computer) is mere instruction to apply the exception.
Accordingly, claim 6 is ineligible.
Claim 7
The autoencoder computer program product of claim 3, wherein the encode the query further comprises:
mask one or more of the tokens of the plurality of tokens to form a first encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
embed a first soft computer prompt with the first encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
set the first soft computer prompt to the prompt vector; and [Step 2A Prong 1: YES. Mental Process.]
mask a different one or more of the tokens of the plurality of tokens to form a second encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
embed a second soft computer prompt with the second encoder configuration; and [Step 2A Prong 1: YES. Mental Process.]
set the second soft computer prompt to a positive sample vector [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. The steps of masking tokens, embedding the prompts with a particular configuration, and setting the prompts to the prompt vector encompass mental processes.
Step 2A Prong 2/Step 2B: NO. Recitation of a generic computer component (e.g., soft computer) is mere instruction to apply the exception.
Accordingly, claim 7 is ineligible.
Claim 8
The autoencoder computer program product of claim 5, wherein the execution of the program instructions further causes the computing device to:
inference similarity of a selected data vector in the minibatch by a pairwise comparison to the positive sample vector and the negative sample vector; and [Step 2A Prong 1: YES. Mental Process.]
in response to determining that the selected data vector is similar to the positive sample vector, set the selected data vector to a positive data sample vector; and [Step 2A Prong 1: YES. Mental Process.]
in response to determining that the selected data vector is dissimilar to the positive sample vector, set the selected data vector to a negative data sample vector [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Inferencing similarity by pairwise comparison of vectors is capable of being performed mentally. Setting the vector to a positive or negative data vector based on the result of the comparison is also something that can be performed mentally.
Accordingly, claim 8 is ineligible.
Claim 9
The autoencoder computer program product of claim 8, wherein the determine the contrastive loss comprises a predetermined contrastive loss objective for a first positive data sample and a first negative data sample in the minibatch [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Using a predetermined contrastive loss objective as the contrastive loss is a mental process.
Accordingly, claim 9 is ineligible.
Claim 10
The autoencoder computer program product of claim 9, wherein the predetermined contrastive loss objective comprises a boundary condition in the representation space for the first positive data sample and the first negative data sample [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Treating the predetermined contrastive loss object as a boundary condition in the representation space encompasses a mental process.
Accordingly, claim 10 is ineligible.
Claim 11
The autoencoder computer program product of claim 10, wherein the predetermined contrastive loss objective comprises a maximum value between:
zero; and [Step 2A Prong 1: YES. Mathematical Concept.]
a contrastive representation state between the first positive data sample, the first negative data sample, and an annular boundary condition around the prompt vector [Step 2A Prong 1: YES. Mathematical Concept.].
Step 2A Prong 1: YES. Making the predetermined contrastive loss object a maximum value between zero and a representation state between a first positive sample, first negative sample and an annular boundary condition encompasses a mathematical concept.
Accordingly, claim 11 is ineligible.
Claim 12
The autoencoder computer program product of claim 11, wherein the determine the contrastive loss comprises pushing the first negative data sample to establish an annular margin around the prompt vector [Step 2A Prong 1: YES. Mathematical Concept and/or Mental Process.].
Step 2A Prong 1: YES. “Pushing” the negative sample to establish an annular margin around the prompt vector encompasses a mental process and/or a mathematical addition to the margin.
Accordingly, claim 12 is ineligible.
Claim 13
An autoencoder apparatus for optimizing machine learning by soft prompting a machine learning language model (“MLLM”) trained on a corpus of unlabeled sample data for natural language classification, the autoencoder apparatus comprising: [Step 1: YES. Apparatus is a statutory category.]
an encoder configured to encode a query into a soft computer prompt corresponding to a target response from the MLLM; [Step 2A Prong 1: YES. Mental Concept. Step 2A Prong 2/Step 2B: NO. Recitation of an encoder is mere instruction to apply the exception.]
a computer readable storage medium having program instructions embodied therewith, wherein an execution of the program instructions by a computer processor causes a computing device to: [Step 2A Prong 2/Step 2B: NO. Recitation of generic computer components is mere instruction to apply exception.]
encode a query into a soft computer prompt corresponding to a target response from the MLLM; [Step 2A Prong 1: YES. Mental Process.]
embed the soft computer prompt into a multidimensional prompt vector in a representation space of the unlabeled sample data; [Step 2A Prong 1: YES. Mental Process.]
embed a minibatch of the unlabeled sample data into a plurality of multidimensional data vectors in the representation space; [Step 2A Prong 1: YES. Mental Process.]
determine a contrastive loss from the plurality of multidimensional data vectors in relation to the multidimensional prompt vector in the representation space; and [Step 2A Prong 1: YES. Mathematical Concept.]
in response to determining that the contrastive loss is nonzero, train the MLLM to reduce the contrastive loss [Step 2A Prong 2/Step 2B: NO. Mere instruction to apply exception.].
Step 1: YES. An apparatus comprising a computer readable storage medium having program instructions embodied therewith is directed to a statutory category.
Step 2A Prong 1: YES. Encoding a query into a prompt corresponding to a target response, embedding the prompt into a multidimensional prompt vector in a representation space, embedding a minibatch of the unlabeled sample data into multidimensional data vectors in the representation space, are steps capable of being performed mentally or with the aid of pen and paper. Determining a contrastive loss from the multidimensional data vectors in relation to the prompt vector in the representation space corresponds to a mathematical calculation.
Step 2A Prong 2: NO. Recitation of generic computer components is mere instruction to apply the exception. Training the MLLM to reduce a loss is mere instruction to apply the exception.
Accordingly, claim 13 is ineligible.
Claim 14
The autoencoder apparatus of claim 13, wherein the encode the query further comprises:
allocate a computer programmable memory to a plurality of tokens; [Step 2A Prong 1: YES. Mental process. Step 2A Prong 2: NO. Recitation of generic computer components is mere instruction to apply exception.]
divide the query into a plurality of query segments; [Step 2A Prong 1: YES. Mental Process.]
store each query segment to one of the tokens in the plurality of tokens; and [Step 2A Prong 2: NO. Mere instruction to apply judicial exception. Step 2B: NO. Storing information is well understood routine and conventional.]
mask one of the tokens to withhold the corresponding query segment from the soft computer prompt [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Allocating tokens, dividing the query into segments and masking one of the tokens is capable of being performed manually by a user with the aid of pen and paper and, therefore, encompasses a mental process.
Step 2A Prong 2/Step 2B: Recitation of “computer programmable memory” and “storing each query segment to one of the tokens…” is mere instruction to apply the exception. In 2B, storing information is well understood, routine and conventional. Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93.
Accordingly, claim 14 is ineligible.
Claim 15
The autoencoder apparatus of claim 13, wherein the determine the contrastive loss further comprises:
embed a copy of the prompt vector into a positive sample vector in the representation space; and [Step 2A Prong 1: YES. Mental Process.]
embed a sample data outside the minibatch into a negative sample vector in the representation space [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Embedding a vector and sample data into vectors is capable of being performed mentally or with the aid of pen and paper and, therefore, encompasses a Mental Process.
Accordingly, claim 15 is ineligible.
Claim 16
The autoencoder apparatus of claim 15, wherein the positive sample vector comprises an augmented copy of the prompt vector [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. An augmented copy of the prompt vector can be generated manually by a user and encompasses a mental process.
Accordingly, claim 16 is ineligible.
Claim 17
The autoencoder apparatus of claim 14, wherein the execution of the program instructions further causes the computing device to:
mask one or more of the tokens of the plurality of tokens to form a first encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
embed a first soft computer prompt with the first encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
set the first soft computer prompt to the prompt vector; and [Step 2A Prong 1: YES. Mental Process.]
mask a different one or more of the tokens of the plurality of tokens to form a second encoder configuration; [Step 2A Prong 1: YES. Mental Process.]
embed a second soft computer prompt with the second encoder configuration; and [Step 2A Prong 1: YES. Mental Process.]
set the second soft computer prompt to a positive sample vector [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Masking tokens may be performed manually by a user with the aid of pen and paper. Embedding the prompt in a first or second configuration can also be performed manually or with the aid of pen and paper. Setting the prompt to a prompt vector encompasses a mental process.
Step 2A Prong 2/Step 2B: NO. Recitation of a generic computer component (e.g., soft computer) is mere instruction to apply the exception.
Accordingly, claim 17 is ineligible.
Claim 18
The autoencoder apparatus of claim 17, wherein the execution of the program instructions further causes the computing device to:
inference similarity of a selected data vector in the minibatch by a pairwise comparison to a positive sample vector and a negative sample vector; and [Step 2A Prong 1: YES. Mental Process.]
in response to determining that the selected data vector is similar to the positive sample vector, set the selected data vector to a positive data sample vector; and [Step 2A Prong 1: YES. Mental Process.]
in response to determining that the selected data vector is dissimilar to the positive sample vector, set the selected data vector to a negative data sample vector [Step 2A Prong 1: YES. Mental Process.].
Step 2A Prong 1: YES. Inferencing similarity by pairwise comparison of vectors is capable of being performed mentally. Setting the vector to a positive or negative data vector based on the result of the comparison is also something that can be performed mentally.
Accordingly, claim 18 is ineligible.
Claim 19
The autoencoder apparatus of claim 18, wherein the determine the contrastive loss comprises pushing a first negative data sample vector to establish an annular margin around the prompt vector [Step 2A Prong 1: YES. Mathematical Concept and/or Mental Process.].
Step 2A Prong 1: YES. “Pushing” the negative sample to establish an annular margin around the prompt vector encompasses a mental process and/or a mathematical addition to the margin.
Accordingly, claim 19 is ineligible.
Claim 20
A computer system for optimizing machine learning by soft prompting a machine learning language model (“MLLM”) trained on a corpus of unlabeled sample data for natural language classification, the computer system having a processor, a computer-readable memory, a computer-readable tangible storage device, and program instructions stored on the computer-readable storage device for execution by a processor via the computer-readable memory, wherein the computer system is configured to perform a method, comprising: [Step 1: YES. Computer system is a statutory category. Step 2A Prong 2/Step 2B: Recitation of generic computer components is mere instruction to apply exception.]
encode a query into a soft computer prompt corresponding to a target response from the MLLM; [Step 2A Prong 1: YES. Mental Process.]
embed the soft computer prompt into a multidimensional prompt vector in a representation space of the unlabeled sample data; [Step 2A Prong 1: YES. Mental Process.]
embed a minibatch of the unlabeled sample data into a plurality of multidimensional data vectors in the representation space; [Step 2A Prong 1: YES. Mental Process.]
determine a contrastive loss from the plurality of multidimensional data vectors in relation to the multidimensional prompt vector in the representation space; and [Step 2A Prong 1: YES. Mathematical Concept.]
in response to determining that the contrastive loss is nonzero, then train the MLLM to reduce the contrastive loss [Step 2A Prong 2/Step 2B: NO. Mere instruction to apply exception.].
Step 1: YES. A computer system having a processor, a computer-readable memory, a computer-readable tangible storage device, and program instructions stored on the computer-readable storage device is directed to a statutory category.
Step 2A Prong 1: YES. Encoding a query into a prompt corresponding to a target response, embedding the prompt into a multidimensional prompt vector in a representation space, embedding a minibatch of the unlabeled sample data into multidimensional data vectors in the representation space, are steps capable of being performed mentally or with the aid of pen and paper. Determining a contrastive loss from the multidimensional data vectors in relation to the prompt vector in the representation space corresponds to a mathematical calculation.
Step 2A Prong 2: NO. Recitation of generic computer components is mere instruction to apply the exception. Training the MLLM to reduce a loss is mere instruction to apply the exception.
Accordingly, claim 20 is ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yan, Yuanmeng, et al. "Consert: A contrastive framework for self-supervised sentence representation transfer." Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021 (“Yan”) in view of Kar (US 2025/0190802 A1; published Jun. 12, 2025).
Regarding claim 1, Yan discloses [a]n autoencoder computer program product for optimizing machine learning by soft prompting a machine learning language model (“MLLM”) trained on a corpus of unlabeled sample data for natural language classification, the autoencoder computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein an execution of the program instructions by a computer processor causes a computing device to: (Yan Section 3.1 (“Our approach is mainly inspired by SimCLR (Chen et al., 2020a). As shown in Figure 2, there are three major components in our framework: • A data augmentation module that generates different views for input samples at the token embedding layer. • A shared BERT encoder that computes sentence representations for each input text. During training, we use the average pooling of the token embeddings at the last layer to obtain sentence representations. • A contrastive loss layer on top of the BERT encoder. It maximizes the agreement between one representation and its corresponding version that is augmented from the same sentence while keeping it distant from other sentence representations in the same batch.”))
encode a query into a soft computer prompt corresponding to a target response from the MLLM; (Yan pg. 5068 1st column (“For each input text x, we first pass it to the data augmentation module, in which two transformations T1 and T2 are applied to generate two versions of token embeddings: ei = T1(x),ej = T2(x), where ei,ej ∈ RL× d, L is the sequence length and d is the hidden dimension.”))
embed the soft computer prompt into a multidimensional prompt vector in a representation space of the unlabeled sample data; (Yan pg. 5068 1st column (“After that, both ei and ej will be encoded by multi-layer transformer blocks in BERT and produce the sentence representations ri and rj through average pooling.”))
embed a minibatch of the unlabeled sample data into a plurality of multidimensional data vectors in the representation space; (Yan pg. 5068 1st column (“Following Chen et al. (2020a), we adopt the normalized temperature-scaled cross-entropy loss (NTXent) as the contrastive objective. During each training step, we randomly sample N texts from Dto construct a mini-batch, resulting in 2N representations after augmentation.”))
determine a contrastive loss from the plurality of multidimensional data vectors in relation to the multidimensional prompt vector in the representation space; and in response to determining that the contrastive loss is nonzero, then train the MLLM to reduce the contrastive loss (Yan pg. 5068 1st column (“Each data point is trained to find out its counterpart among 2(N − 1) in-batch negative samples:
PNG
media_image1.png
116
526
media_image1.png
Greyscale
, where sim(·) indicates the cosine similarity function, τ controls the temperature and 1 is the indicator. Finally, we average all 2N in-batch classification losses to obtain the final contrastive loss Lcon.”)).
Yan does not expressly disclose the autoencoder computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein an execution of the program instructions by a computer processor causes a computing device to (but see Kar ¶ 15 (“ In an embodiment, the present disclosure discloses a non-transitory computer readable storage media. The storage media comprises instructions that when executed causes a processor to receive an input text indicating a query made by a user; and retrieve a response from one or more data sources based on the input text using a self-supervised retrieval model. The self-supervised retrieval model is pre-trained by: providing a plurality of unlabelled texts as training input data to the self-supervised retrieval model; determining a context of each unlabelled text in the plurality of unlabelled texts using one or more Artificial Intelligence (AI) techniques; performing an augmentation operation on each unlabelled text and the context corresponding to each unlabelled text; generating a plurality of positive and negative query-context pairs for the unlabelled text and the corresponding context based on the augmentation operation using the self-supervised retrieval model; and configure the self-supervised retrieval model to retrieve the response from the one or more data sources in response to each unlabelled text.”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Kar to implement the encoder on a system including a processor and executable instructions, at least because doing so would enable self-supervised contrastive learning to be performed on a computer.
Claim 13 is an apparatus claim corresponding to claim 1 and is similarly rejected. Yan does not expressly disclose [a]n autoencoder apparatus for optimizing machine learning by soft prompting a machine learning language model (“MLLM”) trained on a corpus of unlabeled sample data for natural language classification, the autoencoder apparatus comprising: an encoder configured to encode a query into a soft computer prompt corresponding to a target response from the MLLM;. (but see Kar ¶ 15 (“ In an embodiment, the present disclosure discloses a non-transitory computer readable storage media. The storage media comprises instructions that when executed causes a processor to receive an input text indicating a query made by a user; and retrieve a response from one or more data sources based on the input text using a self-supervised retrieval model. The self-supervised retrieval model is pre-trained by: providing a plurality of unlabelled texts as training input data to the self-supervised retrieval model; determining a context of each unlabelled text in the plurality of unlabelled texts using one or more Artificial Intelligence (AI) techniques; performing an augmentation operation on each unlabelled text and the context corresponding to each unlabelled text; generating a plurality of positive and negative query-context pairs for the unlabelled text and the corresponding context based on the augmentation operation using the self-supervised retrieval model; and configure the self-supervised retrieval model to retrieve the response from the one or more data sources in response to each unlabelled text.”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Kar to implement the encoder on a system including a processor and executable instructions, at least because doing so would enable self-supervised contrastive learning to be performed on a computer.
Claim 20 is a system claim corresponding to claim 1 and is similarly rejected. Yan does not expressly disclose [a] computer system for optimizing machine learning by soft prompting a machine learning language model (“MLLM”) trained on a corpus of unlabeled sample data for natural language classification, the computer system having a processor, a computer-readable memory, a computer-readable tangible storage device, and program instructions stored on the computer-readable storage device for execution by a processor via the computer-readable memory (but see Kar ¶ 15 (“ In an embodiment, the present disclosure discloses a non-transitory computer readable storage media. The storage media comprises instructions that when executed causes a processor to receive an input text indicating a query made by a user; and retrieve a response from one or more data sources based on the input text using a self-supervised retrieval model. The self-supervised retrieval model is pre-trained by: providing a plurality of unlabelled texts as training input data to the self-supervised retrieval model; determining a context of each unlabelled text in the plurality of unlabelled texts using one or more Artificial Intelligence (AI) techniques; performing an augmentation operation on each unlabelled text and the context corresponding to each unlabelled text; generating a plurality of positive and negative query-context pairs for the unlabelled text and the corresponding context based on the augmentation operation using the self-supervised retrieval model; and configure the self-supervised retrieval model to retrieve the response from the one or more data sources in response to each unlabelled text.”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Kar to implement the encoder on a system including a processor and executable instructions, at least because doing so would enable self-supervised contrastive learning to be performed on a computer.
Claims 2, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Yan and Kar as applied to claims 1 and 13 above, and further in view of Xu, Ziyun, et al. "Making pre-trained language models end-to-end few-shot learners with contrastive prompt tuning." Proceedings of the sixteenth ACM international conference on web search and data mining. 2023 (“Xu”).
Regarding claim 2, Yan, in view of Kar, discloses the invention of claim 1 as discussed above. Yan does not expressly disclose wherein the execution of the program instructions further causes the computing device to:
inference a response to the soft computer prompt; (but see Xu Section 2.2 (“The input format of our approach is significantly different from previous works to facilitate task-invariant continuous prompt learning. To be more specific, in contrast to [4], we have three additional types of special tokens used as the inputs to the PLM: i) [PRO] (Prompt): the placeholder for continuous prompt embeddings; ii) [TMSK] (Token Mask): the token mask of the input texts for optimizing the auxiliary MLM loss; iii)[OMSK](Output Mask):the mask as a placeholder to generate the output result.”))
determine an MLLM loss by comparing the response to the target response; and (but see Xu Section 2.3 page 441 1st column (“Minimizing L𝐶𝐿(𝑖) alone may be insufficient as it does not consider sample difficulty. For example, if 𝑠𝑖,𝑝 = 0.2 and 𝑠𝑖,𝑝′ = 0.95 where 𝑝,𝑝′ ∈ 𝑃(𝑖). The model should pay more attention to 𝑠𝑖,𝑝 to reach the optima, and less attention to 𝑠𝑖,𝑝′ to avoid model over fitting. Inspired by [42], we introduce pair-wise relaxation factors and propose a new loss function named Pair-wise Cost-sensitive Contrastive Loss (PCCL) as follows:
PNG
media_image2.png
82
574
media_image2.png
Greyscale
”)
in response to determining the MLLM loss is non-zero, train the MLLM to minimize the MLLM loss (but see Xu Section 2.4 page 441 2nd column (“Approximately, the problem of minimizing L𝐶𝐶𝐿(𝑖) ′ is equivalent of optimizing the loss function L𝑇𝐿(𝑖) (with the margin omit ted) as follows:
PNG
media_image3.png
58
560
media_image3.png
Greyscale
”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Xu to optimize the pretrained ML model by optimizing the loss function based on pairwise cost-sensitive contrastive loss, at least because doing so would prevent overfitting.
Regarding claim 15, Yan, in view of Kar, discloses the invention of claim 13 as discussed above. Yan does not expressly disclose wherein the determine the contrastive loss further comprises:
embed a copy of the prompt vector into a positive sample vector in the representation space; and embed a sample data outside the minibatch into a negative sample vector in the representation space (but see Xu page 439 Figure 2
PNG
media_image4.png
564
558
media_image4.png
Greyscale
;
Xu Section 2.3 page 440 (“In our work, we propose a novel verbalizer-free approach to generate model outputs based on PCCL. During training, denote B as the collection of instances in a batch (B⊂D). Each instance 𝑖∈B can be treated as an anchor, with the label denoted as 𝑦𝑖. We also have the positive set 𝑃(𝑖) and the negative set 𝑁(𝑖) w.r.t. the instance 𝑖 and the batch B, i.e., 𝑃(𝑖)={𝑗|𝑗 ≠𝑖,𝑦𝑗 =𝑦𝑖, 𝑗 ∈B}, 𝑁(𝑖)={𝑗|𝑦𝑗≠𝑦𝑖, 𝑗∈B}.”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Xu to encode samples as a positive anchor and negative samples in a contrastive loss model, at least because doing so would enable training the model to more readily distinguish between specific prompts.
Regarding claim 16, Yan, in view of Kar and Xu, discloses the invention of claim 15 as discussed above. Yan further discloses wherein the positive sample vector comprises an augmented copy of the prompt vector (Yan page 5068 Figure 3
PNG
media_image5.png
486
1102
media_image5.png
Greyscale
;
Yan page 5068 Section 3.2 (“We explore four different data augmentation strategies to generate views for contrastive learning, including adversarial attack (Goodfellow et al., 2014; Kurakin et al., 2016), token shuffling, cutoff (Shen et al., 2020) and dropout (Hinton et al., 2012), as illustrated in Figure 3.”)).
Claims 3 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Yan and Kar as applied to claims 1 and 13 above, and further in view of Liu, X., et al. "Gpt understands, too. arXiv preprint arXiv: 2103." (2021) (“Liu”).
Regarding claim 3, Yan, in view of Kar, discloses the invention of claim 1 as discussed above. Yan does not expressly disclose wherein the encode the query further comprises: allocate a computer programmable memory to a plurality of tokens; divide the query into a plurality of query segments; store each query segment to one of the tokens in the plurality of tokens; and mask one of the tokens to withhold the corresponding query segment from the soft computer prompt (but see Liu Section 2.2 (“Prompting was proposed in the format of discrete tokens (Schick and Schütze, 2020). Let [Di] be a discrete prompt token. Each prompt can be described as a template T = {[D0:i], x, [D(i+1):j], y, [D(j+1):k]}, which could organize the labeled data (including the inputs x and the label y) into a sequence of text tokens, such that the task could be reformulated as filling in the blanks of the input text. For example, for the task of predicting a country’s capital (LAMA-TREx P36), a prompt could be “The capital of [INPUT] is [LA BEL].” With a piece of labeled data “(Britain, Lon don)”, the reformulated text would be “The capital of Britain is [MASK].”, where “[MASK]" should predict the given label “London”. Both discrete prompts and discrete data are together mapped into input embeddings:
PNG
media_image6.png
48
422
media_image6.png
Greyscale
through the pretrained embedding layer, where e ∈ R|V|×d.”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Liu to mask one of the tokens in the input text, at least because doing so would enable reformulating the PLM task as filling in the blanks of the input text.
Claim 14 is an apparatus claim corresponding to claim 3 and is similarly rejected.
Claims 4 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Yan, Kar, and Liu as applied to claim 3 above, and further in view of Xu.
Regarding claim 4, Yan, in view of Kar and Liu, discloses the invention of claim 3 as discussed above. Yan does not expressly disclose wherein the determine the contrastive loss further comprises:
embed a copy of the prompt vector into a positive sample vector in the representation space; and embed a sample data outside the minibatch into a negative sample vector in the representation space (but see Xu page 439 Figure 2
PNG
media_image4.png
564
558
media_image4.png
Greyscale
;
Xu Section 2.3 page 440 (“In our work, we propose a novel verbalizer-free approach to generate model outputs based on PCCL. During training, denote B as the collection of instances in a batch (B⊂D). Each instance 𝑖∈B can be treated as an anchor, with the label denoted as 𝑦𝑖. We also have the positive set 𝑃(𝑖) and the negative set 𝑁(𝑖) w.r.t. the instance 𝑖 and the batch B, i.e., 𝑃(𝑖)={𝑗|𝑗 ≠𝑖,𝑦𝑗 =𝑦𝑖, 𝑗 ∈B}, 𝑁(𝑖)={𝑗|𝑦𝑗≠𝑦𝑖, 𝑗∈B}.”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Xu to encode samples as a positive anchor and negative samples in a contrastive loss model, at least because doing so would enable training the model to more readily distinguish between specific prompts.
Regarding claim 5, Yan, in view of Kar, Liu, and Xiu, discloses the invention of claim 4 as discussed above. Yan further discloses wherein the positive sample vector comprises an augmented copy of the prompt vector (Yan page 5068 Figure 3
PNG
media_image5.png
486
1102
media_image5.png
Greyscale
;
Yan page 5068 Section 3.2 (“We explore four different data augmentation strategies to generate views for contrastive learning, including adversarial attack (Goodfellow et al., 2014; Kurakin et al., 2016), token shuffling, cutoff (Shen et al., 2020) and dropout (Hinton et al., 2012), as illustrated in Figure 3.”)).
Claims 6, 7, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Yan, Kar, and Liu as applied to claims 3 and 14 above, and further in view of Meng, Yu, et al. "COCO-LM: Correcting and contrasting text sequences for language model pretraining. CoRR abs/2102.08473 (2021)." arXiv preprint arXiv:2102.08473 (2021) (“Meng”).
Regarding claim 6, Yan, in view of Kar and Liu, discloses the invention of claim 3 as discussed above. Yan et al. does not expressly disclose wherein the encode the query further comprises:
mask one of the tokens of the plurality of tokens to form a first encoder configuration; embed a first soft computer prompt with the first encoder configuration; set the first soft computer prompt to the prompt vector; mask a different one of the tokens of the plurality of tokens to form a second encoder configuration; embed a second soft computer prompt with the second encoder configuration; and set the second soft computer prompt to a positive sample vector (but see Meng Figure 2
PNG
media_image7.png
384
958
media_image7.png
Greyscale
(figure 2 illustrates how the original sequence is masked in two different ways prior to encoding);
Meng Section 3.3 page 4 “COCO-LM also employs an auxiliary Transformer to construct the corrupted text sequence, as in Eqn. (1), but it introduces two new pretraining tasks upon the corrupted sequences to address the challenges previously described. In the rest of this section, we present these two tasks and then the detailed configurations of COCO-LM. Its framework is illustrated in Figure 2.”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Meng to construct the positive and negative samples by corrupting the same text input in two different elements by masking, at least because doing so would combine the benefits of replaced token detection and language modeling. See Meng page 2 (“In this paper, we present a new self-supervised learning approach, COCO-LM, that pretrains Language Models by COrrecting and COntrasting corrupted text sequences. Following ELECTRA-style pretraining, COCO-LM employs an auxiliary model to corrupt the input texts, upon which it introduces two new pretraining tasks for the main Transformer, one at token level and one at sequence level. The token-level task, corrective language modeling (CLM), pretrains the main Transformer to detect and correct the tokens in the corrupted sequences. It uses a multi-task setup to combine the benefits of replaced token detection and language modeling. The sequence-level task, sequence contrastive learning (SCL), pretrains the model to align text sequences originated from the same source sequence and enforce uniformity of the representation space.”).
Claim 17 is an apparatus claim corresponding to claim 6 and is similarly rejected.
Regarding claim 7, Yan, in view of Kar and Liu, discloses the invention of claim 3 as discussed above. Yan does not expressly disclose wherein the encode the query further comprises: mask one or more of the tokens of the plurality of tokens to form a first encoder configuration; embed a first soft computer prompt with the first encoder configuration; set the first soft computer prompt to the prompt vector; and mask a different one or more of the tokens of the plurality of tokens to form a second encoder configuration; embed a second soft computer prompt with the second encoder configuration; and set the second soft computer prompt to a positive sample vector (but see Meng Figure 2
PNG
media_image7.png
384
958
media_image7.png
Greyscale
(figure 2 illustrates how the original sequence is masked in two different ways prior to encoding);
Meng Section 3.3 page 4 “COCO-LM also employs an auxiliary Transformer to construct the corrupted text sequence, as in Eqn. (1), but it introduces two new pretraining tasks upon the corrupted sequences to address the challenges previously described. In the rest of this section, we present these two tasks and then the detailed configurations of COCO-LM. Its framework is illustrated in Figure 2.”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yan to incorporate the teachings of Meng to construct the positive and negative samples by corrupting the same text input in two different elements by masking, at least because doing so would combine the benefits of replaced token detection and language modeling. See Meng page 2 (“In this paper, we present a new self-supervised learning approach, COCO-LM, that pretrains Language Models by COrrecting and COntrasting corrupted text sequences. Following ELECTRA-style pretraining, COCO-LM employs an auxiliary model to corrupt the input texts, upon which it introduces two new pretraining tasks for the main Transformer, one at token level and one at sequence level. The token-level task, corrective language modeling (CLM), pretrains the main Transformer to detect and correct the tokens in the corrupted sequences. It uses a multi-task setup to combine the benefits of replaced token detection and language modeling. The sequence-level task, sequence contrastive learning (SCL), pretrains the model to align text sequences originated from the same source sequence and enforce uniformity of the representation space.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Zeng, Jiali, et al. "Contrastive learning with prompt-derived virtual semantic prototypes for unsupervised sentence embedding." Findings of the association for computational linguistics: EMNLP 2022. 2022.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHID KHAN whose telephone number is (571)270-0419. The examiner can normally be reached M-F, 9-5 est.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571)272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAHID K KHAN/Primary Examiner, Art Unit 2146
