Detailed Action
Applicant amended claims 1, 3, 10, 12, 18 and 20 and presented claims 1-20 on 04/08/2026 for reconsideration.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Owen Du, “Large Language Model Augmented Contrastive Sentence Representation Learning” (Du), in view of Mann et al., Patent No.: US 11,790,411 B1 (Mann).
Claim 1. Du teaches:
A method for assessing one or more textual embeddings using an unlabeled dataset, the method comprising:
retrieving from a database the unlabeled dataset; providing the unlabeled dataset to a language learning model; generating a labeled dataset using the language learning model based at least on the unlabeled dataset; (based on the Spec., ¶ 29, a record is unlabeled dataset, a paraphrase of the unlabeled record is a semantic dataset, an indication of the similarity/dissimilarity between the unlabeled record and the paraphrased record is a labeled dataset: Du, sec. 3.2, “In order to generate 5 paraphrases for each sentence xi in the sentence dataset, we create the following prompts for ChatGPT and Vicuna: 1. Generate 5 paraphrases of the following: xi 2. Generate 5 new sentences, which are semantically similar but lexically and syntactically divergent from the following: xi 3. I want you to act as a paraphrasing tool. I will provide you a sentence and your task is to generate 5 paraphrases. These will act as augmented data that I will use to train a sentence embedding model evaluated on a semantic text similarity task. The sentence is: xi 4. On a scale of 1 to 5, where 1 is the most semantically similar but least lexically divergent and 5 is the least semantically similar but most lexically divergent, generate a paraphrase for each scale of the following: xi 5. Generate 5 paraphrases, where the first paraphrase has the highest semantic similarity but the lowest lexical divergence and the last paraphrase has the lowest semantic similarity and the highest lexical divergence, of the following: xi”)
constructing a proxy task using one or more portions of the labeled dataset, wherein the proxy task comprises the one or more textual embeddings along with one or more evaluation metrics; wherein the one or more textual embeddings comprises a corresponding semantic encoding technique that vectorizes each record in the unlabeled dataset and a corresponding record in a semantic dataset; (based on the Spec., ¶ 29, a record is unlabeled dataset, a paraphrase of the unlabeled record is a semantic dataset, an indication of the similarity/dissimilarity between the unlabeled record and the paraphrased record is a labeled dataset: Du, 3.2.2, “To evaluate the quality of our generated paraphrases we use a simpler method by first calculating the semantic similarity sim. This is done using the cosine similarity of the original sentence embeddings and paraphrase embeddings generated by the supervised RoBERTalarge SimCSE model. Then, we normalize the values so they lie between 0 and 1. To measure the differences in lexical and syntactical structure of the original sentence compared to the paraphrases we use the BLEU score [21]. It counts how many unigrams, bigrams, trigrams and four-grams occur in the hypothesis (original sentence), as well as in the reference (paraphrases). Thus, for our generated paraphrases we aim to reach a high semantic similarity score while keeping the BLEU score low”)
Du did not disclose but Mann discloses “the unlabeled dataset associated with a facility” and selecting one of the one or more machine learning models based on the one or more performance metrics; and optimizing one or more operations in the facility using the selected machine learning model. (Mann, unlabeled, e.g., unclassified customer complaints are labeled/classified using a machine learning model; the accuracy of the machine learning further verified and based on the verified classification a customer complaint is elevated to be resolved: 1:33-55, “a computing system may receive one or more messages including customer service inquiries, such as customer complaints”, 10:30-45, “A machine learning algorithm or function (e.g., a word embedding algorithm) is trained to create machine learning model 210 configured to accept an input sequence of tokens associated with a message and produce, using complaint classification unit 220, any one or combination of an output classification of whether the incoming message is high-risk or low-risk, an output classification of a complaint type associated with the message, and an output classification of a complaint reason associated with the message. Classification unit 220 may generate classifications based on token vector characteristics 207. For example, classification unit 220 may classify an incoming message as high-risk if a set of token vectors associated with the incoming message have greater than a threshold level of similarity to known characteristics of high-risk messages, as identified by token vector characteristics 207. Training unit 230 may output token vector characteristics 207 and token vector database 208 to storage devices 206”; 6:12-21, one operation, e.g., resolving customer compliant, is optimized based on the correctly classified message by addressing and elevating the message: “Elevating a high-risk message may include flagging the high-risk message as including a rather urgent, grave, or serious complaint or criticism of one or more services or aspects of the business or organization that provides message center 12. Such flagging of a high-risk message may cause complaint management system 29 to prioritize an addressing of the high-risk message so that a probability that the high-risk message is resolved is greater than a probability that high-risk messages are resolved by systems that do not flag high-risk messages”)
Du describes that “learning sentence embeddings is a fundamental problem which has been studied thoroughly in past papers. Learning deep representations of sentences allows us to perform various downstream tasks such as text classification, sentiment analysis and machine translation, as the learned embeddings contain information about semantic, syntactic and lexical structures of the sentence” and provides for using data augmentation methods provided by ChatGPT and other large language models (LLM) for generating a sentence as shown above.
Mann describes that “ Machine learning algorithms, such as the function of machine learning model 210, may be trained using a training process to create data-specific models, such as machine learning model 210 based on training data 209. After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., match a set of token vectors representing an incoming message to one or more known characteristics associated with high-risk messages, match a set of token vectors representing an incoming message to one or more known characteristics associated with each complaint type of a set of complaint types, and/or match a set of token vectors representing an incoming message to one or more known characteristics associated with each complaint reason of a set of complaint reasons). The training process may implement a set of training data (e.g., training data 209) to create the model.”
It would have been obvious before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine the applied references for disclosing “the unlabeled dataset associated with a facility” and selecting one of the one or more machine learning models based on the one or more performance metrics; and optimizing one or more operations in the facility using the selected machine learning model” because doing so would allow the person to replace the “sentence dataset” in Du with “service complaints” in Mann and perform a task of text/customer complaint classification using data augmentation methods provided by ChatGPT and other large language models (LLM) for resolving a customer complaint based on the classified sentence/complaint.
Claim 10. Du teaches:
A system for assessing one or more textual embeddings using an unlabeled dataset, the system comprising: a processor; a memory communicatively coupled to the processor, wherein the memory comprises one or more instructions which when executed by the processor, cause the processor to:
retrieve from a database the unlabeled dataset; provide the unlabeled dataset to a language learning model; generate a labeled dataset using the language learning model based at least on the unlabeled dataset; (based on the Spec., ¶ 29, a record is unlabeled dataset, a paraphrase of the unlabeled record is a semantic dataset, an indication of the similarity/dissimilarity between the unlabeled record and the paraphrased record is a labeled dataset: Du, sec. 3.2, a language learning model is prompted to generate paraphrases for unlabeled input; the paraphrases are semantic dataset and an indication of the similarity/dissimilarity is a labeled dataset: “In order to generate 5 paraphrases for each sentence xi in the sentence dataset, we create the following prompts for ChatGPT and Vicuna: 1. Generate 5 paraphrases of the following: xi 2. Generate 5 new sentences, which are semantically similar but lexically and syntactically divergent from the following: xi 3. I want you to act as a paraphrasing tool. I will provide you a sentence and your task is to generate 5 paraphrases. These will act as augmented data that I will use to train a sentence embedding model evaluated on a semantic text similarity task. The sentence is: xi 4. On a scale of 1 to 5, where 1 is the most semantically similar but least lexically divergent and 5 is the least semantically similar but most lexically divergent, generate a paraphrase for each scale of the following: xi 5. Generate 5 paraphrases, where the first paraphrase has the highest semantic similarity but the lowest lexical divergence and the last paraphrase has the lowest semantic similarity and the highest lexical divergence, of the following: xi”)
construct a proxy task using one or more portions of the labeled dataset, wherein the proxy task comprises the one or more textual embeddings along with one or more evaluation metrics, wherein the one or more textual embeddings comprises a corresponding semantic encoding technique that vectorizes each record in the unlabeled dataset and a corresponding record in a semantic dataset; (based on the Spec., ¶ 29, a record is unlabeled dataset, a paraphrase of the unlabeled record is a semantic dataset, an indication of the similarity/dissimilarity between the unlabeled record and the paraphrased record is a labeled dataset: Du, 3.2.2, “To evaluate the quality of our generated paraphrases we use a simpler method by first calculating the semantic similarity sim. This is done using the cosine similarity of the original sentence embeddings and paraphrase embeddings generated by the supervised RoBERTalarge SimCSE model. Then, we normalize the values so they lie between 0 and 1. To measure the differences in lexical and syntactical structure of the original sentence compared to the paraphrases we use the BLEU score [21]. It counts how many unigrams, bigrams, trigrams and four-grams occur in the hypothesis (original sentence), as well as in the reference (paraphrases). Thus, for our generated paraphrases we aim to reach a high semantic similarity score while keeping the BLEU score low”)
Du did not disclose but Mann discloses “the unlabeled dataset associated with a facility” and select one of the one or more machine learning models based on the one or more performance metrics; and optimize one or more operations in the facility using the selected machine learning model. (Mann, unlabeled, e.g., unclassified customer complaints are labeled/classified using a machine learning model; the accuracy of the machine learning further verified and based on the verified classification a customer complaint is elevated to be resolved: 1:33-55, “a computing system may receive one or more messages including customer service inquiries, such as customer complaints”, 10:30-45, “A machine learning algorithm or function (e.g., a word embedding algorithm) is trained to create machine learning model 210 configured to accept an input sequence of tokens associated with a message and produce, using complaint classification unit 220, any one or combination of an output classification of whether the incoming message is high-risk or low-risk, an output classification of a complaint type associated with the message, and an output classification of a complaint reason associated with the message. Classification unit 220 may generate classifications based on token vector characteristics 207. For example, classification unit 220 may classify an incoming message as high-risk if a set of token vectors associated with the incoming message have greater than a threshold level of similarity to known characteristics of high-risk messages, as identified by token vector characteristics 207. Training unit 230 may output token vector characteristics 207 and token vector database 208 to storage devices 206”; 6:12-21, one operation, e.g., resolving customer compliant, is optimized based on the correctly classified message by addressing and elevating the message: “Elevating a high-risk message may include flagging the high-risk message as including a rather urgent, grave, or serious complaint or criticism of one or more services or aspects of the business or organization that provides message center 12. Such flagging of a high-risk message may cause complaint management system 29 to prioritize an addressing of the high-risk message so that a probability that the high-risk message is resolved is greater than a probability that high-risk messages are resolved by systems that do not flag high-risk messages”)
Du describes that “learning sentence embeddings is a fundamental problem which has been studied thoroughly in past papers. Learning deep representations of sentences allows us to perform various downstream tasks such as text classification, sentiment analysis and machine translation, as the learned embeddings contain information about semantic, syntactic and lexical structures of the sentence” and provides for using data augmentation methods provided by ChatGPT and other large language models (LLM) for generating a sentence as shown above.
Mann describes that “ Machine learning algorithms, such as the function of machine learning model 210, may be trained using a training process to create data-specific models, such as machine learning model 210 based on training data 209. After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., match a set of token vectors representing an incoming message to one or more known characteristics associated with high-risk messages, match a set of token vectors representing an incoming message to one or more known characteristics associated with each complaint type of a set of complaint types, and/or match a set of token vectors representing an incoming message to one or more known characteristics associated with each complaint reason of a set of complaint reasons). The training process may implement a set of training data (e.g., training data 209) to create the model.”
It would have been obvious before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine the applied references for disclosing the unlabeled dataset associated with a facility and select one of the one or more machine learning models based on the one or more performance metrics; and optimize one or more operations in the facility using the selected machine learning model” because doing so would allow the person to replace the “sentence dataset” in Du with “service complaints” in Mann and perform a task of text/customer complaint classification using data augmentation methods provided by ChatGPT and other large language models (LLM) for resolving a customer complaint based on the classified sentence/complaint.
Claim 18. Du teaches:
A non-transitory, computer-readable storage medium having stored thereon executable instructions that, when executed by one or more processors, cause the one or more processors to:
retrieve from a database the unlabeled dataset; provide the unlabeled dataset to a language learning model; generate a labeled dataset using the language learning model based at least on the unlabeled dataset; (based on the Spec., ¶ 29, a record is unlabeled dataset, a paraphrase of the unlabeled record is a semantic dataset, an indication of the similarity/dissimilarity between the unlabeled record and the paraphrased record is a labeled dataset: Du, sec. 3.2, a language learning model is prompted to generate paraphrases for unlabeled input; the paraphrases are semantic dataset and an indication of the similarity/dissimilarity is a labeled dataset: “In order to generate 5 paraphrases for each sentence xi in the sentence dataset, we create the following prompts for ChatGPT and Vicuna: 1. Generate 5 paraphrases of the following: xi 2. Generate 5 new sentences, which are semantically similar but lexically and syntactically divergent from the following: xi 3. I want you to act as a paraphrasing tool. I will provide you a sentence and your task is to generate 5 paraphrases. These will act as augmented data that I will use to train a sentence embedding model evaluated on a semantic text similarity task. The sentence is: xi 4. On a scale of 1 to 5, where 1 is the most semantically similar but least lexically divergent and 5 is the least semantically similar but most lexically divergent, generate a paraphrase for each scale of the following: xi 5. Generate 5 paraphrases, where the first paraphrase has the highest semantic similarity but the lowest lexical divergence and the last paraphrase has the lowest semantic similarity and the highest lexical divergence, of the following: xi”)
construct a proxy task using one or more portions of the labeled dataset, wherein the proxy task comprises the one or more textual embeddings along with one or more evaluation metrics, wherein the one or more textual embeddings comprises a corresponding semantic encoding technique that vectorizes each record in the unlabeled dataset and a corresponding record in a semantic dataset; (based on the Spec., ¶ 29, a record is unlabeled dataset, a paraphrase of the unlabeled record is a semantic dataset, an indication of the similarity/dissimilarity between the unlabeled record and the paraphrased record is a labeled dataset: Du, 3.2.2, “To evaluate the quality of our generated paraphrases we use a simpler method by first calculating the semantic similarity sim. This is done using the cosine similarity of the original sentence embeddings and paraphrase embeddings generated by the supervised RoBERTalarge SimCSE model. Then, we normalize the values so they lie between 0 and 1. To measure the differences in lexical and syntactical structure of the original sentence compared to the paraphrases we use the BLEU score [21]. It counts how many unigrams, bigrams, trigrams and four-grams occur in the hypothesis (original sentence), as well as in the reference (paraphrases). Thus, for our generated paraphrases we aim to reach a high semantic similarity score while keeping the BLEU score low”)
Du did not disclose but Mann discloses “the unlabeled dataset associated with a facility” and select one of the one or more machine learning models based on the one or more performance metrics; and optimize one or more operations in the facility using the selected machine learning model. (Mann, unlabeled, e.g., unclassified customer complaints are labeled/classified using a machine learning model; the accuracy of the machine learning further verified and based on the verified classification a customer complaint is elevated to be resolved: 1:33-55, “a computing system may receive one or more messages including customer service inquiries, such as customer complaints”, 10:30-45, “A machine learning algorithm or function (e.g., a word embedding algorithm) is trained to create machine learning model 210 configured to accept an input sequence of tokens associated with a message and produce, using complaint classification unit 220, any one or combination of an output classification of whether the incoming message is high-risk or low-risk, an output classification of a complaint type associated with the message, and an output classification of a complaint reason associated with the message. Classification unit 220 may generate classifications based on token vector characteristics 207. For example, classification unit 220 may classify an incoming message as high-risk if a set of token vectors associated with the incoming message have greater than a threshold level of similarity to known characteristics of high-risk messages, as identified by token vector characteristics 207. Training unit 230 may output token vector characteristics 207 and token vector database 208 to storage devices 206”; 6:12-21, one operation, e.g., resolving customer compliant, is optimized based on the correctly classified message by addressing and elevating the message: “Elevating a high-risk message may include flagging the high-risk message as including a rather urgent, grave, or serious complaint or criticism of one or more services or aspects of the business or organization that provides message center 12. Such flagging of a high-risk message may cause complaint management system 29 to prioritize an addressing of the high-risk message so that a probability that the high-risk message is resolved is greater than a probability that high-risk messages are resolved by systems that do not flag high-risk messages”)
Du describes that “learning sentence embeddings is a fundamental problem which has been studied thoroughly in past papers. Learning deep representations of sentences allows us to perform various downstream tasks such as text classification, sentiment analysis and machine translation, as the learned embeddings contain information about semantic, syntactic and lexical structures of the sentence” and provides for using data augmentation methods provided by ChatGPT and other large language models (LLM) for generating a sentence as shown above.
Mann describes that “ Machine learning algorithms, such as the function of machine learning model 210, may be trained using a training process to create data-specific models, such as machine learning model 210 based on training data 209. After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., match a set of token vectors representing an incoming message to one or more known characteristics associated with high-risk messages, match a set of token vectors representing an incoming message to one or more known characteristics associated with each complaint type of a set of complaint types, and/or match a set of token vectors representing an incoming message to one or more known characteristics associated with each complaint reason of a set of complaint reasons). The training process may implement a set of training data (e.g., training data 209) to create the model.”
It would have been obvious before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine the applied references for disclosing the unlabeled dataset associated with a facility and select one of the one or more machine learning models based on the one or more performance metrics; and optimize one or more operations in the facility using the selected machine learning model” because doing so would allow the person to replace the “sentence dataset” in Du with “service complaints” in Mann and perform a task of text/customer complaint classification using data augmentation methods provided by ChatGPT and other large language models (LLM) for resolving a customer complaint based on the classified sentence/complaint.
Claim 2. The method of claim 1, wherein retrieving the unlabeled dataset comprises:
selecting the unlabeled dataset based on one or more requirements in the facility, wherein a requirement of the one or more requirements corresponds to at least one operation that is to be optimized in the facility; and retrieving the unlabeled dataset based on the selection. (Du, sec. 3, “we decide to take the dataset from Method 1-1 as the final dataset and use its sentences to generate paraphrases”; sec. 1, “Learning deep representations of sentences allows us to perform various downstream tasks such as text classification, sentiment analysis and machine translation, as the learned embeddings contain information about semantic, syntactic and lexical structures of the sentence”; Mann, 3:27-44, a set of messages for handling “customer service inquiries focused on customer accounts with the business or other services provided by the business, e.g., servicing existing accounts, opening new accounts, servicing existing loans, and opening new loans” is selected for classification)
Claims 11 and 19 are rejected under the same rationale as above.
Claim 3. The method of claim 1, wherein providing the unlabeled dataset comprises:
receiving one or more instruction prompts from a user via a user interface, wherein the one or more instruction prompts relate to: generating the semantic dataset relative to the unlabeled dataset and labeling the unlabeled dataset; and inputting the unlabeled dataset along with the one or more instruction prompts to the language learning model. (Du, a language learning model is prompted to generate semantically paraphrases for unlabeled input; the paraphrases are labeled based on the degree of similarity to input: Du, sec. 2.1, “The aim of contrastive learning is to generate effective embeddings of sentences by pulling representations of semantically similar neighbors together”, Du, sec. 3.2, “In order to generate 5 paraphrases for each sentence xi in the sentence dataset, we create the following prompts for ChatGPT and Vicuna: 1. Generate 5 paraphrases of the following: xi 2. Generate 5 new sentences, which are semantically similar but lexically and syntactically divergent from the following: xi 3. I want you to act as a paraphrasing tool. I will provide you a sentence and your task is to generate 5 paraphrases. These will act as augmented data that I will use to train a sentence embedding model evaluated on a semantic text similarity task. The sentence is: xi 4. On a scale of 1 to 5, where 1 is the most semantically similar but least lexically divergent and 5 is the least semantically similar but most lexically divergent, generate a paraphrase for each scale of the following: xi 5. Generate 5 paraphrases, where the first paraphrase has the highest semantic similarity but the lowest lexical divergence and the last paraphrase has the lowest semantic similarity and the highest lexical divergence, of the following: xi”; Mann, 10:30-45, “After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., match a set of token vectors representing an incoming message to one or more known characteristics associated with high-risk messages, match a set of token vectors representing an incoming message to one or more known characteristics associated with each complaint type of a set of complaint types, and/or match a set of token vectors representing an incoming message to one or more known characteristics associated with each complaint reason of a set of complaint reasons). The training process may implement a set of training data (e.g., training data 209) to create the model)
Claims 12 and 20 are rejected under the same rationale as above.
Claim 4. The method of claim 3, wherein generating the labeled dataset comprises:
analyzing the unlabeled dataset along with the one or more instruction prompts by the language learning model, wherein the unlabeled dataset comprises one or more first records; (Du, a prompt instructs a language learning model to generate a paraphrase along with a similarity score/label: Du, sec. 3.2, “In order to generate 5 paraphrases for each sentence xi in the sentence dataset, we create the following prompts for ChatGPT and Vicuna: 1. Generate 5 paraphrases of the following: xi 2. Generate 5 new sentences, which are semantically similar but lexically and syntactically divergent from the following: xi 3. I want you to act as a paraphrasing tool. I will provide you a sentence and your task is to generate 5 paraphrases. These will act as augmented data that I will use to train a sentence embedding model evaluated on a semantic text similarity task. The sentence is: xi 4. On a scale of 1 to 5, where 1 is the most semantically similar but least lexically divergent and 5 is the least semantically similar but most lexically divergent, generate a paraphrase for each scale of the following: xi 5. Generate 5 paraphrases, where the first paraphrase has the highest semantic similarity but the lowest lexical divergence and the last paraphrase has the lowest semantic similarity and the highest lexical divergence, of the following: xi”)
generating the semantic dataset based at least on the unlabeled dataset, wherein the semantic dataset comprises a corresponding second record for each of the one or more first records in the unlabeled dataset; (see above, and sec. 3.3, “Paraphrase Generation” )
comparing each first record in the unlabeled dataset with its corresponding second record in the semantic dataset; (see above, and sec. 3.3, “Paraphrase Generation”)
labeling each first record in the unlabeled dataset along with its corresponding second record in the semantic dataset with a label, wherein the label corresponds to an indicator indicative of a similarity level between a first record in the unlabeled dataset when compared to its corresponding second record in the semantic dataset; and (see above, Du, 3.3.1 “Paraphrase Generation with ChatGPT” and 3.3.2 “Paraphrase Generation with Vicuna”)
outputting the labeled dataset by the language learning model. (see above, Du, 3.3.1 “Paraphrase Generation with ChatGPT” and 3.3.2 “Paraphrase Generation with Vicuna”: “The results show that the paraphrases generated by Vicuna are more similar to the original sentence in terms of semantic similarity, however, in lexical and syntactical structure as well. In section 4.2.1 we analyze whether these paraphrases perform better or worse than examples from ChatGPT”)
Claim 13 is rejected under the same rationale as above.
Claim 5. The method of claim 1, further comprising rendering the labeled dataset on a user interface. (Du, 3.2.2, labeled datasets are rendered on a user interface for further validation: “To evaluate the quality of our generated paraphrases we use a simpler method by first calculating the semantic similarity sim. This is done using the cosine similarity of the original sentence embeddings and paraphrase embeddings generated by the supervised RoBERTalarge SimCSE model. Then, we normalize the values so they lie between 0 and 1”; 3.3.2, ““We find that prompt 2 from section 3.2 does not work well with Vicuna, as it generates new information not contained in the original. Therefore, we use prompt 1 and find following evaluation results for 100 sentences and their respective paraphrases”)
Claim 6. The method of claim 1, further comprising:
employing one or more sampling techniques to categorize the labeled dataset, wherein a sampling technique of the one or more sampling techniques corresponds to a stratified sampling technique; (Du, sec. 4.1, wherein “These embeddings are used in the contrastive loss objective, which pulls the representations of xi and pij (positive pair) closer together and pushes all other examples within the batch (negative examples) further apart” indicates that a stratified sampling technique is used for gathering positive pairs and negative pairs)
categorizing the labeled dataset into the one or more portions using the one or more sampling techniques, wherein the one or more portions comprise training dataset, validation dataset, and observation dataset; (Du, an applied similarity score to labeled data set provides for selecting a portion of the result as desired: sec. 3.2.2, “for our generated paraphrases we aim to reach a high semantic similarity score while keeping the BLEU score low. This leads us to the following metric… where xi is the original sentence and pi is the corresponding set of 5 paraphrases. The score ranges from 0 to 1 and the higher the value the better paraphrase quality we get” and wherein “We find that prompt 2 from section 3.2 does not work well with Vicuna, as it generates new information not contained in the original” indicates observing and observation dataset)
rendering the observation dataset on a user interface for validation from a user associated with the facility; and (as noted above, an applied similarity score to labeled data set provides for selecting a portion of the result as desired and “We find that prompt 2 from section 3.2 does not work well with Vicuna, as it generates new information not contained in the original” indicates an observation dataset is observed)
receiving, via the user interface, feedback from the user on the observation dataset. (as noted above, an applied similarity score to labeled data set provides for selecting a portion of the result as desired and “We find that prompt 2 from section 3.2 does not work well with Vicuna, as it generates new information not contained in the original” indicates an observation dataset is observed and wherein “Therefore, we use prompt 1 and find following evaluation results for 100 sentences and their respective paraphrases” is a user feedback for using prompt1)
Claims 14 is rejected under the same rationale as above.
Claim 7. The method of claim 6, wherein construction of the proxy task comprises:
creating the proxy task using the training dataset and the validation dataset from the labeled dataset; vectorizing one or more textual representations in each first record of the unlabeled dataset and its corresponding second record in the semantic dataset using a corresponding textual embedding of the one or more textual embeddings; and defining the one or more evaluation metrics to measure similarity between respective vectors of each first record of the unlabeled dataset and its corresponding second record in the semantic dataset. (This claim is not implementable because claim 6 requires at least one of “training dataset, validation dataset, and observation dataset” be created and used. As noted above in claim 6, an applied similarity score to labeled data set provides for selecting a portion of the result as desired and as noted in claim 1, quality of generated paraphrase is evaluated Du, 3.2.2, “To evaluate the quality of our generated paraphrases we use a simpler method by first calculating the semantic similarity sim. This is done using the cosine similarity of the original sentence embeddings and paraphrase embeddings generated by the supervised RoBERTalarge SimCSE model. Then, we normalize the values so they lie between 0 and 1. To measure the differences in lexical and syntactical structure of the original sentence compared to the paraphrases we use the BLEU score [21]. It counts how many unigrams, bigrams, trigrams and four-grams occur in the hypothesis (original sentence), as well as in the reference (paraphrases). Thus, for our generated paraphrases we aim to reach a high semantic similarity score while keeping the BLEU score low”)
Claims 15 is rejected under the same rationale as above.
Claim 8. The method of claim 7, wherein executing the proxy task comprises:
comparing the respective vectors of each first record of the unlabeled dataset and its corresponding second record in the semantic dataset; measuring a similarity score using the one or more evaluation metrics based on the comparison, wherein the similarity score indicates a degree of similarity between a first record from the unlabeled dataset and a corresponding second record in the semantic dataset; classifying respective records in the training dataset and the validation dataset by corresponding machine learning models based on the similarity score; and measuring an accuracy score for each first record of the unlabeled dataset and its corresponding second record in the semantic dataset based on the classification. (This claim is not implementable because claim 6 requires at least one of “training dataset, validation dataset, and observation dataset” be created and used. See explanation in claim 7)
Claims 16 is rejected under the same rationale as above.
Claim 9. The method of claim 6, further comprising:
determining a model threshold of the selected machine learning model, wherein the model threshold corresponds to a threshold with which the selected machine learning model binarizes one or more predictions; and refining, based on the feedback, the model threshold using the observation dataset and at least one first algorithm, wherein the at least one first algorithm corresponds to Bayesian update. (Du, a Bayesian update is used because based on the result of “prompt 2 from section 3.2 does not work well with Vicuna, as it generates new information not contained in the original” the next step is used as a refined step “Therefore, we use prompt 1 and find following evaluation results for 100 sentences and their respective paraphrases”. Sec. 4.2, “First, we showcase the results from training with paraphrases for 150k sentences. In the following, we compare the results from using paraphrases generated by ChatGPT with paraphrases generated by Vicuna. We use different pre-trained encoder models with varying hyperparameters and present the models that achieve the highest average score for each encoder. The scores achieved by the unsupervised SimCSE counterparts are also shown in table 4.1 for comparison. For all of our models in the table, we use a learning rate of 3e − 5 and batch size of 64. The number of epochs and dropout used during training is shown in table 4.2…Our models using RoBERTa as pre-trained model perform around 6 points better than the SimCSE counterparts…Due to the above findings, we decide to scale the experiment to 1 million sentences for ChatGPT. We extend our paraphrase dataset and generate 5 paraphrases for each of the remaining 850k sentences in the sentence dataset using ChatGPT”)
Claims 17 is rejected under the same rationale as above.
Response to Amendment and Arguments
Applicant’s arguments with respect to amended claims have been considered but are not persuasive for at least the following reason.
Applicant argues that the applied references do not teach the amended feature because “Du does not describe a semantic dataset that is distinct from the unlabeled dataset and that provides corresponding records that are separately vectorized in parallel with the unlabeled records. Rather, Du's LLM generated paraphrases are derived from the unlabeled dataset itself and are used as augmented data for contrastive learning. Du also does not describe multiple candidate textual embeddings as distinct semantic encoding techniques. Instead, Du trains a single sentence embedding model and uses it to output sentence embedding vectors for the original sentences and their paraphrases”.
In response: Du discloses the feature as shown in claim 1 above because based on Spec., ¶ 29, a semantic dataset “comprises records similar to and/or dissimilar to the unlabeled dataset”. Furthermore, the claim does not require “multiple candidate textual embeddings as distinct semantic encoding techniques”. However, Du explicitly discloses feeding inputs into “a pre-trained embedding model like BERT…or RoBERTa” for generating embeddings of sentences as in Secs. 2.1 and 2.3. Evidently, as suggested throughout Du’s disclosure, any available model can be used based on the need of an application.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mohsen Almani whose telephone number is (571)270-7722. The examiner can normally be reached on M-F, 9 AM-5 PM, ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J. Lo can be reached on 571-272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHSEN ALMANI/Primary Examiner, Art Unit 2159
Prosecution Insights