Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
Acknowledgment is made of the Information Disclosure Statement dated 04/10/2025 and 07/01/2025.
The information disclosure statement filed 04/10/2025 in regard to “The office action of the corresponding Taiwanese application No. TW113110902 issued on March 14, 2025” and 07/01/2025 in regard to “The office action of the corresponding Japanese application No.JP2024-047269 issued on June 03, 2025” fail to comply with the provisions of 37 CFR 1.97, 1.98 and MPEP § 609 because they have not been fully translated into English. It has been placed in the application file, but the information referred to therein has not been considered as to the merits. Applicant is advised that the date of any re-submission of any item of information contained in this information disclosure statement or the submission of any missing element(s) will be the date of submission for purposes of determining compliance with the requirements based on the time of filing the statement, including all certification requirements for statements under 37 CFR 1.97(e). See MPEP § 609.05(a).
All the remaining cited references have been considered.
Drawings
The drawings have been received on 03/22/2024. These drawings are accepted.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are:
“intent classifier” in claim 1.
“storage unit” in claim 15.
“processing unit” in claim 15.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 15-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because the claims recite “a processing unit”. Paragraph [0026] of the Specification does not provide a definitive hardware interpretation of the processing unit; therefore, the broadest reasonable interpretation of “a processing unit” is not statutory.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1,
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“producing augmented phrases corresponding to the initial phrases by sentence augmentation;”
“generating, [by the first intent classifier,] first predicted intent labels about the augmented phrases and first confidence levels of the first predicted intent labels;”
“classifying the augmented phrases into augmentation subsets according to comparisons between the first predicted intent labels and the initial intent labels and according to the first confidence levels; and”
“[training a second intent classifier according to a part of the augmentation subsets by curriculum learning, wherein the second intent classifier is configured to] distinguish an intent of an input phrase within a dialogue.”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., producing, generating, classifying, distinguishing). The above limitations in the context of this claim encompass, inter alia, producing augmented phrases, generating predicted intent labels, classifying augmented phrases, distinguishing an intent of an input phrase within a dialogue (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“training a first intent classifier with the initial dataset;”
“[generating,] by the first intent classifier, [first predicted intent labels about the augmented phrases and first confidence levels of the first predicted intent labels;]”
“[training a second intent classifier according to a part of the augmentation subsets by curriculum learning, wherein the second intent classifier is configured to] distinguish an intent of an input phrase within a dialogue.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
The limitations:
“obtaining an initial dataset comprising initial phrases and initial intent labels about the initial phrases;”
As drafted, amount to insignificant extra-solution activities, which do not integrate a judicial exception into a practical application. For example, the additional elements of "obtaining an initial dataset" amount to mere data gathering and data storage, respectively, which are insignificant extra-solution activities that do not integrate a judicial exception into a practical application. See MPEP 2106.05(g).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“training a first intent classifier with the initial dataset;”
“[generating,] by the first intent classifier, [first predicted intent labels about the augmented phrases and first confidence levels of the first predicted intent labels;]”
“[training a second intent classifier according to a part of the augmentation subsets by curriculum learning, wherein the second intent classifier is configured to] distinguish an intent of an input phrase within a dialogue.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are insignificant extra-solution activities or mere instructions to apply an exception. (i.e., the additional element describes a unit for applying the abstract ideas). Insignificant extra-solution activities and mere instructions to apply an exception cannot provide an inventive concept. Moreover, receiving, communicating, and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP 2106.05(d)(II) ("The courts have recognized the following computer functions as well-understood, routine, and conventional functions ... i. Receiving or transmitting data over a network ... iv. Storing and retrieving information in memory") (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)).
The claim is not patent eligible.
Regarding Claim 2,
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“classifying the augmented phrases having the first predicted intent labels matching with the initial intent labels and having the first confidence levels over a first confidence threshold into a first augmentation subset;”
“classifying the augmented phrases having the first predicted intent labels matching with the initial intent labels and having the first confidence levels below the first confidence threshold into a second augmentation subset;”
“classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the first confidence levels over a second confidence threshold into a third augmentation subset; and”
“classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the first confidence levels below the second confidence threshold into a fourth augmentation subset.”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., classifying). The above limitations in the context of this claim encompass, inter alia, classifying augmented phrases according to a threshold (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: Please see the corresponding analysis of Claim 1.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
Regarding Claim 3,
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the corresponding analysis of Claim 1.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“training the second intent classifier according to the initial dataset and the first augmentation subset during a first round of curriculum learning; and”
“training the second intent classifier according to the initial dataset, the first augmentation subset and the second augmentation subset during a second round of curriculum learning.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“training the second intent classifier according to the initial dataset and the first augmentation subset during a first round of curriculum learning; and”
“training the second intent classifier according to the initial dataset, the first augmentation subset and the second augmentation subset during a second round of curriculum learning.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
The claim is not patent eligible.
Regarding Claim 4,
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the corresponding analysis of Claim 1.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“wherein the third augmentation subset and the fourth augmentation subset are not utilized to train the second intent classifier.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“wherein the third augmentation subset and the fourth augmentation subset are not utilized to train the second intent classifier.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
The claim is not patent eligible.
Regarding Claim 5,
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“generating, [by the second intent classifier,] second predicted intent labels about the augmented phrases and second confidence levels of the second predicted intent labels;”
“classifying the augmented phrases into updated augmentation subsets with reference to the second predicted intent labels and the second confidence levels; and”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., generating, classifying). The above limitations in the context of this claim encompass, inter alia, generating predicted intent labels, classifying augmented phrases (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“[generating,] by the second intent classifier, [second predicted intent labels about the augmented phrases and second confidence levels of the second predicted intent labels;]”
“training a third intent classifier according to the updated augmentation subsets by curriculum learning.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“[generating,] by the second intent classifier, [second predicted intent labels about the augmented phrases and second confidence levels of the second predicted intent labels;]”
“training a third intent classifier according to the updated augmentation subsets by curriculum learning.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
The claim is not patent eligible.
Regarding Claim 6,
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“classifying the augmented phrases having the second predicted intent labels matching with the initial intent labels and having the second confidence levels over a first confidence threshold into a first updated augmentation subset;”
“classifying the augmented phrases having the second predicted intent labels matching with the initial intent labels and having the second confidence levels below the first confidence threshold into a second updated augmentation subset;”
“classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the second confidence levels over a second confidence threshold into a third updated augmentation subset; and”
“classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the second confidence levels below the second confidence threshold into a fourth updated augmentation subset.”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., classifying). The above limitations in the context of this claim encompass, inter alia, classifying augmented phrases according to a threshold (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: Please see the corresponding analysis of Claim 1.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
Regarding Claim 7,
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the corresponding analysis of Claim 1.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“training the third intent classifier according to the initial dataset and the first updated augmentation subset during a first round of curriculum learning; and
“training the third intent classifier according to the initial dataset, the first updated augmentation subset and the second updated augmentation subset during a second round of curriculum learning; and”
“training the third intent classifier according to the initial dataset, the first updated augmentation subset, the second updated augmentation subset and the third updated augmentation subset during a third round of curriculum learning.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“training the third intent classifier according to the initial dataset and the first updated augmentation subset during a first round of curriculum learning; and
“training the third intent classifier according to the initial dataset, the first updated augmentation subset and the second updated augmentation subset during a second round of curriculum learning; and”
“training the third intent classifier according to the initial dataset, the first updated augmentation subset, the second updated augmentation subset and the third updated augmentation subset during a third round of curriculum learning.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
The claim is not patent eligible.
Regarding Claim 8,
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the corresponding analysis of Claim 1.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“wherein the fourth updated augmentation subset is not utilized to train the third intent classifier.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“wherein the fourth updated augmentation subset is not utilized to train the third intent classifier.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
The claim is not patent eligible.
Regarding Claim 9,
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the corresponding analysis of Claim 1.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“wherein the second predicted intent labels generated by the second intent classifier about the augmented phrases are utilized as ground truths in training the third intent classifier.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“wherein the second predicted intent labels generated by the second intent classifier about the augmented phrases are utilized as ground truths in training the third intent classifier.”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a classifier (e.g., by using these elements as tools).
The claim is not patent eligible.
Regarding Claim 10,
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“rewriting the initial phrases [by a large language model (LLM)] to produce the augmented phrases.”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., rewriting). The above limitations in the context of this claim encompass, inter alia, rewriting the initial phrases (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“[rewriting the initial phrases] by a large language model (LLM) [to produce the augmented phrases.]”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a machine-learning based model (e.g., by using these elements as tools).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“[rewriting the initial phrases] by a large language model (LLM) [to produce the augmented phrases.]”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a machine-learning based model (e.g., by using these elements as tools).
The claim is not patent eligible.
Regarding Claim 11,
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“translating the initial phrases in a first language [by a translation model] into intermediate phrases in a second language different from the first language; and”
“translating the intermediate phrases in the second language [by the translation model] into the augmented phrases in the first language.”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., translating). The above limitations in the context of this claim encompass, inter alia, translating the initial phrases into a first language, translating the intermediate phrases into the second language (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The limitations:
“[translating the initial phrases in a first language] by a translation model [into intermediate phrases in a second language different from the first language; and]”
“[translating the intermediate phrases in the second language] by the translation model [into the augmented phrases in the first language.]”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a machine-learning based model (e.g., by using these elements as tools).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The limitations:
“[translating the initial phrases in a first language] by a translation model [into intermediate phrases in a second language different from the first language; and]”
“[translating the intermediate phrases in the second language] by the translation model [into the augmented phrases in the first language.]”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). Specifically, they amount to mere instructions to apply the exception using a machine-learning based model (e.g., by using these elements as tools).
The claim is not patent eligible.
Regarding Claim 12,
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 12 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“replacing wordings in the initial phrases with synonyms related to the wordings for producing the augmented phrases.”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., replacing). The above limitations in the context of this claim encompass, inter alia, replacing wordings in the initial phrases with synonyms (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: Please see the corresponding analysis of Claim 1.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
Regarding Claim 13,
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“inserting random noises to the initial phrases for producing the augmented phrases.”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., inserting). The above limitations in the context of this claim encompass, inter alia, inserting random noises to the initial phrases (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: Please see the corresponding analysis of Claim 1.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
Regarding Claim 14,
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 14 is directed to a method, i.e., a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“generating a response according to the intent of the input phrase.”
As drafted, under their broadest reasonable interpretation, cover concepts performed in human mind (including an observation, evaluation, judgement, or opinion, e.g., generating). The above limitations in the context of this claim encompass, inter alia, generating a response (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong Two Analysis: Please see the corresponding analysis of Claim 1.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
Regarding Claim 15,
Claim 15 recites a system for performing steps similar of claim 1 and is rejected with the same rationale, mutatis mutandis, in view of the following additional elements, considered individually and as an ordered combination with the additional elements identified above, failing to integrate the abstract idea into a practical application or amount to significantly more than the abstract idea:
“a storage unit, configured to store computer-executable instructions; and”
“a processing unit, coupled with the storage unit, the processing unit is configured to execute the computer-executable instructions to:”
This is a recitation of generic computing components to be used in performing the abstract idea, which does not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea. See MPEP 2106.05(f).
Regarding Claim 16,
Claim 16 recites a system for performing steps substantially similar to those of claim 2 and is rejected with the same rationale, mutatis mutandis.
Regarding Claim 17,
Claim 17 recites a system for performing steps substantially similar to those of claims 3 and 4 and is rejected with the same rationale, mutatis mutandis.
Regarding Claim 18,
Claim 18 recites a system for performing steps substantially similar to those of claim 5 and is rejected with the same rationale, mutatis mutandis.
Regarding Claim 19,
Claim 19 recites a system for performing steps substantially similar to those of claim 6 and is rejected with the same rationale, mutatis mutandis.
Regarding Claim 20,
Claim 20 recites a system for performing steps substantially similar to those of claims 7 and 8 and is rejected with the same rationale, mutatis mutandis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Okur et al. (Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System); hereinafter Okur in view of Wei et al. (Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning); hereinafter Wei
Claim 1 is rejected over Okur and Wei.
Regarding claim 1, Okur teaches a language processing method, comprising: (Okur [page 1, Abstract]: “Our goal is to build Artificial Intelligence (AI) systems that can assist children in their learning process. Within such complex frameworks, Spoken Dialogue Systems (SDS) are crucial building blocks to handle efficient task-oriented communication with children in game-based learning settings. We are working towards a multimodal dialogue system for younger kids learning basic math concepts. Our focus is on improving the Natural Language Understanding (NLU) module of the task-oriented SDS pipeline with limited datasets.”)
obtaining an initial dataset comprising initial phrases and initial intent labels about the initial phrases; (Okur [page 5, 6.1. Dataset]: “We conduct our experiments on the KidSpace NLU datasets having utterances from multimodal math learning experiences (i.e., Planting and Watering activities) designed for 5-to-8 years-old kids … The NLU datasets have a limited number of utterances, which are manually annotated for intent types that we de fined for each use-case or learning game/activity (see section 3).”
training a first intent classifier with the initial dataset; (Okur [page 6, 6.3. Data Augmentation]: “For this MITL approach, we first trained the NLU model (DIET+ConveRT) on the original/seed dataset, then classified the paraphrased samples using this initial NLU model to obtain successful predictions.”)
producing augmented phrases corresponding to the initial phrases by sentence augmentation; (Okur [page 5, 5.2. Paraphrase Generation]: “we developed a data augmentation module via training a sequence-to-sequence paraphrasing model to generate paraphrased samples from the original seed utterances to augment the NLU training data.”)
generating, by the first intent classifier, first predicted intent labels about the augmented phrases and first confidence levels of the first predicted intent labels; (Okur [page 6, 6.3. Data Augmentation]: “success conf90: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful pre dictions) with a confidence score of 0.9 or higher. Another MITL approach using the same initial NLU model as in success. The confidence check ensures better noise filtering, and the threshold of 0.9 is chosen empirically after checking the confi dence histograms on paraphrased samples.”)
classifying the augmented phrases into augmentation subsets according to comparisons between the first predicted intent labels and the initial intent labels and (Okur [page 6, 6.3. Data Augmentation]: “success: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful predictions) … We assume the para phrased samples should belong to the same class as seed samples, and the idea is to filter out noisy synthetic samples that belong to other classes.”)
according to the first confidence levels; and (Okur [page 6, 6.3. Data Augmentation]: “success conf90: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful pre dictions) with a confidence score of 0.9 or higher. Another MITL approach using the same initial NLU model as in success. The confidence check ensures better noise filtering, and the threshold of 0.9 is chosen empirically after checking the confidence histograms on paraphrased samples.”)
training a second intent classifier according to a part of the augmentation subsets [by curriculum learning,] (Okur [page 7]: “Augmenting only the paraphrased samples that are classified (with high confidence) into the same label as their seed samples (i.e., success conf90) is the best performing approach, boosting the performance by nearly 4% (compared to training on original samples).”;)
wherein the second intent classifier is configured to distinguish an intent of an input phrase within a dialogue. (Okur [page 6, 6.2. NLU Results]: “we have updated the NLU component in our multimodal SDS pipeline (see Figure 2) by replacing the TF+BERT model (i.e., previously best-performing Baseline + BERT + Transformer model in Sahay et al. (2019))with this promising DIET+ConveRT model.”; and [page 3, 4.1. Architecture]: “We develop NLU models performing Intent Recognition and Entity Extraction to interpret user utterances. Then we pass these user intents and entities together with multimodal inputs, such as user actions and objects, into the DM component.”)
Claim 15 is rejected over Okur and Wei.
Regarding claim 15, Okur teaches a language processing system, comprising: a storage unit, configured to store computer-executable instructions; and a processing unit, coupled with the storage unit, the processing unit is configured to execute the computer-executable instructions to: (Okur [page 1, Abstract]: “Our goal is to build Artificial Intelligence (AI) systems that can assist children in their learning process. Within such complex frameworks, Spoken Dialogue Systems (SDS) are crucial building blocks to handle efficient task-oriented communication with children in game-based learning settings. We are working towards a multimodal dialogue system for younger kids learning basic math concepts. Our focus is on improving the Natural Language Understanding (NLU) module of the task-oriented SDS pipeline with limited datasets.”)
The remainder of claim 15 is claim 1 in the form of a system and is rejected for the same reasons as claim 1 stated above.
Claims 2, 3, 4, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Okur and Wei in further view of Northcutt et al. (Confident Learning: Estimating Uncertainty in Dataset Labels); hereinafter Northcutt
Claim 2 is rejected over Okur, Wei and Northcutt with the incorporation of claim 1.
Regarding claim 2, Okur teaches classifying the augmented phrases having the first predicted intent labels matching with the initial intent labels and having the first confidence levels over a first confidence threshold into a first augmentation subset; (Okur [page 6, 6.3. Data Augmentation]: “success conf90: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful pre dictions) with a confidence score of 0.9 or higher.”)
classifying the augmented phrases having the first predicted intent labels matching with the initial intent labels and having the first confidence levels below the first confidence threshold into a second augmentation subset; (Okur [page 6, 6.3. Data Augmentation]: “success: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful predictions).”; and “success conf90: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful predictions) with a confidence score of 0.9 or higher.”)
classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the first confidence levels over a second confidence threshold into a third augmentation subset; and (Okur [page 6, 6.3. Data Augmentation]: “all conf90: Augment only the paraphrased samples that are classified into any intent type (regardless of their seed samples’ intent class) with a confidence score of 0.9 or higher. Another MITL approach using the same initial NLU model as in success. We removed the assumption that paraphrased samples should be of the same class as their seed samples and still augment them into the predicted class samples if confidence is high.”; and “success conf90: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful predictions) with a confidence score of 0.9 or higher. Another MITL approach using the same initial”)
Okur does not appear to explicitly teach classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the first confidence levels below the second confidence threshold into a fourth augmentation subset.
However, Northcutt teaches classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the first confidence levels below the second confidence threshold into a fourth augmentation subset. (Northcutt [page 1, Abstract]: “Confident learning (CL) is an alternative approach which focuses instead on label quality by characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence. Whereas numerous studies have developed these principles independently, here, we combine them, building on the assumption of a class-conditional noise process to directly estimate the joint distribution between noisy (given) labels and uncorrupted (unknown) labels.”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the confident learning of Northcutt to improve validation accuracy (Northcutt, page 17). Okur and Northcutt are analogous art because they both concern labeling and confidence.
Claim 3 is rejected over Okur, Wei and Northcutt with the incorporation of claim 1.
Regarding claim 3, Okur does not appear to explicitly teach training the second intent classifier according to the initial dataset and the first augmentation subset during a first round of curriculum learning; and
training the second intent classifier according to the initial dataset, the first augmentation subset and the second augmentation subset during a second round of curriculum learning.
However, Wei teaches training the second intent classifier according to the initial dataset and the first augmentation subset during a first round of curriculum learning; and (Wei [page 5494, Our Approach]: “(1) Two-stage curriculum data augmentation calls for one stage of training with only original data, fol lowed by one stage of training with augmented data of fixed temperature.”; and [page 5495]: “For the two-stage curriculum, we started by training on original data only, and when validation loss converges, we introduce augmented data of fixed temperature at an augmented to original data ratio of 4:1.”)
training the second intent classifier according to the initial dataset, the first augmentation subset and the second augmentation subset during a second round of curriculum learning. (Wei [page 5500]: “Curriculum two-stage training employs a first stage of only original data and a second stage of augmented data, using the same number of updates as single-stage training in total. We determined the number of updates in the first stage based on when training loss plateaued in the training plot for training with no augmentation.”; and [5494]: “we can create an artificial curriculum in the data by leveraging controlled application of data augmentation, starting by training on only original data and then adding augmented data with a higher levels of noising as training progresses.”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the curriculum learning of Wei to improve training performance (Wei, Abstract). Okur and Wei are analogous art because they both concern data augmentation.
Claim 4 is rejected over Okur, Wei and Northcutt with the incorporation of claim 1.
Regarding claim 4, Okur teaches wherein the third augmentation subset and the fourth augmentation subset are not utilized to train the second intent classifier. (Okur [page 6, 6.3. Data Augmentation]: “success: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful predictions). For this MITL approach, we first trained the NLU model (DIET+ConveRT) on the original/seed dataset, then classified the paraphrased samples using this initial NLU model to obtain successful predictions. We assume the para phrased samples should belong to the same class as seed samples, and the idea is to filter out noisy synthetic samples that belong to other classes.”)
Dependent claim 16 is claim 2 in the form of a system and is rejected for the same reasons as claim 2 stated above. For the rejection of the limitations specifically pertaining to the system of claim 15, see the rejection of claim 15 above.
Dependent claim 17 is claims 3 and 4 in the form of a system and is rejected for the same reasons as claims 3 and 4 stated above. For the rejection of the limitations specifically pertaining to the system of claim 15, see the rejection of claim 15 above.
Claims 5 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Okur and Wei in view of Mi et al. (Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems); hereinafter Mi
Claim 5 is rejected over Okur, Wei and Mi with the incorporation of claim 1.
Regarding claim 5, Okur does not appear to explicitly teach generating, by the second intent classifier, second predicted intent labels about the augmented phrases and second confidence levels of the second predicted intent labels;
classifying the augmented phrases into updated augmentation subsets with reference to the second predicted intent labels and the second confidence levels; and
training a third intent classifier according to the updated augmentation subsets [by curriculum learning.]
However, Mi teaches generating, by the second intent classifier, second predicted intent labels about the augmented phrases and second confidence levels of the second predicted intent labels; (Mi [page 1, Abstract]: “It first trains a Teacher on the labeled samples. The Teacher then iteratively generates pseudo labels for the most confident subset of unlabeled samples to train a better Student.”; and “In the beginning of an ST iteration, the Teacher first makes predictions on U. For every data input x ∈ U, the Teacher predicts the label of x as
y
^
x
=
F
T
(
x
)
. We set the predicted score of the prediction
y
^
x
as the confidence score sx for this prediction.)
classifying the augmented phrases into updated augmentation subsets with reference to the second predicted intent labels and the second confidence levels; and (Mi [page 4]: “In each iteration, the Selector chooses top-k instances from U with the highest confidence scores, and assigns the corresponding predictions
y
^
x
as labels to them.”)
training a third intent classifier according to the updated augmentation subsets [by curriculum learning.] (Mi [page 4, Iterative Student training]: “At the end of each iteration, the Teacher model is overridden by the current Student to be used in the next iteration. We reinitialize the Student in every iteration to avoid over-fitting the initial and earlier data in Lin multiple training iterations.”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the iterative self-training approach of Mi to improve pre-trained models when only a small number of labeled data are available (Mi, page 1, Abstract). Okur and Mi are analogous art because they both concern data augmentation and intent classification.
Okur does not appear to explicitly teach by curriculum learning.
However, Wei teaches by curriculum learning. (Wei [page 1, Abstract]: “we present a simple training strategy called curriculum data augmentation, which leverages curriculum learning by first training on only original examples and then introducing augmented data as training progresses.”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the curriculum learning of Wei to improve training performance (Wei, Abstract). Okur and Wei are analogous art because they both concern data augmentation.
Dependent claim 18 is claim 5 in the form of a system and is rejected for the same reasons as claim 5 stated above. For the rejection of the limitations specifically pertaining to the system of claim 15, see the rejection of claim 15 above.
Claims 6, 7, 8, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Okur, Wei and Mi in further view of Northcutt et al. (Confident Learning: Estimating Uncertainty in Dataset Labels); hereinafter Northcutt
Claim 6 is rejected over Okur, Wei, Mi and Northcutt with the incorporation of claim 1.
Regarding claim 6, Okur teaches classifying the augmented phrases having the second predicted intent labels matching with the initial intent labels and having the second confidence levels over a first confidence threshold into a first updated augmentation subset; (Okur [page 6, 6.3. Data Augmentation]: “success conf90: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful pre dictions) with a confidence score of 0.9 or higher.”)
classifying the augmented phrases having the second predicted intent labels matching with the initial intent labels and having the second confidence levels below the first confidence threshold into a second updated augmentation subset; (Okur [page 6, 6.3. Data Augmentation]: “success: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful predictions).”; and “success conf90: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful predictions) with a confidence score of 0.9 or higher.”)
classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the second confidence levels over a second confidence threshold into a third updated augmentation subset; and (Okur [page 6, 6.3. Data Augmentation]: “all conf90: Augment only the paraphrased samples that are classified into any intent type (regardless of their seed samples’ intent class) with a confidence score of 0.9 or higher. Another MITL approach using the same initial NLU model as in success. We removed the assumption that paraphrased samples should be of the same class as their seed samples and still augment them into the predicted class samples if confidence is high.”; and “success conf90: Augment only the paraphrased samples that are classified correctly into the same intent class as their seed samples (successful predictions) with a confidence score of 0.9 or higher. Another MITL approach using the same initial”)
Okur does not appear to explicitly teach classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the second confidence levels below the second confidence threshold into a fourth updated augmentation subset.
However, Northcutt teaches classifying the augmented phrases having the first predicted intent labels mismatching with the initial intent labels and having the second confidence levels below the second confidence threshold into a fourth updated augmentation subset. (Northcutt [page 1, Abstract]: “Confident learning (CL) is an alternative approach which focuses instead on label quality by characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence. Whereas numerous studies have developed these principles independently, here, we combine them, building on the assumption of a class-conditional noise process to directly estimate the joint distribution between noisy (given) labels and uncorrupted (unknown) labels.”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the confident learning of Northcutt to improve validation accuracy (Northcutt, page 17). Okur and Northcutt are analogous art because they both concern labeling and confidence.
Claim 7 is rejected over Okur, Wei, Mi and Northcutt with the incorporation of claim 1.
Regarding claim 7, Okur does not appear to explicitly teach training the third intent classifier according to the initial dataset and the first updated augmentation subset during a first round of [curriculum] learning; and
training the third intent classifier according to the initial dataset, the first updated augmentation subset and the second updated augmentation subset during a second round of [curriculum] learning; and
training the third intent classifier according to the initial dataset, the first updated augmentation subset, the second updated augmentation subset and the third updated augmentation subset during a third round of [curriculum] learning.
However, Mi teaches teach training the third intent classifier according to the initial dataset and the first updated augmentation subset during a first round of [curriculum] learning; and (Mi [page 4]: “In each iteration, the Selector chooses top-k instances from U with the highest confidence scores, and assigns the corresponding predictions
y
^
x
as labels to them.”)
training the third intent classifier according to the initial dataset, the first updated augmentation subset and the second updated augmentation subset during a second round of [curriculum] learning; and (Mi [page 4]: “In each iteration, the Selector chooses top-k instances from U with the highest confidence scores, and assigns the corresponding predictions
y
^
x
as labels to them.”)
training the third intent classifier according to the initial dataset, the first updated augmentation subset, the second updated augmentation subset and the third updated augmentation subset during a third round of [curriculum] learning. (Mi [page 4]: “In each iteration, the Selector chooses top-k instances from U with the highest confidence scores, and assigns the corresponding predictions
y
^
x
as labels to them.”)
Okur does not appear to explicitly teach curriculum learning.
However, Wei teaches curriculum learning. (Wei [page 1, Abstract]: “we present a simple training strategy called curriculum data augmentation, which leverages curriculum learning by first training on only original examples and then introducing augmented data as training progresses.”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the curriculum learning of Wei to improve training performance (Wei, Abstract). Okur and Wei are analogous art because they both concern data augmentation.
Claim 8 is rejected over Okur, Wei, Mi and Northcutt with the incorporation of claim 1.
Regarding claim 8, Okur does not appear to explicitly teach wherein the fourth updated augmentation subset is not utilized to train the third intent classifier.
However, Mi teaches wherein the fourth updated augmentation subset is not utilized to train the third intent classifier. (Mi [page 4]: “In each iteration, the Selector chooses top-k instances from U with the highest confidence scores, and assigns the corresponding predictions
y
^
x
as labels to them.”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the iterative self-training approach of Mi to improve pre-trained models when only a small number of labeled data are available (Mi, page 1, Abstract). Okur and Mi are analogous art because they both concern data augmentation and intent classification.
Dependent claim 19 is claims 3 and 4 in the form of a system and is rejected for the same reasons as claims 3 and 4 stated above. For the rejection of the limitations specifically pertaining to the system of claim 15, see the rejection of claim 15 above.
Dependent claim 20 is claims 7 and 8 in the form of a system and is rejected for the same reasons as claims 7 and 8 stated above. For the rejection of the limitations specifically pertaining to the system of claim 15, see the rejection of claim 15 above.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Okur, Wei, Mi and Northcutt in view of Tanaka et al. (Joint Optimization Framework for Learning with Noisy Labels); hereinafter Tanaka
Claim 9 is rejected over Okur, Wei, Mi Northcutt and Tanaka with the incorporation of claim 1.
Regarding claim 9, Okur does not appear to explicitly teach wherein the second predicted intent labels generated by the second intent classifier about the augmented phrases are utilized as ground truths in training the third intent classifier.
However, Tanaka teaches wherein the second predicted intent labels generated by the second intent classifier about the augmented phrases are utilized as ground truths in training the third intent classifier. (Tanaka [page 1, Abstract]: “we propose a joint optimization framework of learning DNN parameters and estimating true labels.”; and [page 2, 2.3. Self-training and pseudo-labeling]: “we completely replace all labels by pseudo-labels and use the same for training.”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the optimization framework of Tanaka to improve performance of the classifier (Tanaka, page 1). Okur and Tanaka are analogous art because they both concern label noise.
Claims 10, 11, 12, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Okur, Wei and Mi in further view of Chen et al. (Data Augmentation for Intent Classification); hereinafter Chen
Claim 10 is rejected over Okur, Wei, Mi and Chen with the incorporation of claim 1.
Regarding claim 10, Okur does not appear explicitly teach rewriting the initial phrases by a large language model (LLM) to produce the augmented phrases.
However, Chen teaches rewriting the initial phrases by a large language model (LLM) to produce the augmented phrases. (Chen [page 2, 2.2 Latent Perturbation]: “We also pretrain a model to perform Paraphrasing using the PAWS, QQP and MRPC corpora (Zhang et al., 2019; Iyer et al., 2017; Dolan and Brockett, 2005). Specifically, we use a BART model which takes the original utterance as the input sequence and produces a paraphrased utterance as the output sequence (Lewis et al., 2020).”; and [page 2, 2.4 Auxilliary Dataset]: “we fine-tune a Language Model (LM) to decode new text given an intent. Concretely, a dataset containing labeled utterances are fed into GPT2 (Brown et al., 2020), where each example starts with the intent tag and ends with the utterance text,”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the intent classification task of Chen to generate quality data and improve model performance (Chen, page 2, 2 Methods and Experiments). Okur and Chen are analogous art because they both concern intent classification and data augmentation.
Claim 11 is rejected over Okur, Wei, Mi and Chen with the incorporation of claim 1.
Regarding claim 11, Okur does not appear to explicitly teach translating the initial phrases in a first language by a translation model into intermediate phrases in a second language different from the first language; and
translating the intermediate phrases in the second language by the translation model into the augmented phrases in the first language.
However, Chen teaches translating the initial phrases in a first language by a translation model into intermediate phrases in a second language different from the first language; and (Chen [page 2, 2.2 Latent Perturbation]: “New data can also be generated by mapping the raw utterance into a latent embedding space and back into an alternate surface form. (2a) Back-translation encodes the original utterance in a separate language,”)
translating the intermediate phrases in the second language by the translation model into the augmented phrases in the first language. (Chen [page 2, 2.2 Latent Perturbation]: “and then translates back into English to produce a new example”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the intent classification task of Chen to generate quality data and improve model performance (Chen, page 2, 2 Methods and Experiments). Okur and Chen are analogous art because they both concern intent classification and data augmentation.
Claim 12 is rejected over Okur, Wei, Mi and Chen with the incorporation of claim 1.
Regarding claim 12, Okur does not appear explicitly teach replacing wordings in the initial phrases with synonyms related to the wordings for producing the augmented phrases.
However, Chen teaches replacing wordings in the initial phrases with synonyms related to the wordings for producing the augmented phrases. (Chen [page 2, 2.1 Surface Form Alteration]: “(1b) Synonym Replacement first determines the part-of-speech (POS) for each utterance token, and then chooses tokens for replacement only when the POS tag is either a noun, verb, or adjective. Finding synonyms in this manner avoids stop words and produces more grammatical augmentations. We use Wordnet as our source (Miller, 1995)”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the intent classification task of Chen to generate quality data and improve model performance (Chen, page 2, 2 Methods and Experiments). Okur and Chen are analogous art because they both concern intent classification and data augmentation.
Claim 13 is rejected over Okur, Wei, Mi and Chen with the incorporation of claim 1.
Regarding claim 13, Okur does not appear to explicitly teach inserting random noises to the initial phrases for producing the augmented phrases.
However, Chen teaches inserting random noises to the initial phrases for producing the augmented phrases. (Chen [page 2, 2.1 Surface Form Alteration]: “One way of augmenting natural language data is to alter the surface form of the text. (1a) Easy Data Augmentation (EDA) produces new examples by randomly deleting, inserting or swapping the order of tokens within the seed utterance (Wei and Zou, 2019).”)
It would have been obvious before the effective filing date to combine the data augmentation of Okur with the intent classification task of Chen to generate quality data and improve model performance (Chen, page 2, 2 Methods and Experiments). Okur and Chen are analogous art because they both concern intent classification and data augmentation.
Claim 14 is rejected over Okur, Wei and Mi and Chen with the incorporation of claim 1.
Regarding claim 14, Okur teaches generating a response according to the intent of the input phrase. (Okur [page 4]: “When the verbal response types are predicted, based on the output classes of DM, the NLG module retrieves actual bot responses,”)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID H TRAN whose telephone number is (703)756-1525. The examiner can normally be reached M-F 9:30 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H TRAN/Examiner, Art Unit 2147
/ERIC NILSSON/Primary Examiner, Art Unit 2151
Prosecution Insights