DETAILED ACTION
Introduction
This Final Office Action is in response to amendments and remarks filed on May 26, 2026, for the application with serial number 18/960,490.
Claims 1, 2, 6, 14, 15, and 19 are amended.
Claims 5 and 18 are canceled.
Claims 1-4, 6-17 and 19-22 are pending.
Response to Remarks/Amendments
35 USC §101 Rejections
The Applicant traverses the rejection of the claims as being directed to an ineligible abstract idea, contending that the present claims recite an improvement to a technology. See Remarks p. 7. The Examiner respectfully disagrees. Labeling issues is not a technology or technical field. The claims merely recite the use of a known machine learning process for its intended purpose. The use of the zero shot classification model amount to, at best, a technological environment that does not provide a practical application or significantly more than the recite abstract idea. The claimed process could be implemented mentally or on paper by a human being, but a generic computer hardware employing a machine learning model is recited for implementation. No apparent improvement to machine learning is recited in the claims. Contrary to the Applicant’s assertions, audit risk classification is not a technology. The claims merely recite the use of a zero shot classification model for its intended purpose – natural language processing according to a related or likely outcome.
The Applicant further points to lack of conventionality as evidence of subject matter eligibility. See Remarks pp. 9-10. In response, the Examiner points out that lack of conventionality is not evidence of subject matter eligibility. Additional elements outside the scope of the abstract idea have been considered, but they have been found to amount to generic computer hardware employing a machine learning algorithm.
The rejection for lack of subject matter eligibility is updated and maintained.
35 USC §103 Rejections
The Applicant traverses the rejection of the independent claims as being obvious over Wang ‘920 in view of Sewak; contending that Wang ‘920 is fundamentally different from the recited invention. See Remarks p. 12. The Examiner respectfully disagrees. Determining the likelihood that an expense will trigger an audit is the same as categorizing the risk of an audit. In the Remarks, the Applicant’ appears to conflate the use of an internal audit with an external audit. The term “audit,” itself, is both vague and broad. The present claims recite a process for analyzing information gleaned from an internal audit to assess risks. See Specification ¶[0002]. Wang ‘920 processes expense information to determine the risk of an external audit. (An internal audit, by its nature, carries no risk). Therefore, Wang ‘920 reads on the present claims. Contrary to the Applicant’s assertions, Wang ‘920 teaches expense descriptions, which meet the broadest reasonable interpretation of “issue description” in the present claims. The language is extremely broad.
The Applicant additionally contends that Wang ‘920 does not recite the use of “description: hypothesis pairs in a zero shot classification model. See Remarks pp. 12-13. In response, the Examiner submits that this argument ignores the combination of the Sewak reference with Wang ‘920. Wang ‘920 discloses expense descriptions associated with a label, and Sewak discloses the association of descriptive text with a label. The Examiner notes that what is being claimed here is intrinsic to human thought – the association of a label or name with a description. The claimed method is obvious, as evidenced by the combination of Wang ‘920 and Sewak. Contrary to the Applicant’s assertions, Sewak determines relevance by determining the relevance of documents. See at least Sewak ¶[0002]-[0003].
The Applicant additionally contends that the stated motivation to combine Sewak with Wang ‘920 is deficient. See Remarks pp. 13-14. The Examiner respectfully disagrees. Combining Sewak with Wang ‘920 would provide a manner of labeling text data, as taught by Sewak. A motivational statement does not require proof that a method in the primary reference be “abandoned,” as alleged by the Applicant. A proper motivational statement only need demonstrate a motivation to try the combination. The substitution of Sewak’s zero shot modeling technique in the model taught by Wang ‘920 would lead the skilled artisan to the claimed invention – with no unpredictable results.
The rejection of the dependent claims stands or falls with the rejection of the independent claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The Manual of Patent Examining Procedure (MPEP) provides detailed rules for determining subject matter eligibility for claims in §2106. Those rules provide a basis for the analysis and finding of ineligibility that follows.
Claims 1-4, 6-17 and 19-22 are rejected under 35 U.S.C. 101. The claimed invention is directed to non-statutory subject matter because the claimed invention recites a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Under Step 1 of the subject matter eligibility analysis, claims(s) 1-4, 6-17 and 19-22 are all directed to one of the four statutory categories of invention. However, under step 2A, prong one, the claims recite a judicial exception: labeling issues from an audit (as evidenced by the preamble of exemplary independent claim 1), an abstract idea. Certain methods of organizing human activity are ineligible abstract ideas, including managing personal behavior or relationships or interactions between people. See MPEP §2106.04(a). The limitations of exemplary claim 1 include: “receiving . . . an issue description . . . comprising a text description;” “combining . . . the text description with a plurality of hypotheses texts to generate a plurality of description: hypothesis pairs . . . associated with a sub-risk description;” “applying each of the description: hypothesis pairs . . . to a [classification model] outputting a relevance of the sub-risk;” “determining . . . a label score . . . based on the relevance;” “filtering . . . each of the label scores;” and “outputting the top n labels.” The steps are all steps for managing personal behavior related to the abstract idea of labeling issues from an audit that, when considered alone and in combination, are part of the abstract idea of labeling issues from an audit. The dependent claims further recite steps for managing personal behavior that are part of the abstract idea of labeling issues from an audit. These claim elements, when considered alone and in combination, are considered to be abstract ideas because they are directed to a method of organizing human activity which includes classifying audit risks based on descriptions of financial issues and accounting.
Under step 2A, prong two, of the subject matter eligibility analysis, a claim that recites a judicial exception must be evaluated to determine whether the claim provides a practical application of the judicial exception. Additional elements of the independent claims amount to generic computer hardware that does not provide a practical application (no hardware is recited in independent claim 1; a computer readable medium in independent claim 14; and a computing system in dependent claim 22). See MPEP §2106.04(d)[I]. The claims do not recite an improvement to another technology or technical field, nor do they recite an improvement to the functioning of the computer itself. See MPEP §2106.05(a). The claims do recite the use of a zero-shot classification model, but the abstract idea of labeling issues from an audit is generally linked to a computing environment with a zero-shot classification model for implementation. Therefore, the zero-shot classification model does not provide a practical application or significantly more than the recited abstract idea. See MPEP §2106.05(h). Because the claims only recite use of a generic computer, they do not apply the judicial exception with a particular machine. See MPEP §2106.05(b). Under step 2B of the subject matter eligibility analysis, the claims do not integrate the abstract idea into a judicial exception. Referring to the additional elements provided in the analysis in step one, above, the generic computer hardware does not provide significantly more than the recited abstract idea. See MPEP §2106.05(f).
For these reasons, the claims do not provide a practical application of the abstract idea, nor do they amount to significantly more than an abstract idea under step 2B of the subject matter eligibility analysis. Using a generic computer to implement an abstract idea does not provide an inventive concept. Therefore, the claims recite ineligible subject matter under 35 USC §101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 6, 14, and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to Wang et al. (hereinafter ‘WANG ‘920’) in view of US 20220414137 A1 to Sewak et al. (hereinafter ‘SEWAK’).
Claim 1 (Currently Amended)
WANG ‘920 discloses a method of automatically labelling issues from an internal audit (see abstract and ¶[0039]; real-time expense auditing to compute audit risk scores as a function of expense descriptions. The output include a label), the method comprising:
receiving, by a computing system (see ¶[0115] and Fig. 6; a computer system), an issue description from audit information of an internal audit, the issue description comprising a text description of an internal audit issue (see again abstract; compute audit risk scores as a function of expense descriptions);
combining, by the computing system, the text description with a plurality of hypotheses texts to generate a plurality of description: hypothesis pairs (see ¶[0040], [0091], and [0102]; labeled training data includes input/output pairs in which each input is labeled with a desired output. An example training dataset may include one or more labels. Receive a query including a description of a new hypothetical or actual expense), each of the plurality of hypotheses texts associated with a sub-risk description for a sub-risk in a risk taxonomy (see ¶[0093]; the system generates a set of feature vectors for labeled examples. Example features include categorical information about what type of expense was incurred. Include rules about the types of expenses that are permissible and conditions where expenses are reimbursable).
WANG does not specifically disclose, but SEWAK discloses, applying each of the description:hypothesis pairs to a zero-shot classification model, the zero-shot classification model outputting a relevance of the sub-risk associated with the hypothesis for each description: hypothesis pair; determining, by the computing system. a label score for each sub-risk based on the relevance (see abstract and ¶[0002]-[0004], [00039], & [0050]; a zero-shot generative mode is generally a mode of a generative NLP model capable of generating text without fine-tuning with a specific type of data. A generative NLP model generally receives an input text string and produces a generative result that is text, which is generated at the prompting of the input text string. Estimate a likelihood that the label applies to candidate text. Score the relevance of documents).
filtering, by the computing system, each of the label scores to identify a top n labels for the issue description, where n is a whole number greater than 1 (see ¶[0175]; rank positive example results based on cosine similarity. See also ¶[0067]; probability of a label class is above an acceptable threshold level).
WANG further discloses outputting the top n labels for the issue description to identify relevant sub-risks associated with the issue description (see ¶[0027] and [0039]-[0049]; compute an output that may be a label, classification, or categorization. Predict whether an expense is an audit risk and formulate a natural language response).
WANG discloses a real-time expense auditing and machine learning system that labels audit risks based on expense descriptions. SEWAK discloses automatic labeling of text data using a zero-shot generative model, where a ranked list of candidate results is created based on a threshold probability of accuracy. It would have been obvious for one of ordinary skill in the art at the time of invention to include the zero-shot generative model and ranking as taught by SEWAK in the system executing the method of WANG with the motivation to label audit risk based on expense descriptions.
Claim 6 (Currently Amended)
The combination of WANG ‘920 and SEWAK discloses the method as set forth in claim 1.
WANG ‘920 does not specifically disclose, but SEWAK discloses, wherein the filtering comprises: aggregating a plurality label scores for hypothesis associated with the same sub-risk; and filtering on the aggregated label scores (see ¶[0067]; probability of a label class is above an acceptable threshold level).
WANG discloses a real-time expense auditing and machine learning system that labels audit risks based on expense descriptions. SEWAK discloses automatic labeling of text data using a zero-shot generative model, where a ranked list of candidate results is created based on a threshold probability of accuracy. It would have been obvious for one of ordinary skill in the art at the time of invention to include the ranking as taught by SEWAK in the system executing the method of WANG with the motivation to label audit risk based on expense descriptions.
Claim 14 (Currently Amended)
WANG ‘920 discloses a non-transitory computer readable medium storing instructions, which when executed by a processor of a computing device configure the computing device to perform a method (see ¶[0119]-[0120] a storage medium with a program) comprising: receiving an issue description from audit information of an internal audit, the issue description comprising a text description of an internal audit issue (see again abstract; compute audit risk scores as a function of expense descriptions);
combining the text description with a plurality of hypotheses texts to generate a plurality of description: hypothesis pairs (see ¶[0040], [0091], and [0102]; labeled training data includes input/output pairs in which each input is labeled with a desired output. An example training dataset may include one or more labels. Receive a query including a description of a new hypothetical or actual expense), each of the plurality of hypotheses texts associated with a sub-risk description for a sub-risk in a risk taxonomy (see ¶[0093]; the system generates a set of feature vectors for labeled examples. Example features include categorical information about what type of expense was incurred. Include rules about the types of expenses that are permissible and conditions where expenses are reimbursable).
WANG does not specifically disclose, but SEWAK discloses, applying each of the description: hypothesis pairs to a zero-shot classification model, the zero-shot classification model outputting a relevance of the sub-risk associated with the hypothesis for each description: hypothesis pair; determining a label score for each sub-risk based on the relevance (see abstract and ¶[0002]-[0004], [00039], & [0050]; a zero-shot generative mode is generally a mode of a generative NLP model capable of generating text without fine-tuning with a specific type of data. A generative NLP model generally receives an input text string and produces a generative result that is text, which is generated at the prompting of the input text string. Estimate a likelihood that the label applies to candidate text. Score the relevance of documents).
filtering each of the label scores to identify a top n labels for the issue description, where n is a whole number greater than 1 (see ¶[0175]; rank positive example results based on cosine similarity. See also ¶[0067]; probability of a label class is above an acceptable threshold level).
WANG further discloses outputting the top n levels for the issue description to identify relevant sub-risks associated with the issue description (see ¶[0027] and [0039]-[0049]; compute an output that may be a label, classification, or categorization. Predict whether an expense is an audit risk and formulate a natural language response).
WANG discloses a real-time expense auditing and machine learning system that labels audit risks based on expense descriptions. SEWAK discloses automatic labeling of text data using a zero-shot generative model. It would have been obvious for one of ordinary skill in the art at the time of invention to include the zero-shot generative model as taught by SEWAK in the system executing the method of WANG with the motivation to label audit risk based on expense descriptions.
Claim 22 (Original)
WANG ‘920 discloses a computing system comprising: a processor for executing instructions; and a memory storing instructions (see ¶[0029] and Fig. 1; a computer network), which when executed by the processor configure the computing system to perform a method according to claim 1 (see claim 1 rejection, which additionally references SEWAK).
Claim(s) 2, 3, 15, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to WANG ‘920 et al. in view of US 20220414137 A1 to SEWAK et al. as applied to claim 1 above, and further in view of US 20230196105 A1 to Wang et al. (hereinafter WANG ‘105’).
Claim 2 (Currently Amended)
The combination of WANG ‘920 and SEWAK discloses the method as set forth in claim 1.
The combination of WANG ‘920 and SEWAK does not specifically disclose, but WANG ‘105 discloses, wherein the filtering comprises: applying a generative large-language model (LLM) to the issue description and the hypothesis texts to determine if the issue description is relevant to the hypothesis text (see ¶[0019] and [0034]; text classification into categories given a premise sequence and a hypothesis sequence based on similarity. A language model neural network may include large language models).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. WANG ‘105 discloses classification tasks using hypothesis sequences that are used to compute a similarity between input sequences to classify the input. It would have been obvious for one of ordinary skill in the art to use the large language model using hypothesis sequences as taught by WANG ‘105 in the system executing the method of WANG ‘920 with the motivation to label audit risks based on descriptions.
Claim 3 (Original)
The combination of WANG ‘920, SEWAK, and WANG ‘105 discloses the method as set forth in claim 2.
WANG ‘920 does not specifically disclose, but WANG ‘105 discloses, wherein only issue descriptions with a label score above a threshold are applied to the generative LLM (see ¶[0079]-[0080]; if both (i) the highest probability in the probability distribution exceeds the threshold probability for the training step and (ii) the highest probability is for a category that is different than the target category identified in the sampled auto-labeled training example, the system refrains from training the task neural network on the sampled auto-labeled training example at the training step (step 506).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. WANG ‘105 discloses classification tasks using hypothesis sequences that are used to compute a similarity between input sequences to classify the input, where only categories with a threshold probability for classification are used as training examples. It would have been obvious for one of ordinary skill in the art to use the thresholds as taught by WANG ‘105 in the system executing the method of WANG ‘920 with the motivation to label audit risks based on descriptions using a large language model.
Claim 15 (Currently Amended)
The combination of WANG ‘920 and SEWAK discloses the computer readable medium as set forth in claim 14.
The combination of WANG ‘920 and SEWAK does not specifically disclose, but WANG ‘105 discloses, wherein the filtering comprises: applying a generative large-language model (LLM) to the issue description and the hypothesis texts to determine if the issue description is relevant to the hypothesis text (see ¶[0019] and [0034]; text classification into categories given a premise sequence and a hypothesis sequence based on similarity. A language model neural network may include large language models).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. WANG ‘105 discloses classification tasks using hypothesis sequences that are used to compute a similarity between input sequences to classify the input. It would have been obvious for one of ordinary skill in the art to use the large language model using hypothesis sequences as taught by WANG ‘105 in the system executing the method of WANG ‘920 with the motivation to label audit risks based on descriptions.
Claim 16 (Original)
The combination of WANG ‘920, SEWAK, and WANG ‘105 discloses the computer readable medium as set forth in claim 15.
WANG ‘920 does not specifically disclose, but WANG ‘105 discloses, wherein only issue descriptions with a label score above a threshold are applied to the generative LLM (see ¶[0079]-[0080]; if both (i) the highest probability in the probability distribution exceeds the threshold probability for the training step and (ii) the highest probability is for a category that is different than the target category identified in the sampled auto-labeled training example, the system refrains from training the task neural network on the sampled auto-labeled training example at the training step (step 506).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. WANG ‘105 discloses classification tasks using hypothesis sequences that are used to compute a similarity between input sequences to classify the input, where only categories with a threshold probability for classification are used as training examples. It would have been obvious for one of ordinary skill in the art to use the thresholds as taught by WANG ‘105 in the system executing the method of WANG ‘920 with the motivation to label audit risks based on descriptions using a large language model.
Claim(s) 7 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to WANG ‘920 et al. in view of US 20220414137 A1 to SEWAK et al. as applied to claims 1 and 6 above, and further in view of US 20230113956 A1 to Lal et al. (hereinafter ‘LAL’).
Claim 7 (Original)
The combination of WANG ‘920 and SEWAK discloses the method as set forth in claim 6.
The combination of WANG ‘920 and SEWAK does not specifically disclose, but LAL discloses, wherein the filtering further comprises: for all hypothesis associated with sub-risks grouped by a common risk, filtering to a top m sub-risks for the risk grouping, where m is a whole number less than n (see ¶[0149]; identify one or more relevant tags 1460 by filtering a listing of available tags 1460 by category 1450 and/or subcategory).
WANG discloses a real-time expense auditing and machine learning system that labels audit risks based on expense descriptions. SEWAK discloses automatic labeling of text data using a zero-shot generative model, where a ranked list of candidate results is created based on a threshold probability of accuracy. LAL discloses tags for sub-categories of categories. It would have been obvious to include the tags for sub-categories as taught by LAL in the system executing the method of WANG ‘920 with the motivation to label audit risks.
Claim 19 (Currently Amended)
The combination of WANG ‘920 and SEWAK discloses the computer readable medium as set forth in claim 14.
WANG ‘920 does not specifically disclose, but SEWAK discloses, discloses, wherein the filtering comprises: aggregating a plurality label scores for hypothesis associated with the same sub-risk; filtering on the aggregated label scores (see ¶[0067]; probability of a label class is above an acceptable threshold level).
WANG discloses a real-time expense auditing and machine learning system that labels audit risks based on expense descriptions. SEWAK discloses automatic labeling of text data using a zero-shot generative model, where a ranked list of candidate results is created based on a threshold probability of accuracy. It would have been obvious for one of ordinary skill in the art at the time of invention to include the ranking as taught by SEWAK in the system executing the method of WANG with the motivation to label audit risk based on expense descriptions
The combination of WANG ‘920 and SEWAK does not specifically disclose, but LAL discloses, and for all hypothesis associated with sub-risks grouped by a common risk, filtering to a top m sub-risks for the risk grouping, where m is a whole number less than n (see ¶[0149]; identify one or more relevant tags 1460 by filtering a listing of available tags 1460 by category 1450 and/or subcategory).
WANG discloses a real-time expense auditing and machine learning system that labels audit risks based on expense descriptions. SEWAK discloses automatic labeling of text data using a zero-shot generative model, where a ranked list of candidate results is created based on a threshold probability of accuracy. LAL discloses tags for sub-categories of categories. It would have been obvious to include the tags for sub-categories as taught by LAL in the system executing the method of WANG ‘920 with the motivation to label audit risks.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to WANG ‘920 et al. in view of US 20220414137 A1 to SEWAK et al. as applied to claim 1 above, and further in view of US 20200065387 A1 to Matthews et al. (hereinafter ‘MATTHEWS’).
Claim 8 (Original)
The combination of WANG ‘920 and SEWAK discloses the method as set forth in claim 1.
The combination of WANG ‘920 and SEWAK does not specifically disclose, but MATTHEWS discloses, further comprising cleaning the issue description to normalize the issue description (see ¶[0058]-[0059]; the control platform 100 can clean and process the input data normalize data, remove spaces, make text in the same case, and so on. This cleaning and processing can refer to a bag-of-words model, for example. Natural language processing to process large natural language corpa).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on expense descriptions. MATTHEWS discloses report processing through natural language processing techniques that includes cleaning and processing input data and normalizing the data. It would have been obvious to clean and normalize text input as taught by MATTHEWS in the system executing the method of WANG ‘920 with the motivation to label audit risks based on expense descriptions.
Claim(s) 9 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to WANG ‘920 et al. in view of US 20220414137 A1 to SEWAK et al. as applied to claim 1 above, and further in view of US 20150243285 A1 to Lane et al. (hereinafter ‘LANE’).
Claim 9 (Original)
The combination of WANG ‘920 and SEWAK discloses the method as set forth in claim 1.
The combination of WANG ‘920 and SEWAK does not specifically disclose, but LANE discloses, wherein each of one or more of the sub-risks in the risk taxonomy are associated with a plurality of hypothesis (see ¶[0050]; assign an n-best hypotheses list to each arc and state).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. LANE discloses speech recognition that may include a large language model with multiple hypotheses for an arc and state. It would have been obvious to include multiple hypotheses as taught by LANE in the system executing the method of WANG ‘920 with the motivation to apply and efficiently prune a large language model (see LANE ¶[0009]).
Claim 10 (Original)
The combination of WANG ‘920, SEWAK, and LANE discloses the method as set forth in claim 9.
WANG ‘920 does not specifically disclose, but LANE discloses, wherein the plurality of hypothesis are based on different portions of the sub-risk description in the risk taxonomy (see ¶[0009]; large language models contain millions of unique entries and billions of n-gram contexts).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. LANE discloses speech recognition that may include a large language model with multiple hypotheses for an arc and state derived from a large language model with millions of entries. It would have been obvious to include multiple hypotheses derived from millions of entries as taught by LANE in the system executing the method of WANG ‘920 with the motivation to apply and efficiently prune a large language model (see LANE ¶[0009]).
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to WANG ‘920 et al. in view of US 20220414137 A1 to SEWAK et al. as applied to claim 1 above, and further in view of US 20220101115 A1 to Zhou et al. (hereinafter ‘ZHOU’).
Claim 12 (Original)
The combination of WANG and SEWAK discloses the method as set forth in claim 1.
The combination of WANG and SEWAK does not specifically disclose, but ZHOU discloses, further comprising: receiving a hypothesis; determining relevant portions of the issue description to the selected hypothesis; and highlighting the relevant portions of the issue description in a user interface display (see ¶[0024]; keywords relevant to an issue in a labeled sentence can be highlighted).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. ZHOU discloses highlighting relevant keywords in descriptive natural language text that are relevant to a label. It would have been obvious to highlight words in a description that are relevant to a label as taught by ZHOU in the system executing the method of WANG with the motivation to provide information regarding relevant keywords when labeling text.
Claim(s) 4 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to WANG ‘920 et al. in view of US 20220414137 A1 to SEWAK et al. and US 20230196105 A1 to WANG ‘105 et al. as applied to claims 1 and 2 above, and further in view of US 20220138572 A1 to Song et al. (hereinafter ‘SONG’).
Claim 4 (Original)
The combination of WANG ‘920, SEWAK, and WANG ‘105 discloses the method as set forth in claim 3.
The combination of WANG ‘920, SEWAK, and WANG ‘105 does not specifically disclose, but SONG discloses, wherein the hypothesis text applied to the generative LLM is a simplified version of the hypothesis text applied to the zero-shot classification model (see ¶[0038], [0065]-[0068], [0085], and [0098]; predict the values of unseen tokens over a large scale and general domain language corpa. Learn nuances ahead of time rather than performing them on downstream tasks. Concatenate a list of tokenized sentences up to a 512 token limit in pre-training. Some labels only occur in the test set. When pre-training, sentences with less than five words were removed).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. SONG discloses zero-shot pre-training that includes learning nuances ahead of time and limiting token phrases in pre-training. It would have been obvious to include the pre-training as taught by SONG in the system executing the method of WANG ‘920 with the motivation to conduct efficient pretraining of a language model (see SONG ¶[0096]).
Claim 17 (Original)
The combination of WANG ‘920, SEWAK, and WANG ‘105 discloses the computer readable medium as set forth in claim 16.
The combination of WANG ‘920, SEWAK, and WANG ‘105 does not specifically disclose, but SONG discloses, wherein the hypothesis text applied to the generative LLM is a simplified version of the hypothesis text applied to the zero-shot classification model (see ¶[0038], [0065]-[0068], [0085], and [0098]; predict the values of unseen tokens over a large scale and general domain language corpa. Learn nuances ahead of time rather than performing them on downstream tasks. Concatenate a list of tokenized sentences up to a 512 token limit in pre-training. Some labels only occur in the test set. When pre-training, sentences with less than five words were removed).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. SONG discloses zero-shot pre-training that includes learning nuances ahead of time and limiting token phrases in pre-training. It would have been obvious to include the pre-training as taught by SONG in the system executing the method of WANG ‘920 with the motivation to conduct efficient pretraining of a language model (see SONG ¶[0096]).
Claim(s) 11 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to WANG ‘920 et al. in view of US 20220414137 A1 to SEWAK et al. as applied to claim 1 above, and further in view of US 20150243285 A1 to Lane et al. (hereinafter ‘LANE’) as applied to claims 1 and 9 above, and further in view of US 20220366135 A1 to Patel et al. (hereinafter ‘PATEL’).
Claim 11 (Original)
The combination of WANG ‘920, SEWAK, and LANE discloses the method as set forth in claim 9.
The combination of WANG ‘920, SEWAK, and LANE does not explicitly disclose, but PATEL discloses, wherein the plurality of hypothesis are based on different phrasing of a same portion of the same sub-risk description in the risk taxonomy (see ¶[0015]; certain embodiments may require zero-training data given a pre-trained model for accessing syntax dependency trees, parts of speech tags, and tokens, and can easily extend to languages other than English since the heuristics built on recognizing patterns in the syntax dependency tree operate on a universal dependencies framework which is defined in over 100 languages).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. PATEL discloses a pre-trained model that extends into languages other than English. It would have been obvious to include multiple languages as taught by PATEL in the system executing the method of WANG ‘920 with the motivation to provide a pre-trained model for labeling audit risks.
Claim 20 (Original)
The combination of WANG ‘920 and SEWAK discloses the computer readable medium as set forth in claim 14.
The combination of WANG ‘920 and SEWAK does not specifically disclose, but LANE discloses, wherein each of one or more of the sub-risks in the risk taxonomy are associated with a plurality of hypothesis (see ¶[0050]; assign an n-best hypotheses list to each arc and state).
, wherein the plurality of hypothesis are based on one or more of: different portions of the sub-risk description in the risk taxonomy (see ¶[0009]; large language models contain millions of unique entries and billions of n-gram contexts).
The combination of WANG ‘920, SEWAK, and LANE does not explicitly disclose, but PATEL discloses, and different phrasing of a same portion of the same sub-risk description in the risk taxonomy (see ¶[0015]; certain embodiments may require zero-training data given a pre-trained model for accessing syntax dependency trees, parts of speech tags, and tokens, and can easily extend to languages other than English since the heuristics built on recognizing patterns in the syntax dependency tree operate on a universal dependencies framework which is defined in over 100 languages).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. PATEL discloses a pre-trained model that extends into languages other than English. It would have been obvious to include multiple languages as taught by PATEL in the system executing the method of WANG ‘920 with the motivation to provide a pre-trained model for labeling audit risks.
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. LANE discloses speech recognition that may include a large language model with multiple hypotheses for an arc and state. It would have been obvious to include multiple hypotheses as taught by LANE in the system executing the method of WANG ‘920 with the motivation to apply and efficiently prune a large language model (see LANE ¶[0009]).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. PATEL discloses a pre-trained model that extends into languages other than English. It would have been obvious to include multiple languages as taught by PATEL in the system executing the method of WANG ‘920 with the motivation to provide a pre-trained model for labeling audit risks.
Claim(s) 13 and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210073920 A1 to WANG ‘920 et al. in view of US 20220414137 A1 to SEWAK et al. and US 20220101115 A1 to ZHOU et al. as applied to claims 1 and 12 above, and further in view of US 20230368773 A1 to Mishra (hereinafter ‘MISHRA’).
Claim 13 (Original)
The combination of WANG ‘920, SEWAK, and ZHOU discloses the method as set forth in claim 12.
The combination of WANG ‘920, SEWAK, and ZHOU does not specifically disclose, but MISHRA discloses, wherein determining the relevant portions of the issue description comprises: generating a plurality of text groupings based on pairings of sentences in issue description (see ¶[0091]-[0092]; parse alphanumeric characters into tokens. Assign semantic meaning to tokens with relevancy above a threshold);
applying each of text groupings, combined with the hypothesis, to the zero shot classifier to provide a text group scoring for the hypothesis (see ¶[0091]-[0092]; parse alphanumeric characters into tokens. Assign semantic meaning to tokens with relevancy above a threshold).
MISHRA does not specifically disclose, but ZHOU discloses, selecting the text grouping with the highest text group scoring for highlighting (see ¶[0024]; keywords relevant to an issue in a labeled sentence can be highlighted. See also ¶[0046]; rank results in order of confidence level to a threshold from the highest for determinations in downstream processing).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. MISHRA discloses a virtual agent using a large language model that assigns a semantic meaning to a token when a relevancy is above a threshold. It would have been obvious for one of ordinary skill in the art at the time of inventio to apply a threshold to tokenized alphanumeric characters as taught by MISHRA in the system executing the method of WANG ‘920 with the motivation to interpret language using a large language model.
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. ZHOU discloses highlighting relevant keywords in descriptive natural language text that are relevant to a label. It would have been obvious to highlight words in a description that are relevant to a label as taught by ZHOU in the system executing the method of WANG with the motivation to provide information regarding relevant keywords when labeling text.
Claim 21 (Original)
The combination of WANG ‘920 and SEWAK discloses the computer readable medium as set forth in claim 14.
The combination of WANG and SEWAK does not specifically disclose, but ZHOU discloses, further comprising: receiving a hypothesis; determining relevant portions of the issue description to the selected hypothesis (see ¶[0024]; keywords relevant to an issue in a labeled sentence can be highlighted).
The combination of WANG ‘920, SEWAK, and ZHOU does not specifically disclose, but MISHRA discloses, highlighting the relevant portions of the issue description in a user interface display, wherein determining the relevant portions of the issue description comprises: generating a plurality of text groupings based on pairings of sentences in issue description (see ¶[0091]-[0092]; parse alphanumeric characters into tokens. Assign semantic meaning to tokens with relevancy above a threshold);
applying each of text groupings, combined with the hypothesis, to the zero shot classifier to provide a text group scoring for the hypothesis (see ¶[0091]-[0092]; parse alphanumeric characters into tokens. Assign semantic meaning to tokens with relevancy above a threshold).
MISHRA does not specifically disclose, but ZHOU discloses, selecting the text grouping with the highest text group scoring for highlighting (see ¶[0024]; keywords relevant to an issue in a labeled sentence can be highlighted. See also ¶[0046]; rank results in order of confidence level to a threshold from the highest for determinations in downstream processing).
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. MISHRA discloses a virtual agent using a large language model that assigns a semantic meaning to a token when a relevancy is above a threshold. It would have been obvious for one of ordinary skill in the art at the time of inventio to apply a threshold to tokenized alphanumeric characters as taught by MISHRA in the system executing the method of WANG ‘920 with the motivation to interpret language using a large language model.
WANG ‘920 discloses a real-time expense auditing and machine learning system that labels audit risks based on descriptions. ZHOU discloses highlighting relevant keywords in descriptive natural language text that are relevant to a label. It would have been obvious to highlight words in a description that are relevant to a label as taught by ZHOU in the system executing the method of WANG with the motivation to provide information regarding relevant keywords when labeling text.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RICHARD N SCHEUNEMANN whose telephone number is (571)270-7947. The examiner can normally be reached M-F 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patricia Munson can be reached at 571-270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD N SCHEUNEMANN/ Primary Examiner, Art Unit 3624
Prosecution Insights