DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 8-10 and 15-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anand et al. (US PGPUB No. 2023/0177024; Pub. Date: Jun. 8, 2023) in view of Venter et al. (US Patent No.: 10,860,931; Date of Patent: Dec. 8, 2020) and Higgins et al. (US PGPUB No. 2024/0104391; Pub. Date: Mar. 28, 2024).
Regarding independent claim 1,
Anand discloses a computer-implemented method, comprising: receiving a custom-defined data schema, the custom-defined data schema being constructed according to a structured data syntax; See Paragraph [0035], (Disclosing a low-latency data access and analysis system configured for automatic data modeling. External database servers may store data in a format and schema determined externally from an internal database analysis portion 2200 such as by a second or third party associated with an external data source portion 2100.) See Paragraph [0041], (The system comprises internal database analysis portion 2200 which may import enterprise data from external database server 2120, i.e. receiving a custom-defined data schema, the custom-defined data schema being constructed according to a structured data syntax;)
converting the custom-defined data schema into a language model modifier that restricts outputs based on preceding outputs; See Paragraph [0215], (The analytical model may be embodied as a time series model such as an autoregressive integrated moving average (ARIMA) model which may learn functions/parameters based on time series data and may generate predicted values corresponding to temporal locations subsequent to temporal locations indicated in predicate results data, i.e. converting the custom-defined data schema into a language model modifier that restricts outputs based on preceding outputs (e.g. the ARIMA model generates outputs based on information from previous time windows).)
integrating the language model modifier with an autoregressive machine-learned language model to modify output scores of the autoregressive machine-learned language model; See Paragraph [0215], (The analytical model may be embodied as an ARIMA model which may learn functions/parameters based on time series data and may generate predicted values corresponding to temporal locations subsequent to temporal locations indicated in predicate result data.) See Paragraph [0235], (The analysis system may generate and output data relating to analytical model parameters such that said parameters may be modified or updated, i.e. integrating the language model modifier with an autoregressive machine-learned language model to modify output scores of the autoregressive machine-learned language model;
and applying the autoregressive machine-learned language model to the unstructured data to generate a structured dataset that follows the custom-defined data schema, See FIG. 4 & Paragraph [0198], (FIG. 4 illustrates method 400 comprising step 4400 of obtaining a trained model to used to generate output data at step 4900. Note [0215] wherein the analytical model is an ARIMA model, i.e. and applying the autoregressive machine-learned language model to the unstructured data to generate a structured dataset that follows the custom-defined data schema)
The examiner notes that Anand discloses applying an autoregressive ARIMA model but does not explicitly describe the type of data to which the ARIMA model is applied to.
wherein generating the structured dataset comprises: generating a first output from the autoregressive machine-learned language model; See Paragraph [0198], (Method 400 comprises step 4800 of generating an analytical model results data query followed by step 4900 of outputting data as predicate results data, i.e. wherein generating the structured dataset comprises: generating a first output from the autoregressive machine-learned language model;)
Anand does not disclose the step of receiving a data file comprising unstructured data;
Venter discloses the step of receiving a data file comprising unstructured data; See Col. 8, lines 35-40, (Disclosing a system for performing predictive analysis. Unstructured data may be processed through trained extractors and detectors to generate structured data. The structured data may further be used to generate predictive models or be used as input to predictive models to generate predictions.) See Col. 5, lines 18-24, (The system may apply a set of heuristics or algorithms based on the form of an unstructured data source. Note Col. 3, lines 43-44 wherein exemplary algorithms include autoregressive integrated moving average (ARIMA) and other variations of autoregressive algorithms, i.e. receiving a data file comprising unstructured data;)
Anand and Venter are analogous art because they are in the same field of endeavor, predictive analysis. It would have been obvious to anyone having ordinary skill in the art before the effective filing date to modify the system of Anand to include the method of processing unstructured data using predictive analysis algorithms as disclosed by Venter. Col. 17, lines 32-41 of Venter discloses that predictive analytics allows users to identify and quantify problems and opportunities related to one or more performance metrics. Predictive analysis is performed in tandem with root-cause analysis which additionally allows users to identify, quantify and rank influencers of performance metrics which can cause any upcoming problems.
Anand-Venter does not disclose the step of receiving a set of tokens representing candidates of a second output succeeding the first output, each token associated with a score;
identifying a rule in the language model modifier using the first output; modifying one or more scores of the tokens that violate the rule;
and selecting one of the tokens as the second output based on the modified scores.
Higgins discloses the step of receiving a set of tokens representing candidates of a second output succeeding the first output, each token associated with a score; See FIG. 4 & Paragraph [0076], (Disclosing a system for training a language model for performing a reasoning task. FIG. 4 illustrates method 400 comprising step 420 of processing an input query using an auto-regressive language model configured to process a current input sequence to generate a probability distribution of tokens in a vocabulary for the next token in the output. Note [0041] wherein each token is associated with a probability representing the likelihood that the text token immediately follows an input, i.e. receiving a set of tokens representing candidates of a second output succeeding the first output, each token associated with a score;)
identifying a rule in the language model modifier using the first output; See Paragraph [0056], (System 100 may generate candidate reasoning traces for selecting output tokens using a policy language model.) See FIG. 4, (Method 400 comprises step 420 of processing a query using the trained language model to generate candidate output reasoning trace using the policy language model, i.e. identifying a rule in the language model modifier using the first output (e.g. the trained model utilizes a policy language model to characterize outputs).)
modifying one or more scores of the tokens that violate the rule; See Paragraph [0060], (Reward model 140 is configured to generate output tokens indicating correctness for a determined reasoning step by indicating each step as 'correct' or 'incorrect'.) See FIG. 4 & Paragraph [0077], (Method 400 comprises step 430 of selecting a best reasoning trace using the reward model trained to select the best reasoning trace to output, i.e. modifying one or more scores of the tokens that violate the rule (e.g. the trained model may assign labels to text that may be updated over time or by user feedback).)
and selecting one of the tokens as the second output based on the modified scores. See FIG. 4, (Method 400 comprises step 440 of outputting the best reasoning trace having the highest performance score as the 'correct' final answer, i.e. and selecting one of the tokens as the second output based on the modified scores.)
Anand-Vender and Higgins are analogous art because they are in the same field of endeavor, predictive analysis. It would have been obvious to anyone having ordinary skill in the art before the effective filing date to modify the system of Anand-Vender to include the method of using an auto-regressive language model to determine a correct and optimal output based on correctness of reasoning traces as disclosed by Higgins. Paragraph [0011] of Higgins discloses that the use of a reasoning trace allows the system to provide a human-interpretable explanation that allows human users to understand the reasons behind a response provided by the language model.
Regarding dependent claim 2,
As discussed above with claim 1, Anand-Vender-Higgins discloses all of the limitations.
Anand further discloses the step wherein the language model modifier includes a finite state machine. See Paragraph [0147], (Relational analysis unit 377 may implement one or more finite state machines, i.e. wherein the language model modifier includes a finite state machine.)
Regarding dependent claim 3,
As discussed above with claim 1, Anand-Vender-Higgins discloses all of the limitations.
Higgins further discloses the step wherein the score includes a probability distribution indicating a likelihood of the associated token being selected as the second output. See Paragraph [0041], (Language model 110 may be embodied as a neural network configured to process an input to generate an output that includes a probability distribution over a set of text tokens in a vocabulary of text tokens. The probably for each token represents the likelihood that the text token immediately follows the input, i.e. wherein the score includes a probability distribution indicating a likelihood of the associated token being selected as the second output (e.g. the probability associated with each token).)
Regarding independent claim 8,
The claim is analogous to the subject matter of independent claim 1 directed to a computer system and is rejected under similar rationale.
Regarding dependent claim 9,
The claim is analogous to the subject matter of dependent claim 2 directed to a computer system and is rejected under similar rationale.
Regarding dependent claim 10,
The claim is analogous to the subject matter of dependent claim 3 directed to a computer system and is rejected under similar rationale.
Regarding independent claim 15,
The claim is analogous to the subject matter of independent claim 1 directed to a non-transitory, computer readable medium and is rejected under similar rationale.
Regarding dependent claim 16,
The claim is analogous to the subject matter of dependent claim 2 directed to a non-transitory, computer readable medium and is rejected under similar rationale.
Regarding dependent claim 17,
The claim is analogous to the subject matter of dependent claim 3 directed to a non-transitory, computer readable medium and is rejected under similar rationale.
Claim(s) 4, 11 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anand in view of Venter and Higgins as applied to claim 1 above, and further in view of Li et al. (US Patent No. 11,341,339; Date of Patent: May 24, 2022).
Regarding dependent claim 4,
As discussed above with claim 1, Anand-Vender-Higgins discloses all of the limitations.
Anand-Vender-Higgins does not disclose the step wherein modifying one or more scores of the tokens comprises: setting the one or more scores of the tokens to be zero.
Li discloses the step wherein modifying one or more scores of the tokens comprises: setting the one or more scores of the tokens to be zero. See FIG. 4 & Col. 9, lines 15-27, (Disclosing a system for creating and calibrating a natural-language understanding (NLU) machine learning model. FIG. 4 illustrates machine learning service 130 configured to modify the output of a trained model 112 with a tuned score modifying function 114 wherein the model may set a confidence score to a value between 0 and 1, with 0 being the least confidence and 1 being the most confidence, i.e. wherein modifying one or more scores of the tokens comprises: setting the one or more scores of the tokens to be zero (e.g. the confidence score may be set to any value between and inclusive of 0 and 1).)
Anand, Vender, Higgins and Li are analogous art because they are in the same field of endeavor, predictive analysis. It would have been obvious to anyone having ordinary skill in the art before the effective filing date to modify the system of Anand-Vender-Higgins to include the method of calibrating a machine learning model including setting confidence scores for values of text objects as disclosed by Li. Col. 6, lines 2-13 of Li disclose that the system allows a user to finetune a modifying function without modifying an existing model or generating another model to better fit the output from the model, input into the function etc. This allows for finetuning of the function to create a modified output that is better calibrated than the unmodified output from the model.
Regarding dependent claim 11,
The claim is analogous to the subject matter of dependent claim 4 directed to a computer system and is rejected under similar rationale.
Regarding dependent claim 18,
The claim is analogous to the subject matter of dependent claim 4 directed to a non-transitory, computer readable medium and is rejected under similar rationale.
Claim(s) 5, 12 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anand in view of Venter and Higgins as applied to claim 1 above, and further in view of Dangoor et al. (US PGPUB No. 2023/0237053; Pub. Date: Jul. 27, 2023).
Regarding dependent claim 5,
As discussed above with claim 1, Anand-Vender-Higgins discloses all of the limitations.
Anand-Vender-Higgins does not disclose the step wherein the structured data syntax includes one or more of hypertext markup language (HTML), extensible markup language (XML), JavaScript object notation (JSON), structured query language (SQL).
Dangoor discloses the step wherein the structured data syntax includes one or more of hypertext markup language (HTML), extensible markup language (XML), JavaScript object notation (JSON), structured query language (SQL). See Paragraph [0008], (Disclosing a system for training a large language model with query auto-completion training data. The system may train a large language model on new and previously stored software-related queries such as SQL codes that have had aliases removed or normalized from the and having the model predict what the hidden syntax elements are, i.e. wherein the structured data syntax includes structured query language (SQL).)
Anand, Vender, Higgins and Dangoor are analogous art because they are in the same field of endeavor, predictive analysis. It would have been obvious to anyone having ordinary skill in the art before the effective filing date to modify the system of Anand-Vender-Higgins to include the method of using training an LLM to auto-complete SQL queries as disclosed by Dangoor. Paragraph [0008] of Dangoor discloses that the auto-completion process allows users to receive relevant real-time suggestions for completing complex software-related queries.
Regarding dependent claim 12,
The claim is analogous to the subject matter of dependent claim 5 directed to a computer system and is rejected under similar rationale.
Regarding dependent claim 19,
The claim is analogous to the subject matter of dependent claim 5 directed to a non-transitory, computer readable medium and is rejected under similar rationale.
Claim(s) 6-7, 13-14 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Anand in view of Venter and Higgins as applied to claim 1 above, and further in view of Mukherjee (US PGPUB No. 2021/0110912; Pub. Date: Apr. 15, 2021).
Regarding dependent claim 6,
As discussed above with claim 1, Anand-Vender-Higgins discloses all of the limitations.
Anand-Vender-Higgins does not disclose the step of accessing a set of policy rules for processing a transaction request on the data file;
determining whether the data file meet the set of policy rules;
and in response to determining that the data file does not meet at least one of the set of policy rules, perform an action related to the at least one of the set of policy rules.
Mukherjee discloses the step of accessing a set of policy rules for processing a transaction request on the data file; See Paragraph [0038], (Disclosing a system for improved report collaboration including interacting with a reporting assistant. Reporting assistant 216 reviews information of a report and provides rule violations and report improvements using AI technologies such as machine learning or deep learning, i.e. accessing a set of policy rules for processing a transaction request on the data file (e.g. machine learning techniques are applied to a report document which includes detecting rule violations).)
determining whether the data file meet the set of policy rules; See FIG. 6 & Paragraph [0084], (FIG. 6 illustrates method 600 comprises step 618 of evaluating a rule condition related to a content and context of a report. A rule may have multiple potential actions that can be executed based on certain conditions of the report, report context, or extra information from healthcare information system 226 or other engines, i.e. determining whether the data file meet the set of policy rules;)
and in response to determining that the data file does not meet at least one of the set of policy rules, perform an action related to the at least one of the set of policy rules. See FIG. 5 [0076], (FIG. 5 illustrates input/output signals directed to rule engine 502 which outputs a violation 510 to a reporting assistant or report analysis engine for output to a user for correction, i.e. in response to determining that the data file does not meet at least one of the set of policy rules, perform an action related to the at least one of the set of policy rules.)
Anand-Vender-Higgins and Mukherjee are analogous art because they are in the same field of endeavor, machine learning techniques for processing documents. It would have been obvious to anyone having ordinary skill in the art before the effective filing date to modify the system of Anand-Vender-Higgins to include the step of identifying rule violations in a document as disclosed by Mukherjee. Paragraph [0044] of Mukherjee discloses that the reporting system displays report update actions or on-the-fly reminders in the form of icons, prompts and violations around an area of interest in the report. This allows the user to receive report improvement suggestions and incorporate said changes during report authoring and generation.
Regarding dependent claim 7,
As discussed above with claim 6, Anand-Vender-Higgins-Mukherjee discloses all of the limitations.
Mukherjee further discloses the step wherein performing an action related to the at least one of the set of policy rules comprises: transmitting, via a user interface, a notification to a user informing the user about violation of the at least one of the set of policy rules. See Paragraph [0076] rule engine 502 may output a violation 510 to a reporting assistant or report analysis engine for output to a user for correction, i.e. transmitting, via a user interface, a notification to a user informing the user about violation of the at least one of the set of policy rules.)
Regarding dependent claim 13,
The claim is analogous to the subject matter of dependent claim 6 directed to a computer system and is rejected under similar rationale.
Regarding dependent claim 14,
The claim is analogous to the subject matter of dependent claim 7 directed to a computer system and is rejected under similar rationale.
Regarding dependent claim 20,
The claim is analogous to the subject matter of independent claim 6 directed to a non-transitory, computer readable medium and is rejected under similar rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Fernando M Mari whose telephone number is (571)272-2498. The examiner can normally be reached Monday-Friday 7am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J. Lo can be reached at (571) 272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FMMV/Examiner, Art Unit 2159 /ANN J LO/Supervisory Patent Examiner, Art Unit 2159