DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. JP2023-131099, filed on 08/10/2023.
Response to Amendment
The amendments filed 5/15/2026 have been accepted and considered in this office action. Claims 1-3, and 5-7 have been amended. Claims 1-8 are pending.
Response to Arguments
Applicant’s arguments with respects to claims 1-8 have been considered but are moot in view of new grounds of rejection necessitated by the applicant’s amendments to the claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Saito (US 20200356891 A1) in view of Tago et al. (hereinafter Tago) (US 20230290116 A1) in further view of Reza et al. (hereinafter Reza) (US 20230237277 A1).
Regarding claim 1, Saito discloses:
(Currently Amended) A computer system (Saito, P[0020]), wherein comprising:
the computer system an interpretation sentence generation computing device that is connected over a network to an inference system that computing device (Saito P[0026]: "the example architecture 100 includes a client device 102, a network 106, and a server system 104", P[0029]: " the server system 104 can host a machine learning-based (ML-based) platform... Further, the server system 104 can host an explanation platform that provides natural language explanations" (server/network structure discloses computing devices connected over a network, where the ML-based platform is the inference device and the explanation platform is the interpretation sentence generation device)), which receives, from a terminal device, input data including a plurality of features (Saito, P[0030]: "bank statements (BSs) 202 and invoices (IVs) 204 are provided as input to an explanation platform 206" (discloses input data received by the system), P[0034]: "pairs of fields from the documents are considered… These features are referred to as pairwise features." (input data includes plural pairwise features)) and performs an inference using an inference model to provide a prediction value that is output by the inference device as an inference result (Saito, P[0030]: "the ML model 210 matches a bank statement 202 to an invoice 204 and outputs the match as a prediction.", "the prediction includes a score indicating a likelihood that the bank statement 202 matches the invoice 204." (the ML model is the inference model, the match is the inference result, and the likelihood score is the prediction value)),
wherein the interpretation sentence generation computing device stores a large-scale language model that receives an instruction sentence as an input and outputs an interpretation sentence for interpreting a result of the inference, and first text template information that stores first template data (Saito, P[0048]: "each feature code is mapped to one or more natural language explanations" (maps to stored first text template information), "each natural language explanation is provided as an explanation template having one or more parameters that are to be filled in" (maps to first template data)) in which a characteristic of a contribution value indicating a magnitude of a contribution to the result of the inference of a feature in a group constituted with one or more features is associated with a template (Saito, P[0041]: "the LIME framework is able to see which features contribute more or less to the prediction around a certain locality of the original input. For each feature, the LIME framework provides an importance score" (importance score maps to contribution value, more or less maps to magnitude of contribution to the inference result), P[0042]: "positive scores indicate that the pairwise feature contributed towards the prediction, while negative scores indicate that the pairwise feature contributed negatively towards the prediction" (positive/negative score amps to a characteristic of the contribution value), P[0053]: "The importance score is used to select a natural language explanation template from the set of natural language templates" (maps to associating/selecting a template based on the characteristic/magnitude of the contribution value)
acquires, from the inference computing device, the input data submitted to the inference device (Saito, P[0040]: "the LIME framework accepts three inputs: the original input data, the interpretable feature representation of the inputs, and the ML model" (maps to acquiring the input data submitted to the inference device)), and the prediction value output as the result of inference (Saito, P[0031]: "the prediction is provided to the explanation framework 212" (maps to acquiring the prediction value/inference result from the inference model/device)),
calculates the contribution value of each of the plurality of features (Saito, P[0041]: "For each feature, the LIME framework provides an importance score" (maps to calculating contribution value for each feature)) using the input data, the result of the inference, and the inference model (Saito, P[0040]: "the LIME framework accepts three inputs: the original input data, the interpretable feature representation of the inputs, and the ML model" (maps to using the input data and inference model), P[0041]: "makes random perturbations to the input to the ML model to observe how the perturbations affect the predictions" (maps to using the inference result/prediction behavior)),
the first template data by referring to the first text template information (Saito, P[0053]: "The importance score is used to select a natural language explanation template from the set of natural language templates" (maps to referring to stored first text template information to identify first template data)) based on the characteristic of the contribution value (Saito, P[0053]: "The importance score is used to select a natural language explanation template from the set of natural language templates" (Saito supplies the based on" part because the template is selected based on the importance score/contribution value)
generates, based on the acquired first template data and the feature included in the group (Saito, P[0053]: "values of one or more parameters of the selected natural language explanation template are determined from the documents, and are used to populate the natural language explanation template" (maps to generating text based on acquired template data and feature/input values)),
and outputs an interpretation sentence for interpreting the result of the inference (Saito, P[0031]: "the natural language platform 214 processes the raw explanation to provide a natural language explanation 208 for the predictions." (maps the output natural language explanation to an interpretation sentence for interpreting the inference result/prediction)), and
Saito does not explicitly disclose:
of the instruction sentence
generates a plurality of groups each constituted with one or more of the features
acquires, for each of the plurality of groups
of the feature included in the group
the instruction sentence to be input to the large-scale language model
accesses a large-scale language model that receives the instruction sentence as an input
outputs the interpretation sentence obtained by inputting the instruction sentence to the large-scale language model. However, Tago discloses:
generates a plurality of groups each constituted with one or more of the features (Tago, Abstract: "dividing input data including a plurality of items related to data to be estimated that is a target of an estimation process using a machine learning model into a plurality of groups based on a predetermined condition" (maps to generating a plurality of groups from input features/items), P[0041]: "The divided feature graph generated from the feature graph of the data to be estimated corresponds to an example of a “group”. In other words, the feature value division unit 12 divides the input data including the items related to the data to be estimated into a plurality of groups" (maps the generated divided feature graph/groups to groups constituted with features/items)),
acquires, for each of the plurality of groups (Tago, P[0041]: "divides the input data including the items related to the data to be estimated into a plurality of groups" (maps to the existence of a plurality of groups for which the later template acquisition is applied)),
of the feature included in the group (Tago, P[0041]: "The divided feature graph generated from the feature graph of the data to be estimated corresponds to an example of a “group”. In other words, the feature value division unit 12 divides the input data including the items related to the data to be estimated into a plurality of groups" (Tago supplies the group structure and the idea that features/items/feature values are included in the group)),
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Saito in view of Tago. Doing so would have provided Saito’s selected natural language explanation templates based on feature and importance score (Saito, P[0040]-P[0041], P[0053]), with Reza’s prompt templates having a prompt/aspect/bank and creating prompting functions for language model input (Reza, P[0038]-P[0041]). Doing so would have predictable combined Sait’s contribution/importance score based template selection with Reza’s prompt based language model generation to improve flexibility and fluency of generated explanations.
The combination of Saito and Tago does not explicitly disclose:
of the instruction sentence
the instruction sentence to be input to the large-scale language model
accesses a large-scale language model that receives the instruction sentence as an input
outputs the interpretation sentence obtained by inputting the instruction sentence to the large-scale language model
However, Reza discloses:
of the instruction sentence (Reza, P[0038]: "each of the prompting templates comprises a prompt for a text example, an aspect extracted from the text example, and a blank" (maps the template to a prompt/instruction sentence template rather than merely final explanation template)),
the instruction sentence to be input to the large-scale language model (Reza, P[0041]: "The prompting functions 120 with the masked prompting templates are then input into the model." (maps to accessing a language model that receives the instruction/prompt as input)),
accesses a large-scale language model that receives the instruction sentence as an input (Reza, P[0041]:" The prompting functions 120 with the masked prompting templates are then input into the model" (maps to accessing a language model that receives the instruction/prompt as input))
outputs the interpretation sentence obtained by inputting the instruction sentence to the large-scale language model (Reza, P[0041], prompting functions are input into the model and the model predicts text which maps to obtaining text output by inputting the instruction/prompt to the language model).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Saito in view of Tago and Reza. Doing so would have provided Tago’s grouping of input data/items into plural groups (Tago, P[0040]-P[0041]) with Reza’s prompt template and language model prompting framework (Reza, P[0038]-P[0041]) and Saito’s importance score based selection and population of natural language explanation templates (Saito, P[0053]). The combination would have predictably allowed Saito’s contribution aware explanation templates to be organized by Tago’s related feature groups and used as Reza style prompt instruction sentences for a language model, improving flexibility and fluency of the generated interpretation.
Regarding claim 5, claim 5 recites the method corresponding to the system described in claim 1 and is rejected for the same reasons described above.
Claims 2, 3, 6 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Saito (US 20200356891 A1) in view of Tago et al. (hereinafter Tago) (US 20230290116 A1) in further view of Reza et al. (hereinafter Reza) (US 20230237277 A1) and Tunstall-Pedoe et al. (hereinafter Tunstall-Pedoe) (US 20230259705 A1).
Regarding claim 2, the combination of Saito, Tago, and Reza disclose the computer system according to claim 1.
Saito further discloses:
wherein the
interpretation sentence generation computing device computer system
stores second text template information that stores second template data (Saito, P[0048]: "each feature code is mapped to one or more natural language explanations" (maps to stored second text template information), P[0048]: " each natural language explanation is provided as an explanation template having one or more parameters that are to be filled in." (maps to second template data)) in which a relationship between a value of the feature and the contribution value is associated with a second template for verbalizing the relationship (Saito, P[0048]: "The values that are used to populate the parameters are provided from the data itself" (maps to the value of the feature), P[0041]: "For each feature, the LIME framework provides an importance score" (maps to contribution value), P[0053]: "The importance score is used to select a natural language explanation template from the set of natural language templates." (maps the feature value/contribution value relationship to a selected template for verbalizing the relationship)),
analyzes the relationship for the feature for which the relationship needs to be analyzed (Saito, P[0041]: "makes random perturbations to the input to the ML model to observe how the perturbations affect the predictions" (maps to analyzing the relationship between the feature/input and its contribution to the prediction), "the LIME framework is able to see which features contribute more or less to the prediction" (maps to analyzing which feature has what contribution relationship)),
specifies the second template data by referring to the second text template information (Saito, P[0053]: "a feature code is determined for each pairwise feature, and the feature code is used to identify a set of natural language explanation templates" (maps to referring to second text template information and identifying possible second template data)) based on a result of the analysis (Saito, P[0053]: "The importance score is used to select a natural language explanation template from the set of natural language templates" (maps the analysis result/contribution score to specifying the selected second template data)),
generates a relationship document that verbalizes the relationship, based on the specified second template data and the feature for which the relationship needs to be analyzed (Saito, P[0053]: "values of one or more parameters of the selected natural language explanation template are determined from the documents, and are used to populate the natural language explanation template to provide a natural language explanation." (maps to generating a relationship document/text that verbalizes the feature/contribution relationship using specified template data and feature values))
Saito, in combination with Tago and Reza, does not explicitly disclose:
executes language processing for determining whether the interpretation sentence includes a fact inconsistent with the relationship corresponding to the relationship document
excludes, from the interpretation sentence to be output, the interpretation sentence including the fact inconsistent with the relationship indicated in the relationship document
However, Tunstall-Pedoe discloses:
executes language processing for determining whether the interpretation sentence includes a fact inconsistent with the relationship corresponding to the relationship document (Tunstall-Pedoe, P[0034]: "Extracting from the natural language a collection of one or more factual assertions asserted within the natural language" (maps to language processing the interpretation sentence), P[0035]: "Checking the one or more factual assertions for factual accuracy", P[1045]: "Continuations generated by the LLM can be translated into UL using the techniques described herein and the UL validated for factual inaccuracies or other contradictions" (maps to checking LLM-generated text for inconsistency with represented relationship/fact information)), and
excludes, from the interpretation sentence to be output, the interpretation sentence including the fact inconsistent with the relationship indicated in the relationship document (Tunstall-Pedoe: P[1045]: "Where such problems are found, an alternative continuation not including the factual inaccuracy could be generated by the LLM or by the UL (or similar) system before being displayed to the user. Alternatively, the incorrect assertion could be removed from the language displayed to the user" (maps to excluding/removing inconsistent facts or an inconsistent interpretation sentence before output)).
It would have been prima facie obvious to one of ordinary skill in the art before the earliest filing date of the claimed invention to have modified Saito in view of Tago, Reza, and Tunstall-Pedoe. Doing so would have provided Tago’s grouping of input data/items into plural groups (Tago, P[0040]-P[0041]), with Reza’s language model prompt generation (Reza, P[0038]-P[0041]), and Tunstall-Pedoe’s validation of LLM-generated continuations for factual inaccuracies or contradictions (Tunstall-Pedoe, P[1045]), with Saito’s feature importance based template selection (Saito, P[0053]). This combination would have predictably generated grouped, contribution aware interpretation text while reducing inaccurate or inconsistent language model output.
Regarding claim 3, the combination of Saito, Tago, Reza, and Tunstall-Pedoe discloses the computer system according to claim 2.
The combination further discloses:
wherein
the interpretation sentence generation computing device computer system selects a plurality of target features based on magnitudes of the contribution values of the plurality of features (Saito, P[0042]: "The output of the explanation framework is a list of tuples that include the pairwise features and their respective importance scores." (maps to features and their contribution values), P[0042]: "the relative importance of each pairwise feature can be ascertained by comparing it to others" (maps to selecting target features based on magnitudes/relative values of the contribution scores))
generates the group having the plurality of target features as elements (Tago, P[0031]: "the division definition is conditions for collecting the parts assumed to have similar functions, and the feature values of the parts" (maps to collecting feature values/features into a group), P[0031]: "the division definition specifies a set of items to be the similar bases for estimation." (maps to group having a plurality of target feature/item elements), P[0039]: "adds the feature value corresponding to each part included in the divided feature graph", P[0041]: "divides the input data including the items related to the data to be estimated into a plurality of groups" (maps to generating a group having feature/feature-value elements)).
Regarding claim 6, claim 6 recites the method corresponding to the system described in claim 2 and is rejected for the same reasons described above.
Regarding claim 7, claim 7 recites the method corresponding to the system described in claim 3 and is rejected for the same reasons described above.
Claims 4 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Saito (US 20200356891 A1) in view of Tago et al. (hereinafter Tago) (US 20230290116 A1) in further view of Reza et al. (hereinafter Reza) (US 20230237277 A1), Tunstall-Pedoe et al. (hereinafter Tunstall-Pedoe) (US 20230259705 A1), Chan et al. (hereinafter Chan) (US 20190325333 A1), and MacNeil et al. (hereinafter MacNeil) (Prompt Middleware: Mapping Prompts for Large Language Models to UI Affordances).
Regarding claim 4, the combination of Saito, Tago, Reza, and Tunstall-Pedoe discloses the computer system according to claim 3.
The combination of Saito, Tago, Reza, and Tunstall-Pedoe does not explicitly disclose:
an interface configured to correct the target feature;
and
an interface configured to input or correct the relationship.
However, Chan discloses:
comprising:
an interface configured to correct the target feature (Chan, P[0112]: "an indication to modify a feature value associated with a feature to be a particular value for all entries is received" (maps to an interface receiving a correction/modification for a target feature), P[0112]: "the feature is modified to be the particular value for all entries" (maps to correcting the target feature));
and
an interface configured to input or correct the relationship (Chan, P[0047]: "A partial dependence plot may show how machine-learned response functions change based on the values of an input feature of interest" (maps to displaying/inputting the feature value/model response relationship), P[0112]: "an indication to modify a feature value associated with a feature to be a particular value for all entries is received." (maps to correcting the feature value side of the relationship through an interface)).
It would have been prima facie obvious to one of ordinary skill in the art before the earliest filing date of the claimed invention to have modified Saito in view of Tago, Reza, Tunstall-Pedoe, and Chan. Doing so would have provided Tago’s grouping of input data/items into plural groups (Tago, P[0040]-P[0041]), with Reza’s language model prompt generation (Reza, P[0038]-P[0041]), Tunstall-Pedoe’s validation of LLM-generated continuations for factual inaccuracies or contradictions (Tunstall-Pedoe, P[1045]), and Chan’s interface for modifying feature values and updating model-interpretation information (Chan, P[0100]-P[0104]) with Saito’s feature importance based template selection (Saito, P[0053]). This combination would have predictably allowed correction of feature information used in the generated explanation while preserving grouped, contribution-aware, and fact-checked natural language interpretation.
Chan, in combination with Saito, Tago, Reza, and Tunstall-Pedoe does not explicitly disclose:
an interface configured to select the instruction sentence to be input to the large-scale language model
However, MacNeil discloses:
an interface configured to select the instruction sentence to be input to the large-scale language model (MacNeil, Abstract: "template-based prompt middleware to enable feedback seekers to specify the types of feedback they want to receive as options in a UI. These options are composed using a template to form a feedback request prompt to GPT-3." (maps the selected UI options to an instruction sentence input to an LLM));
It would have been prima facie obvious to one of ordinary skill in the art before the earliest filing date of the claimed invention to have modified Saito in view of Tago, Reza, Tunstall-Pedoe, Chan, and MacNeil. Doing so would have provided Tago’s grouping of input data/items into plural groups (Tago, P[0040]-P[0041]), with Reza’s language model prompt generation (Reza, P[0038]-P[0041]), Tunstall-Pedoe’s validation of LLM-generated continuations for factual inaccuracies or contradictions (Tunstall-Pedoe, P[1045]), Chan’s interface for modifying feature values and updating model-interpretation information (Chan, P[0100]-P[0104]), and MacNeil’s user interface for selecting prompt options that are composed using a template to form a prompt to GPT-3 (MacNeil, Abstract) with Saito’s feature importance based template selection (Saito, P[0053]). This combination would have predictably allowed user correction of feature information and user selection of prompt/instruction options while generating grouped, contribution-aware, and fact-checked natural language explanations.
Regarding claim 8, claim 8 recites the method corresponding to the system described in claim 4 and is rejected for the same reasons described above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHASHIDHAR S MANOHARAN whose telephone number is (571)272-6772. The examiner can normally be reached M-F 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at 571-272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHASHIDHAR SHANKAR MANOHARAN/Examiner, Art Unit 2655
/ANDREW C FLANDERS/Supervisory Patent Examiner, Art Unit 2655
Prosecution Insights