Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the following communication: Non-Provisional Application filed May 10, 2024.
Claims 1-20 are pending in the case. Claims 1, 9 and 17 are independent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Johnson et al. (hereinafter Johnson) U.S. Patent Publication No. 2014/0298199 in view of Deng et al. (hereinafter Deng) “Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering” 2022 and further in view of Besta et al. (hereinafter Besta) “Graph of Thoughts: Solving Elaborate Problems with Large Language Models” Published March 24, 2024.
With respect to independent claim 1, Johnson teaches a networked computer system (see e.g., Para [44][45]- “The network 102 may include local network connections and remote connections in various embodiments, such that the QA system 100 may operate in environments of any size, including local and global, e.g., the Internet.”) comprising:
a data storage server storing a data source including information associated with a plurality of evidence documents (see e.g., Para [45][46] - “the QA system 100 may receive input from the network 102, a corpus of electronic documents 106 or other data, a content creator 108, content users, and other possible sources of input. In one embodiment, some or all of the inputs to the QA system 100 may be routed through the network 102. The various computing devices 104 on the network 102 may include access points for content creators and content users.”); and
a data analysis computer server including one or more data analysis processors coupled to the data storage server and to an artificial intelligence (AI) computer system, the one or more data analysis processors programmed to execute an algorithm (see e.g., Fig. 7A-7E 9,10 and Para [115]-[120]) including the steps of:
rendering a data analysis input screen on a display device of a user computing device, the data analysis input screen including a research question input prompt (see e.g., Fig. 7A-7E Para [115]-“As shown in FIG. 8, the operation starts by receiving an input question from a user or client device of a user (step 810), such as via the question input portion 710 of the GUI 700 in FIG. 7A, for example. The input question is provided to a QA system which generates an initial listing of candidate answers, corresponding confidence measures, and corresponding evidence passages, relevance scores, and links to source documents for each of the candidate answers (step 820).”);
receiving a research question from a user via the research question input prompt (see e.g., Para [115]-“the operation starts by receiving an input question from a user or client device of a user (step 810), such as via the question input portion 710 of the GUI 700 in FIG. 7A,”);
querying the AI computer system to extract evidence from the plurality of evidence documents included in the data source based on the plurality of decomposed questions (see e.g., Para [73] - “The queries may be applied to one or more databases storing information about the electronic texts, documents, articles, websites, and the like, that make up the corpus of data/information. The queries being applied to the corpus of data/information generate results identifying potential hypotheses for answering the input question which can be evaluated.”);
querying the AI computer system to select one or more evidence documents associated with the one or more entry-level answers (see e.g., Fig. 6 and Para [23][95] -“the evidence passage portion of the GUI may be organized by candidate answer with the evidence passages contributing to the confidence score of the candidate answer being displayed in association with the candidate answer.”); and
generating a data structure by: determining a corresponding confidence score associated with each entry-level answer (see e.g., Johnson Para [81][95][102][115][116]]); and identifying a corresponding evidence document used in determining each entry-level answer (see e.g., Fig. 6 and Para [23][95] -“ the evidence passage portion of the GUI may be organized by candidate answer with the evidence passages contributing to the confidence score of the candidate answer being displayed in association with the candidate answer.”);
generating a final answer to the research question based on the data structure (see e.g., Para [80]); and rendering a data analysis results screen on the display device displaying the final answer, the one or more entry-level answers, and information included in the data structure including the corresponding confidence score and the corresponding evidence document associated with each entry-level answer (see e.g., Fig. 6 and Para [23][95]).
Johnson does not expressly show the features discussed below. However, Johnson expressly teaches decompose questions into queries (see e.g., Para [73] - “The identified major features may then be used during the question decomposition stage 530 to decompose the question into one or more queries that may be applied to the corpus of data/information in order to generate one or more hypotheses. The queries may be generated in any known or later developed query language, such as the Structure Query Language (SQL), or the like.”) Furthermore, Deng teaches querying the AI computer system to establish a question decomposition data structure including a plurality of decomposed questions based on the received research question (see e.g., Abstract Sect. 3.1 Algorithm 1) and querying the AI computer system to determine one or more entry-level answers associated with each decomposed question based on the extracted evidence from the plurality of evidence documents (see e.g., Fig. 2 and Sect. 3.5 ). Both Johnson and Deng are directed to computer auto answer generation methods. Accordingly, it would have been obvious to the skilled artisan before the effective filing date of the claimed invention having Johnson and Deng in front of them to modify the system of Johnson to include the above feature. The motivation to combine Johnson and Deng comes from Deng. Deng discloses the motivation to decompose a question into several simpler question so that question can be better understood and more accurate answer can be generated (see e.g. sect 3.1). This motivation for combination also applies to the remaining claims which depend on this combination.
Johnson-Deng does not expressly show generating a reasoning graph data structure. However, Besta teaches similar feature (see e.g., Page 17682 – “see e.g., Page 17682 – “The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information (“LLM thoughts”) are vertices, and edges correspond to dependencies between these vertices.””). Both Johnson and Besta are directed to answer generation methods. Accordingly, it would have been obvious to the skilled artisan before the effective filing date of the claimed invention having Johnson and Besta in front of them to further modify the modified system of Johnson to include the above feature. The motivation to combine Johnson and Besta comes from Besta. Besta discloses the motivation to organize thoughts/reasoning into a graph so that the thoughts/reasoning can be organized in such a way that closer to human thinking (see e.g. Page 17682 Abstract). This motivation for combination also applies to the remaining claims which depend on this combination.
With respect to dependent claim 2, the modified Johnson teaches the one or more data analysis processors is programmed to execute the algorithm including the steps of: generating the reasoning graph data structure including a plurality of extraction nodes (see e.g., Besta Abstract – “The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information (“LLM thoughts”) are vertices, and edges correspond to dependencies between these vertices.”) associated with the one or more entry-level answers indicating evidence data used in generating the one or more entry-level answers (see e.g., Para [73][74][95]- “The evidence passage engine 636 comprises logic for generating a portion of the GUI output that lists the evidence passage contributing to the confidence score for each individual candidate answer. That is, the evidence passage portion of the GUI may be organized by candidate answer with the evidence passages contributing to the confidence score of the candidate answer being displayed in association with the candidate answer.”).
With respect to dependent claim 3, the modified Johnson teaches the one or more data analysis processors is programmed to execute the algorithm including the steps of: generating a corresponding extraction node associated with a corresponding entry-level answer by: identifying the corresponding evidence document used in determining the corresponding entry-level answer (see e.g., Para [112]-[115]) and including a document ID associated with the corresponding evidence document (see e.g., Para [112] [115]– “This information may include a name of the source document, a veracity measure for the source document as may be generated by the QA system, topic or category information associated with the source document, or even the actual content of the source document for review by the user,”).
With respect to dependent claim 4, the modified Johnson teaches the one or more data analysis processors is programmed to execute the algorithm including the steps of: generating the corresponding extraction node by: identifying evidence text included in the corresponding evidence document used in determining the corresponding entry-level answer and establishing an evidence location ID associated with identified evidence text (see e.g., Para [112] – “at least a portion of the content of the source document in close proximity to the portion of the source document used to generate the evidence passage,”).
With respect to dependent claim 5, the modified Johnson teaches the one or more data analysis processors is programmed to execute the algorithm including the steps of: receiving an evidence view request from the user (see e.g., Para [103]) to view evidence information associated with a user selected entry-level answer (see e.g., Para [112] – “As shown in FIG. 7E, the evidence passages of the evidence passage portion 730 are output with corresponding drill-down GUI elements 739.”); and rendering an evidence window displaying an evidence trace including the evidence data associated with the user selected entry-level answer (see e.g., Para [112] – “This display of source document information 780 may include various information about the source document in the corpus from which the evidence passage was generated.”).
With respect to dependent claim 6, the modified Johnson teaches the one or more data analysis processors is programmed to execute the algorithm including the steps of: querying the reasoning graph data structure to identify the corresponding extraction node associated with the user selected entry-level answer (see e.g., Fig. 7E Para [103][112] and Besta Page 17682 Abstract – Besta discloses the reasoning graph and Johnson discloses source/evidence association); querying the corresponding extraction node to identify the document ID and the evidence location ID (see e.g., Para [103][112); querying the data source to retrieve corresponding evidence text based on the document ID and the evidence location ID (see e.g., Para [103][112]); and rendering the evidence window displaying the evidence trace including corresponding evidence text (see e.g., Para [112]-[115] – “This display of source document information 780 may include various information about the source document in the corpus from which the evidence passage was generated.”)).
With respect to dependent claim 7, the modified Johnson teaches the one or more data analysis processors is programmed to execute the algorithm including the steps of: receiving user modified entry-level answer data via a custom value prompt displayed with the evidence window (see e.g., Para [110]-[111] – “the user is free to enter any text into the free-form text field 728 and have that text added as a new candidate answer to the ranked listing of candidate answers 724. ”); generating a user modified data node associated with the user modified entry-level answer data and modifying the reasoning graph data structure to include the user modified data node (see e.g., Para [110]-[111] – “the free-form text field 728 and have that text added as a new candidate answer to the ranked listing of candidate answers 724. ” Besta teaches the reasoning graph – see discussion above with respect to claim 1); and modifying the final answer based on the modified reasoning graph data structure (see e.g., Para [110]-[111] – “The newly entered candidate answer is evaluated by the QA system as described above so as to generate a corresponding set of entries 760 in the evidence passage portion 730 and entry in the candidate answer portion 720 with corresponding confidence score and the like. This may require re-evaluation of the ranked listing of candidate answers based on the confidence score generated for the new candidate answer relative to the other candidate answers.” Besta teaches the reasoning graph – see discussion above with respect to claim 1).
With respect to dependent claim 8, the modified Johnson teaches generating each extraction node including a node ID; and upon receiving the user selected entry-level answer, identifying a corresponding node ID associated with the user selected entry-level answer; and querying the reasoning graph data structure to identify the corresponding extraction node associated with the corresponding node ID (Para [103] – selected answer automatically presents the evidence associated with it. Besta teaches the reasoning graph – see discussion above with respect to claim 1).
Claim 9 is rejected for the similar reasons discussed above with respect to claim 1.
Claim 10 is rejected for the similar reasons discussed above with respect to claim 2.
Claim 11 is rejected for the similar reasons discussed above with respect to claim 3.
Claim 12 is rejected for the similar reasons discussed above with respect to claim 4.
Claim 13 is rejected for the similar reasons discussed above with respect to claim 5.
Claim 14 is rejected for the similar reasons discussed above with respect to claim 6.
Claim 15 is rejected for the similar reasons discussed above with respect to claim 7.
Claim 16 is rejected for the similar reasons discussed above with respect to claim 8.
Claim 17 is rejected for the similar reasons discussed above with respect to claim 1.
Claim 18 is rejected for the similar reasons discussed above with respect to claim 2.
Claim 19 is rejected for the similar reasons discussed above with respect to claims 3 and 4.
With respect to dependent claim 20, the modified Johnson teaches receiving an evidence view request from the user to view evidence information associated with a user selected entry-level answer (Para [103] –“automatically bring to the forefront of the GUI 700 display the corresponding evidence passages for the selected candidate answers.”); querying the reasoning graph data structure to identify the corresponding extraction node associated with the user selected entry-level answer; querying the corresponding extraction node to identify the document ID and the evidence location ID; querying the data source to retrieve corresponding evidence text based on the document ID and the evidence location ID (Para [103] [112] Fig. 7E – “In response to a user selecting a drill-down GUI element 739 of a corresponding evidence passage 770, a corresponding display of source document information 780 is provided. “); and rendering an evidence window displaying an evidence trace including the corresponding evidence text associated with the user selected entry-level answer (see e.g., Para [112] – “This display of source document information 780 may include various information about the source document in the corpus from which the evidence passage was generated.”).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain.” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co. v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert. denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F.3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir. 2005); Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEIYONG WENG whose telephone number is (571)270-1660. The examiner can normally be reached on Mon.-Fri. 8 am to 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Matthew Ell, can be reached on (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://portal.uspto.gov/external/portal. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/PEI YONG WENG/Primary Examiner, Art Unit 2141
Prosecution Insights