DETAILED ACTIONNotice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-10 are pending and have been examined.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/31/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119(a)-(d). The present application claims priority to Japanese patent application JP 2023-186944, filed October 31, 2023. The effective filing date of the claimed invention is therefore October 31, 2023. Certified copies of the priority document have not been received; applicant is advised to perfect the foreign priority claim under 37 CFR 1.55.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 (Statutory Category). Claim 1 is directed to an information processing system (a machine), claim 9 is directed to an information processing method (a process), and claim 10 is directed to a non-transitory computer readable storage medium (an article of manufacture). Each claim is therefore directed to a statutory category of invention.
Step 2A, Prong One (Recitation of a Judicial Exception). Independent claims 1, 9, and 10 recite, in pertinent part, acquiring a question based on input from a user, determining whether the question relates to a predetermined field, acquiring answer basis information corresponding to the question from a database relating to the predetermined field, creating an answer to the question based on the acquired answer basis information, and sending information answering the question to the user. These limitations, under their broadest reasonable interpretation, recite a mental process - i.e., concepts performed in the human mind, including observation, evaluation, and judgment. A person (for example, a reference librarian or a subject-matter expert) can receive a question, judge in the mind whether the question falls within a particular field, consult reference material relating to that field, formulate an answer from that reference material, and convey the answer to the inquirer. The recitation of a “language model” that creates the answer does not remove the limitation from the mental-process grouping, because the claim recites the language model at a high level of generality merely as a tool that produces the answer; the claimed determining and answer-formulating steps mirror the evaluation and judgment that a person performs in the mind. The claims therefore recite an abstract idea.
Step 2A, Prong Two (Integration into a Practical Application). The judicial exception is not integrated into a practical application. Beyond the abstract idea, claim 1 recites the additional elements of “at least one processor,” “at least one memory device,” a “database,” and a “language model” (claims 9 and 10 recite corresponding additional elements). These additional elements amount to mere instructions to apply the abstract idea on generic computer components and to use a generically-recited language model as a tool to perform the abstract idea (see MPEP 2106.05(f)). Acquiring the question based on input from a user is insignificant extra-solution activity in the nature of data gathering, and sending information answering the question to the user is insignificant extra-solution activity in the nature of outputting/transmitting a result (see MPEP 2106.05(g)). The limitation that the database “relat[es] to the predetermined field” is a field-of-use limitation that does not impose meaningful limits on practicing the abstract idea (see MPEP 2106.05(h)). The claims do not recite any improvement to the functioning of a computer or to any other technology or technical field; the claimed advance, if any, lies in the abstract idea itself and not in the way the recited generic components or language model operate. Accordingly, the additional elements do not integrate the abstract idea into a practical application, and the claims are directed to the abstract idea.
Step 2B (Inventive Concept). The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration, the additional elements of a processor, a memory device, a database, and a language model used to acquire, store, retrieve, generate, and transmit data are recited at a high level of generality and perform well-understood, routine, and conventional functions previously known in the art (see MPEP 2106.05(d); receiving and storing data, retrieving information from a database, and transmitting data over a network are recognized as well-understood, routine, and conventional computer functions). Using a language model to generate a natural-language answer from retrieved information was likewise a well-understood, routine, and conventional practice in the art as of the effective filing date, as evidenced by the prior art of record (see, e.g., Gomes (US 2021/0343295 A1), discussed below). Considered individually and as an ordered combination, the additional elements amount to no more than mere instructions to apply the exception using generic computer components and therefore do not provide an inventive concept. The claims are not patent eligible.
Dependent Claims. The dependent claims have been considered and do not cure the deficiencies of the independent claims. Claims 2, 3, and 4 recite further determinations and conditional responses (creating an answer from the question when the question does not relate to the predetermined field; determining whether the question relates to a field for which an answer is to be refused; and indicating that the question is not answerable), which further describe the abstract mental process of judging a question and deciding how to respond, and do not add any additional element beyond the generically-recited processor and language model. Claims 5 and 6 recite that a “for-determination language model” performs the determination, which merely applies a generically-recited model as a tool to perform the abstract determining step. Claim 7 recites that the answer is created based on the question and the answer basis information, which is part of the abstract idea. Claim 8 recites generating a feature vector from the question and acquiring answer basis information associated with a similar feature vector; this recites a mathematical concept (representing text as a vector and computing similarity) and/or further describes the abstract retrieval step at a high level of generality, and does not recite a technical improvement to the database or to computer functionality. The dependent claims are therefore also directed to an abstract idea without significantly more and are rejected for the same reasons as the independent claims.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) Conclusion. The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 2-7 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor regards as the invention.
Each of claims 2-7 recites “the plurality of instructions” (e.g., claim 2: “wherein the plurality of instructions cause the at least one processor to …”). There is insufficient antecedent basis for this limitation in the claims. Independent claim 1, from which claims 2-7 depend, recites “at least one memory device storing instructions”, not “a plurality of instructions.” It is therefore unclear whether “the plurality of instructions” refers to the previously recited “instructions” of claim 1 or to a different set of instructions. For purposes of examination, “the plurality of instructions” is being interpreted as referring to the “instructions” recited in claim 1. Applicant may overcome this rejection by amending claim 1 to recite “a plurality of instructions,” or by amending claims 2-7 to recite “the instructions.”
Note: Claim 8 (which recites “the database”, for which claim 1 provides antecedent basis) and claim 10 (which independently recites “a plurality of instructions”) do not contain this defect and are not rejected on this ground.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless -
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 5, 7, 9, and 10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Gomes et al. (US 2021/0343295 A1), hereinafter Gomes.
Claims 5 and 7 are examined as best understood in view of the indefiniteness identified under 35 U.S.C. 112(b) above; specifically, the recitation “the plurality of instructions” is treated as the “instructions” recited in claim 1.
Regarding claims 1, 9, and 10, Gomes teaches:
(claim 1) An information processing system, comprising: at least one processor; and at least one memory device storing instructions which, when executed by the at least one processor, cause the at least one processor to: (Gomes, ¶¶ [0055]-[0057]; FIG. 8, a general-purpose computing device having a processor 820, memory 830, and a storage device 860 storing modules/instructions that control the processor; reading on claim 1 (system; Gomes claim 9), claim 9 (method; Gomes claim 1, FIG. 7), and claim 10 (non-transitory computer readable storage medium; Gomes claim 16, ¶ [0057]))
acquire a question based on input from a user; (Gomes, ¶¶ [0025]-[0026], [0048] (step 704); FIGS. 1, 7 - the user enters a textual query that the chatbot receives)
acquire a determination result as to whether the question relates to a predetermined field; (Gomes, ¶¶ [0027], [0043], [0048] (step 712); FIGS. 2 (domain classification model 208), 7 - the trained machine-learning classifier determines the identified subject/domain of the query (e.g., PAYROLL, BENEFITS, GENERAL))
acquire answer basis information corresponding to the question from a database relating to the predetermined field; (Gomes, ¶¶ [0027]-[0029], [0049] (steps 714-718); FIGS. 2 (212), 7 - based on the identified subject/domain, the system identifies and retrieves restricted data from the restricted database and fills a response template with that data)
request, when it is determined that the question relates to the predetermined field, a for-answer language model to create an answer to the question based on the acquired answer basis information, and acquire an answer from the for-answer language model; and (Gomes, ¶¶ [0035], [0038]-[0039], [0049] (step 720); FIG. 2 (machine comprehension model 214), the machine comprehension model receives the query and the facts retrieved from the restricted database and, by converting both to embeddings and applying an attention mechanism, predicts the tokens forming the reply)
send information answering the question to the user based on the acquired answer. (Gomes, ¶¶ [0024], [0035], [0049] (step 722); FIG. 7, upon the comprehension score exceeding the threshold, the generated response is provided to the requestor)
Claims 9 and 10 recite limitations commensurate in scope with claim 1 (claim 9 in method form; claim 10 in non-transitory computer readable storage medium form) and are rejected on the same basis as set forth above for claim 1.
Regarding claim 5, Gomes teaches claim 1, and further teaches:
(claim 5, first limitation) wherein the plurality of instructions cause the at least one processor to request a for-determination language model to determine whether the question relates to the predetermined field, (Gomes, ¶¶ [0027], [0037], [0043]; FIGS. 2 (208), 7 (step 712), a trained machine-learning classifier (e.g., logistic regression, multi-layer perceptron, or a BERT-based model per ¶ [0037]) classifies the query into a domain)
and to determine whether the question relates to the predetermined field based on information output by the for-determination language model. (Gomes, ¶ [0027], the classifier outputs, for a new phrase, the probability of belonging to each domain distribution, and the subject/domain determination is made based on that output)
Regarding claim 7, Gomes teaches claim 1, and further teaches:
(claim 7) wherein the plurality of instructions cause the at least one processor to request the for-answer language model to create an answer to the question based on the question and the acquired answer basis information, and acquire an answer from the for-answer language model. (Gomes, ¶¶ [0038]-[0039], the machine comprehension model receives the query together with the facts obtained from the restricted database and outputs the probabilities of the tokens forming the answer)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 3, 4, and 6 are examined as best understood in view of the indefiniteness identified under 35 U.S.C. 112(b) above; specifically, the recitation “the plurality of instructions” is treated as the “instructions” recited in claim 1.
Claims 2, 3, and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Gomes in view of Roberts et al., “How Much Knowledge Can You Pack Into the Parameters of a Language Model?” (EMNLP 2020), hereinafter Roberts.
Roberts was published in the Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), before the October 31, 2023 effective filing date of the claimed invention.Regarding claim 2, Gomes teaches claim 1. Gomes does not expressly teach “The information processing system according to claim 1, wherein the plurality of instructions cause the at least one processor to request the for-answer language model to create an answer based on the question, and acquire the answer from the for-answer language model, when it is determined that the question does not relate to the predetermined field.” Specifically, Gomes answers an authorized query from data retrieved from a restricted database via a response template (Gomes, ¶¶ [0027], [0029], [0043]) and does not teach generating an answer from the question itself when the question does not relate to the predetermined field.
Roberts teaches fine-tuning a pre-trained language model to answer questions in their words, “without access to any external context or knowledge” (Roberts, Abstract). Roberts states that, “by feeding the model the input question alone,” the model answers from the knowledge stored in its parameters, a task Roberts terms “closed-book question answering” (Roberts, § 1). Roberts thus teaches creating an answer to a question with the for-answer language model based on the question itself, without any retrieved answer-basis information, which is the limitation Gomes does not teach.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Gomes so that, when it is determined that the question does not relate to the predetermined field, the for-answer language model creates an answer based on the question, as taught by Roberts. Gomes obtains its answers by retrieving data from a restricted database and filling a response template (Gomes, ¶¶ [0027], [0029], [0043]); accordingly, when a question falls outside the predetermined field, no answer-basis information is available in the restricted database and Gomes returns no substantive answer to the user. Roberts demonstrates that a language model can be fine-tuned to answer questions from the knowledge stored in its parameters, with no retrieved context, and that this approach “performs competitively with open-domain systems that explicitly retrieve answers from an external knowledge source” (Roberts, Abstract). Applying Roberts’s closed-book question-answering technique to the for-answer language model of Gomes is the use of a known technique to yield the predictable result of an answer generated from the question alone, and one of ordinary skill would have been motivated to do so in order to enable the system to answer questions that are not represented in the restricted database without adding answer-basis information for those questions to that database.Regarding claim 3, Gomes teaches claim 1, and further teaches determining whether the question relates to a predetermined field and whether the question relates to a field for which an answer is to be refused (Gomes, ¶¶ [0027]-[0028], [0043], [0047]; FIGS. 2 (entitlement and authorization 210), 6; the system classifies the query into a subject/domain and determines whether the user is authorized to receive the answer; a query the user is not authorized to receive, such as the salary query of FIG. 6, is a field for which an answer is to be refused). Gomes does not expressly teach “request, when it is determined that the question does not relate to the predetermined field, and determined that the question does not relate to a field for which an answer is to be refused, the for-answer language model to create an answer based on the question, and acquire the answer from the for-answer language model.” Roberts teaches fine-tuning a pre-trained language model to answer questions, in the words of Roberts, “without access to any external context or knowledge” (Roberts, Abstract), and states that, “by feeding the model the input question alone,” the model answers from the knowledge stored in its parameters, a task Roberts terms “closed-book question answering” (Roberts, § 1). Roberts thus teaches creating an answer to a question with the for-answer language model based on the question itself, without any retrieved answer-basis information, which is the limitation Gomes does not teach for claim 3.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Gomes such that, when the question relates neither to the predetermined field nor to a field for which an answer is to be refused, the for-answer language model creates an answer based on the question, as taught by Roberts. Applying Roberts’s closed-book question-answering technique to the for-answer language model of Gomes is the use of a known technique to yield the predictable result of an answer generated from the question alone, and one of ordinary skill would have been motivated to do so in order to enable the system to answer questions that fall neither in the predetermined field nor in a field for which an answer is to be refused, without adding answer-basis information for those questions to the restricted database of Gomes.
Regarding claim 4, Gomes in view of Roberts renders obvious claim 3. Gomes further teaches “The information processing system according to claim 3, wherein the plurality of instructions cause the at least one processor to send information indicating that the question is not answerable to the user, when it is determined that the question relates to a field for which an answer is to be refused.” (Gomes, ¶ [0047]; FIG. 6, when the user is not authorized to receive the answer, the system generates and provides a response (604) indicating that the user is not authorized to view the information).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Gomes in view of Lee (US 8,301,450 B2), hereinafter Lee.
Regarding claim 6, Gomes teaches claim 5. Gomes does not expressly teach “The information processing system according to claim 5, wherein the plurality of instructions cause the at least one processor to request the for-determination language model to determine whether the question relates to the predetermined field based on the predetermined field and information indicating a topic of the answer basis information corresponding to the question acquired from the database, and to determine whether the question relates to the predetermined field based on information output by the for-determination language model.” Lee teaches determining the topic domain of an input with a for-determination language model. Lee discloses that “The topic-domain-detection module 130 detects a topic domain including a topic of a speaker by inferring the topic based on the meaning of the vocabularies contained in the word lattice,” and that the forward search module, the topic-domain-detection module, and the backward-decoding module perform their functions “with reference to the global language model DB 210,” the “probability factor DB 220” and “the specific topic domain language model DB 230,” respectively (Lee, col. 5, ll. 9-24; col. 7, ll. 16-26; FIGS. 2 (topic-domain-detection module 130, specific topic domain language model DB 230), 3).In particular, Lee discloses that “the topic domain distance calculation module 134 ... calculates the distance from each available topic domain based on the vocabularies contained in the word lattice,” and that “the minimum distance detection module 136 detects the minimum distance relative to the topic domain from among the distances calculated according to the algorithm,” whereby the candidate topic domain having the minimum distance is selected (Lee, col. 5, l. 9 - col. 6, l. 10; col. 6, ll. 31-47; col. 7, ll. 50-67; FIGS. 2, 3). The determination is thereby made based on (i) the candidate topic domains, corresponding to the predetermined field, and (ii) topic information characterizing the contents associated with each domain, corresponding to information indicating a topic of the answer basis information, and is carried out with reference to the global language model database and the specific topic domain language model database - that is, based on information output by a language model (Lee, col. 7, ll. 16-26; col. 8, ll. 52-67). Applying Lee’s topic-domain determination to the field determination of Gomes therefore yields determining whether the question relates to the predetermined field based on the predetermined field and on topic information associated with the contents of the database, and based on information output by the for-determination language model.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to perform the field determination of Gomes based further on topic information associated with the contents of the database, as taught by Lee. Gomes determines whether a question relates to a predetermined field but does not base that determination on topic information characterizing the contents of the field database; Lee determines the topic domain of an input by computing, for each candidate domain, a distance from the vocabularies of the input against per-domain language-model information and selecting the minimum-distance domain. One of ordinary skill would have been motivated to apply Lee’s topic-based determination to Gomes so that the field determination additionally accounts for the topic of the answer basis information, resolving questions whose pertinent field is not evident from surface terms alone and thereby improving the accuracy of the field determination in Gomes.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Gomes in view of Lei et al. (CN 114416927 A), hereinafter Lei.
Regarding claim 8, Gomes teaches claim 1. Gomes does not expressly teach “The information processing system according to claim 1, wherein the database is configured to generate a feature vector from the question, and acquire answer basis information associated with a feature vector similar to the generated feature vector.” Lei teaches generating a semantic vector from the question and retrieving similar results from a knowledge database using that vector. In the English-language machine translation of Lei of record, Lei discloses “inputting the query text information into the semantic classification model, obtaining the target sentence semantic vector, and based on the query text information and the target sentence semantic vector, performing word search and semantic vector search in the preset question and answer knowledge database, obtaining recalling similar result set” (Lei, claim 1; see also steps S20-S23). The target sentence semantic vector corresponds to the feature vector generated from the question, and the recall similar result set obtained by semantic vector search corresponds to answer basis information associated with a feature vector similar to the generated feature vector.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the retrieval of answer basis information in Gomes by generating a feature vector from the question and acquiring answer basis information associated with a similar feature vector, as taught by Lei. Gomes retrieves answer basis information from its database by matching the query, but does not disclose vector-based retrieval; Lei represents the question as a semantic vector and retrieves entries whose semantic vectors are similar, capturing questions that are semantically related but do not share surface terms. One of ordinary skill would have been motivated to make the combination in order to retrieve answer basis information for questions phrased differently from the stored entries, which word-level matching alone would miss, thereby improving the retrieval of relevant answer basis information in Gomes.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
a. Mahajan et al. (CA 3120893 C): mapping natural language utterances associated with a set of topics to a node in a knowledge graph, retrieving a response from a knowledge engine, and transmitting the response to the user.
b. Liu et al. (CN 110162611 A): intelligent customer response employing intent identification, field-scoped retrieval, and word-vector similarity.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUVAL H. LEVENTAL whose telephone number is (571) 270-3130. The examiner can normally be reached Monday-Friday, 8:00 AM - 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PIERRE-LOUIS DESIR, can be reached at (571) 272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YUVAL HAIM LEVENTAL/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Prosecution Insights