DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application and the preliminary amendment filed 1/23/2024. Claims 1-20 are pending and have been examined. Claims 1-20 are rejected.
Priority
The examiner acknowledges the priority benefit to U.S. Provisional Application No. 63/440,534. The present application claims priority to U.S. Provisional Application No. 63/440,534, filed 01/23/2023.
Information Disclosure Statement
Acknowledgment is made of the information disclosure statement filed 6/03/2024, which complies with 37 CFR 1.97. As such, that information disclosure statement has been placed in the application file and the information referred to therein has been considered by the examiner.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(3) because Figures 2-7 include letters which do not measure at least .32 cm. (1/8 inch) in height (see, e.g., most of the lowercase and subscript characters in FIGs. 2-9). See MPEP 507 (A) and 37 CFR 1.84(p)(3): Numbers, letters, and reference characters must measure at least .32 cm. (1/8 inch) in height.
The drawings are also objected to as failing to comply with 37 CFR 1.84(l) because many of the characters in Figures 1-7 (i.e., text/labels in blocks, components, and graphs in FIGs. 1-7, and the legends, labels and values of the x and y axes in FIGs. 3-7) are too light to permit adequate reproduction. See MPEP 507 (A) and 37 CFR 1.84(l): All drawings must be made by a process which will give them satisfactory reproduction characteristics. Every line, number, and letter must be durable, clean, black (except for color drawings), sufficiently dense and dark, and uniformly thick and well-defined. The weight of all lines and letters must be heavy enough to permit adequate reproduction.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
In paragraphs 5-7, 88 and 109-138 in the “Brief Description of the Prior Art”, “DETAILED DESCRIPTION OF THE INVENTION” and “INCORPORATION BY REFERENCE” sections of the specification, several references are referred to (see, e.g., references to non-patent literature “(Yu et al. 2022), (He et al. 2017), (Moon et al. 2019), (Huang et al. 2019).” and “Yang et al. 2015), (Dettmers et al. 2018), (Toutanova et al. 2016) and path-based methods (Das et al. 2017), (Guu, Miller, and Liang 2015), (Lao, Mitchell, and Cohen 2011), (Neelakantan, Roth, and McCallum 2015), (Toutanova et al. 2016), (Yang, Yang, and Cohen 2017), (Rocktӓschel and Riedel 2017).” in paragraph 5, “(Das et al. 2018), (Shen et al. 2018) (Lin, Socher, and Xiong 2018).” in paragraph 6, “(Toutanova et al. 2015)” in paragraph 7, “(Das et al. 2018) and (Shen et al. 2018)” in paragraph 88, and the several references listed in paragraphs 109-138. The listing of references in the specification is not a proper information disclosure statement. 37 CFR 1.98(b) requires a list of all patents, publications, or other information submitted for consideration by the Office, and MPEP § 609.04(a) states, "the list may not be incorporated into the specification but must be submitted in a separate paper." Therefore, unless the references have been cited by the examiner on form PTO-892, they have not been considered. It is noted, however, that applicant appears to have furnished citations to and copies of only some of the references where appropriate (i.e., for the non-patent literature reference “(Toutanova et al. 2015)” referred to in paragraph 7 and some, but not all, of the references listed in the “INCORPORATION BY REFERENCE” section) in the above-referenced information disclosure statement filed 6/03/2024. Appropriate correction is required.
Claim Objections
Claims 2-9 and 11-18 are objected to because of the following informalities:
Claims 2 and 11 recite, using respective similar language, “wherein the step of automatically generating partial labels of a dataset further comprises the step of, determining the start entity and the target entity.” These recitations include an extraneous comma “,” between “of” and “determining”. Appropriate correction is required.
Also, claims 3-9 and 12-18, which each depend directly or indirectly from claims 2 and 11, respectively, are objected to based on their respective dependencies from claims 2 and 11.
Claims 3 and 12 recite, using respective similar language, “wherein the step of automatically generating partial labels of a dataset further comprises the step of, calculating a correct path between the start entity and the target entity.” These recitations include an extraneous comma “,” between “of” and “calculating”. Appropriate correction is required.
Also, claims 4-9 and 13-18, which each depend directly or indirectly from claims 3 and 13, respectively, are objected to based on their respective dependencies from claims 3 and 12.
Claims 6 and 15 recite, using respective similar language, “wherein the step of automatically generating partial labels of a dataset further comprises the step of, generating the partial labels based on the target set.” These recitations include an extraneous comma “,” between “of” and “generating”. Appropriate correction is required.
Also, claims 7-9 and 16-18, which each depend directly or indirectly from claims 6 and 15, respectively, are objected to based on their respective dependencies from claims 6 and 15.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Independent claims 1 and 10 both recite “an optimal reasoning pathway from a start entity to a target entity for a question-and-answer query using the knowledge graph” (see, lines 9-10 of claim 1 and lines 7-8 of claim 10), and independent claim 19 similarly recites “an optimal reasoning pathway from the start entity to the target entity” (see, lines 21-22 of claim 19). The term “optimal reasoning pathway” in claims 1, 10 and 19 is a relative term which renders the claims indefinite. The term “optimal reasoning pathway” is not defined by the claims, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The specification repeats the claim language in stating “an optimal reasoning pathway from a start entity to a target entity for a question-and-answer query using the knowledge graph.” (see, e.g., paragraphs 12, 16 and 21). Therefore, the specification fails to describe or define what is meant by this term. In particular, it is unclear what metrics or standards are used for ascertaining the requisite degree of optimality for the claimed “optimal reasoning pathway.” As such, the specification does not provide a standard for determining the requisite degree of optimality for the claimed “optimal reasoning pathway” in claims 1, 10 and 19. For the purposes of determining patent eligibility and comparison with the prior art, the Examiner is interpreting an “optimal reasoning pathway from a start entity to a target entity for a question-and-answer query using the knowledge graph” as any reasoning pathway from or between a start entity/node/unit to a target entity/node/unit for a question-and-answer query or question using a knowledge graph. Appropriate correction is required.
Claims 3, 12 and 19 recite “calculating a correct path between the start entity and the target entity” (see, lines 2-3 of claims 3 and 12, and line 5 of claim 19). The term “a correct path between the start entity and the target entity” in claims 3, 12 and 19 is a relative term which renders the claims indefinite. The term “a correct path” is not defined by the claims, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The specification repeats the claim language in stating “calculating a correct path between the start entity and the target entity.” and “map each correct path between the start entity and the target entity.” (see, e.g., paragraphs 12, 17-19 and 21) and provides general examples in paragraphs 63, 65 and 81-82. Therefore, the specification fails to describe or define what is meant by this term. In particular, it is unclear what metrics or standards are used for ascertaining the requisite degree of correctness for the claimed “correct path”. As such, the specification does not provide a standard for determining the requisite degree of correctness for the claimed “correct path” in claims 3, 13 and 19. For the purposes of determining patent eligibility and comparison with the prior art, the Examiner is interpreting a “correct path between the start entity and the target entity” as any suitable or usable path or pathway from or between a start entity/node/unit to a target entity/node/unit. Appropriate correction is required.
Claims 4-9, 14-18 and 20, which each depend directly or indirectly from claims 3, 13 and 19, respectively, are rejected under 35 U.S.C. 112(b) as being indefinite under the same rationale as claims 3, 13 and 19.
Claims 7, 8, 16, 17 and 19 each recite “performing a supervised learning module”, “performing the supervised learning module” and “performing a reinforced learning module” (see, line 2 of claim 7, lines 2-3 of claim 8, lines 2-3 of claim 16, lines 2-3 of claim 17 and lines 14 and 17 of claim 19). These recitations are grammatically incorrect, appear to be missing one or more words between “performing” and the articles “a” and “the”, and are unclear. In particular, it is unclear how a module (i.e., a software module/agent/unit/function or a combination of software and/or hardware) is ‘performed’. That is, while a module can be invoked/called or executed, it is unclear how “performing” the recited modules is carried out. The specification merely repeats the claim language in paragraphs 14, 19 and 21 without clarifying or explaining how the recited “performing” of the claimed modules is done. For examination purposes, recitations of “performing” a/the “supervised learning module” are being interpreted as performing the claimed steps/operations via or by a supervised learning/training module, agent, unit or function, and recitations of “performing a reinforced learning module” are being interpreted as performing the claimed steps/operations via or by a reinforced learning/training module, agent, unit or function (see, e.g., the recitations in lines 6-7 of claim 1 and lines 12-13 of claim 10 of “pretraining, via a supervised learning module, a reinforced learning module, or both, a neural network”). Appropriate correction is required.
Claim 20 recites “the at least one processor” in lines 1-2. There is insufficient antecedent basis for this limitation in the claim. No “at least one processor” was previously introduced in this claim, or in its base claim, claim 19. For examination purposes, “the at least one processor” is being interpreted as any processor. Appropriate correction is required.
Claims 17, 18 and 20, which each depend directly or indirectly from claims 16, 17 and 19, respectively, are rejected under 35 U.S.C. 112(b) as being indefinite under the same rationale as claims 17, 18 and 19.
Also, claims 2-9, 11-18 and 20, which each depend directly or indirectly from claims 1, 10 and 19, respectively, are rejected under 35 U.S.C. 112(b) as being indefinite under the same rationale as claims 1, 10 and 19.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis below of the claims’ subject matter eligibility follows the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”) and the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, 89 Fed. Reg. 58128-58138 (July 17, 2024) (“2024 AI SME Update”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Regarding independent claims 1 and 10, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a method, corresponding to a process, claim 10 is directed to a system, corresponding to a machine, which are both one of the statutory categories.
Step 2A Prong One Analysis: The claims are directed to an abstract idea. In particular, the claims recite mental processes that are concepts performed in the human mind (including an observation, evaluation, judgment, opinion).
The claims recite, using respective similar language:
automatically generating … partial labels for a dataset, wherein the partial labels are generated from a subset of the knowledge graph; … and
subsequent to generating partial labels, pretraining the neural network, or both, automatically selecting … an optimal reasoning pathway from a start entity to a target entity for a question-and-answer query using the knowledge graph1.
As drafted, under their broadest reasonable interpretation (BRI), in view of the specification, these limitations cover concepts performed in the human mind (evaluation, judgement, or opinion to generate/create partial labels for data based on observed data values in a subset/portion of a knowledge graph and then evaluation/judgement/opinion to select a reasoning pathway from/between start and target entities for a question-and-answer query/question using/based on observed data values/nodes in the knowledge graph). Given a sufficiently small “dataset” and sufficiently small “subset of the knowledge graph”, nothing in the claims prohibit this process from being performed mentally or with pen and paper.
Regarding the “knowledge graph” and “neural network” limitations, no details of the knowledge graph or neural network are recited and the knowledge graph and neural network are both recited at a high level of generality. Aside from repeating the claim language in paragraphs 11, 16, 21 and 55, and providing general examples in paragraphs 19 and 59, applicant’s specification does not define the “knowledge graph” and “neural network”. Thus, the claimed “knowledge graph” and “neural network”, under the BRI, in light of the specification, could be any knowledge graph including a subset of nodes/data values and any neural network, which could be constructed and updated by hand with pen and paper.
Also, the neural network is recited at a high level of generality and therefore are being interpreted as performing a mental process on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “at least one processor of a computing device” and “neural network” of claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of claim 10), then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claims 1 and 10 recite an abstract idea.
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claims recite these additional elements:
A method of finding reasoning pathways in a knowledge graph, in real-time2, the method comprising the steps of: automatically generating, via at least one processor of a computing device in claim 1,
A path-finding system of finding reasoning pathways in a knowledge graph in claim 10,
via the at least one processor of the computing device, and
via a supervised learning module, a reinforced learning module, or both, a neural network in claims 1 and 10; and
at least one processor; and
a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon that, when executed by the at least one processor, cause a path-finding system to <perform the above noted generating and selecting operations> in claim 10.
The computing device, processor, neural network, computer-readable medium, modules and system are recited at a high level of generality as mere instructions to implement an abstract idea on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The claims also include additional element of pretraining, via a supervised learning module, a reinforced learning module, or both, a neural network. The above-noted additional element in the claims amounts to recitation of the words "apply it" (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer, which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f). In particular, “pretraining, via a supervised learning module, a reinforced learning module, or both, a neural network” is simply generic training to perform the abstract idea of processing received data by learning/training modules (i.e., software modules/units/functions) to create a generically-recited neural network and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Merely asserting that a judicial exception is to be carried out on a generic computer (i.e., with the generically-recited “at least one processor of a computing device”, modules and “neural network” of claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of claim 10) cannot meaningfully integrate the judicial exception into a practical application, and mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. See MPEP § 2106.05(f).
Step 2B Analysis: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of:
“pretraining, via a supervised learning module, a reinforced learning module, or both, a neural network” is generic training to perform the abstract idea and amounts to no more than mere instructions to apply the exception (MPEP 2106.05(f)).
Mere instructions to apply the mental process electronically (i.e., with the generically-recited “at least one processor of a computing device”, modules and “neural network” of claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of claim 10) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method of generating/creating partial labels for data based on observed data values in a subset/portion of a knowledge graph and selecting a reasoning pathway between start and target entities for a question-and-answer query/question using/based on observed data values/nodes in the knowledge graph. Nothing in the claims provide significantly more than this.
The additional elements do not provide an inventive concept, and, therefore, the claims are not patent eligible.
Regarding independent claim 19, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 19 is directed to a method, corresponding to a process, which is one of the statutory categories.
Step 2A Prong One Analysis: The claim is directed to an abstract idea. In particular, the claim recites mental processes that are concepts performed in the human mind (including an observation, evaluation, judgment, opinion).
The claim recites: generating partial labels for a dataset by:
determining a start entity and a target entity;
calculating a correct path between the start entity and the target entity3;
removing, for each correct path, any correct paths that include a self-loop outside of the calculated correct path;
adding to a target set, for each correct path, all parent nodes for each node visited from the start entity to the target entity;
generating the partial labels based on the target set; and
wherein the partial labels are generated from a subset of the knowledge graph; … and
subsequent to generating the partial labels, pretraining the neural network, or both, automatically selecting an optimal reasoning pathway from the start entity to the target entity for a question-and-answer query using the knowledge graph4.
As drafted, under their broadest reasonable interpretation (BRI), in view of the specification, these limitations cover concepts performed in the human mind (evaluation, judgement, or opinion to determine/identify start and target entities, calculate/identify a suitable/usable pathway/path between the start and target entities, remove any of the “correct”/suitable/usable paths that include a self-loop outside of the calculated/identified correct/suitable/usable path, adding to a target set, for each correct suitable/usable path, parent nodes for each node visited from the start entity to the target entity, generate/create partial labels for data based on observed data values in a subset/portion of a knowledge graph and then evaluation/judgement/opinion to select a reasoning pathway from/between start and target entities for a question-and-answer query/question using/based on observed data values/nodes in the knowledge graph). Given a sufficiently small “dataset” and sufficiently small “target set”, set of paths, parent nodes for nodes “visited from the start entity to the target entity” and “subset of the knowledge graph”, nothing in the claim prohibits this process from being performed mentally or with pen and paper.
Regarding the “knowledge graph” and “neural network” limitations, no details of the knowledge graph or neural network are recited and the knowledge graph and neural network are both recited at a high level of generality. Aside from repeating the claim language in paragraphs 11, 16, 21 and 55, and providing general examples in paragraphs 19 and 59, applicant’s specification does not define the “knowledge graph” and “neural network”. Thus, the claimed “knowledge graph” and “neural network”, under the BRI, in light of the specification, could be any knowledge graph including a subset of nodes/data values and any neural network, which could be constructed and updated by hand with pen and paper.
Also, the neural network is recited at a high level of generality and therefore are being interpreted as performing a mental process on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “neural network”), then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claim 19 recites an abstract idea.
Step 2A Prong Two Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites these additional elements:
A method of automatically finding reasoning pathways in a knowledge graph, the method comprising: …
pretraining a neural network by:
performing a supervised learning module5 using the generated partial labels by sampling, via an agent, a policy action on the target set to map each correct path between the start entity and the target entity; and
after performing the supervised learning module, performing a reinforced learning module6 using the sampled policy action by applying the sampled policy action to the dataset to maximize a reward for the agent.
The above-noted additional elements in the claim amount to recitation of the words "apply it" (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer, which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f). In particular, “pretraining a neural network by: performing a supervised learning module using the generated partial labels by sampling, via an agent, a policy action” and “performing a reinforced learning module using the sampled policy action by applying the sampled policy action to the dataset to maximize a reward for the agent” is simply generic training to perform the abstract idea of processing received data by learning/training modules (i.e., software modules/units/functions) to create a generically-recited neural network and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Merely asserting that a judicial exception is to be carried out on a generic computer (i.e., with the generically-recited modules, agent and “neural network” of claim 19 cannot meaningfully integrate the judicial exception into a practical application, and mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. See MPEP § 2106.05(f).
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of:
“pretraining a neural network by: performing a supervised learning module using the generated partial labels by sampling, via an agent, a policy action on the target set to map each correct path between the start entity and the target entity; and
after performing the supervised learning module, performing a reinforced learning module using the sampled policy action by applying the sampled policy action to the dataset to maximize a reward for the agent” is generic training to perform the abstract idea and amount to no more than mere instructions to apply the exception (MPEP 2106.05(f)).
Mere instructions to apply the mental process electronically (i.e., with the generically-recited modules, agent and “neural network” of claim 19) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claim is directed to a method of determining/identifying start and target entities, calculating/identifying a suitable/usable pathway/path between the start and target entities, removing any of the “correct”/suitable/usable paths that include a self-loop outside of the calculated/identified correct/suitable/usable path, adding to a target set, for each correct suitable/usable path, parent nodes for each node visited from the start entity to the target entity, generating/creating partial labels for data based on observed data values in a subset/portion of a knowledge graph and then selecting a reasoning pathway from/between start and target entities for a question-and-answer query/question using/based on observed data values/nodes in the knowledge graph. Nothing in the claim provides significantly more than this.
The additional elements do not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claims 2 and 11, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to method as depending from claim 1 and claim 11 is directed to a system as depending from claim 10, thus the analysis for patent eligibilities of claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, wherein the step of automatically generating partial labels of a dataset further comprises the step of, determining the start entity and the target entity. Under its BRI, in light of the specification, the determining limitation encompasses the mental processes of determining/identifying the start and target entities which is an act of evaluation (i.e., evaluation, judgment, opinion to determine/identify start and target entities based on observation of data in the dataset and nodes in the knowledge graph). This limitation does nothing to alter the fundamental nature of the claims as a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “at least one processor of a computing device” and “neural network” of base claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claims 2 and 11 both recite an abstract idea.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The claims do not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claims are subject-matter ineligible.
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
These claims are not patent eligible.
Regarding claims 3 and 12, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to method as depending from claim 2 and claim 12 is directed to a system as depending from claim 11, thus the analysis for patent eligibilities of claims 2 and 11, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, wherein the step of automatically generating partial labels of a dataset further comprises the step of, calculating a correct path between the start entity and the target entity7. Under its BRI, in light of the specification, the calculating limitation encompasses the mental process of calculating/identifying a suitable/usable pathway/path between the start and target entities which is an act of evaluation (i.e., evaluation, judgment, opinion to identify a correct/suitable path based on observed nodes of the knowledge graph). This limitation does nothing to alter the fundamental nature of the claims as a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “at least one processor of a computing device” and “neural network” of base claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claims 3 and 12 both recite an abstract idea.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The claims do not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claims are subject-matter ineligible.
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
These claims are not patent eligible.
Regarding claims 4 and 13, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to method as depending from claim 3 and claim 13 is directed to a system as depending from claim 12, thus the analysis for patent eligibilities of claims 3 and 12, intervening claims 2 and 11, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, wherein the step of automatically generating partial labels of a dataset further comprises the step of, removing, for each correct path, any correct paths that include a self-loop outside of the calculated correct path. Under its BRI, in light of the specification, the removing limitation encompasses the mental process of removing any of the “correct”/suitable/usable paths that include a self-loop outside of the calculated/identified correct/suitable/usable path which is an act of evaluation (i.e., evaluation, judgment, opinion to remove “correct”/suitable/usable paths having a self-loop outside of the calculated/identified correct/suitable/usable path). This limitation does nothing to alter the fundamental nature of the claims as a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “at least one processor of a computing device” and “neural network” of base claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claims 4 and 13 both recite an abstract idea.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The claims do not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claims are subject-matter ineligible.
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
These claims are not patent eligible.
Regarding claims 5 and 14, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to method as depending from claim 4 and claim 14 is directed to a system as depending from claim 13, thus the analysis for patent eligibilities of claims 4 and 13, intervening claims 2-3 and 11-12, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, wherein the step of automatically generating partial labels of a dataset further comprises the step of, adding to a target set, for each correct path, all parent nodes for each node visited from the start entity to the target entity. Under its BRI, in light of the specification, the adding limitation encompasses the mental process of adding to a target set, for each correct suitable/usable path, parent nodes for each node visited from the start entity to the target entity, which is an act of evaluation (i.e., evaluation, judgment, opinion to selectively add, to the target set, parent nodes for each node visited from the start entity to the target entity for each correct suitable/usable path). This limitation does nothing to alter the fundamental nature of the claims as a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “at least one processor of a computing device” and “neural network” of base claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claims 5 and 14 both recite an abstract idea.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The claims do not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claims are subject-matter ineligible.
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
These claims are not patent eligible.
Regarding claims 6 and 15, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to method as depending from claim 5 and claim 15 is directed to a system as depending from claim 14, thus the analysis for patent eligibilities of claims 5 and 14, intervening claims 2-4 and 11-13, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, wherein the step of automatically generating partial labels of a dataset further comprises the step of, generating the partial labels based on the target set. Under its BRI, in light of the specification, the generating limitation encompasses the mental process of generating/creating partial labels for data based on observed data values in the dataset and the target set, which is an act of evaluation (i.e., evaluation, judgment, opinion to generate/create partial labels for data based on observed data values in the dataset and target set). This limitation does nothing to alter the fundamental nature of the claims as a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “at least one processor of a computing device” and “neural network” of base claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claims 6 and 15 both recite an abstract idea.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
The claims do not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claims are subject-matter ineligible.
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
These claims are not patent eligible.
Regarding claims 7 and 16, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to method as depending from claim 6 and claim 16 is directed to a system as depending from claim 15, thus the analysis for patent eligibilities of claims 6 and 15, intervening claims 2-5 and 11-14, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, wherein the step of pretraining a neural network further comprises the step of, performing a supervised learning module8 using the generated partial labels by sampling, via an agent, a policy action on the target set to map each correct path between the start entity and the target entity. Under its BRI, in light of the specification, the sampling limitation encompasses the mental process of sampling an observed target set and mapping/correlating correct/suitable/usable paths between start and target entities based on observed data values in policy actions and the target set, which is an act of evaluation (i.e., evaluation, judgment, opinion to sample a target set and map/correlate correct/suitable paths between start and target entities based on observed data values in policy actions and the target set). These limitations do nothing to alter the fundamental nature of the claims as a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “agent”, “learning module”, “at least one processor of a computing device” and “neural network” of base claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claims 7 and 16 both recite an abstract idea.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite these additional elements:
performing a supervised learning module using the generated partial labels by sampling, via an agent.
The above-noted additional elements in the claims amount to recitation of the words "apply it" (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer, which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Merely asserting that a judicial exception is to be carried out on a generic computer (i.e., with the generically-recited “learning module” and agent) cannot meaningfully integrate the judicial exception into a practical application, and mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. See MPEP § 2106.05(f).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to the integration of the abstract idea into a practical application, mere instructions to apply the mental process electronically (i.e., with the generically-recited “learning module” and agent) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
These claims are not patent eligible.
Regarding claims 8 and 17, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to method as depending from claim 7 and claim 17 is directed to a system as depending from claim 16, thus the analysis for patent eligibilities of claims 7 and 16, intervening claims 2-6 and 11-15, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: The claims recite, using respective similar language, wherein the step of pertaining a neural network further comprises the step of, subsequent to performing the supervised learning module, performing a reinforced learning module using9 the sampled policy action by applying the sampled policy action to the dataset to maximize a reward for the agent. Under its BRI, in light of the specification, the applying limitation encompasses the mental process of applying a policy action to the dataset to maximize a reward based on an observed sampled policy action and observed values in the dataset, which is an act of evaluation (i.e., evaluation, judgment, opinion to apply a policy action to the dataset in order to maximize a reward based on a sampled policy action and values in the dataset). These limitations do nothing to alter the fundamental nature of the claims as a mental process.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., the “agent”, “learning module”, “at least one processor of a computing device” and “neural network” of base claims 1 and 10 and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) then they fall within the “Mental Processes” grouping of abstract ideas. Accordingly, claims 8 and 17 both recite an abstract idea.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite these additional elements:
performing a reinforced learning module using the sampled policy action by applying the sampled policy.
The above-noted additional elements in the claims amount to recitation of the words "apply it" (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer, which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Merely asserting that a judicial exception is to be carried out on a generic computer (i.e., with the generically-recited “learning module”) cannot meaningfully integrate the judicial exception into a practical application, and mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. See MPEP § 2106.05(f).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to the integration of the abstract idea into a practical application, mere instructions to apply the mental process electronically (i.e., with the generically-recited “learning module”) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
These claims are not patent eligible.
Regarding claims 9, 18 and 20, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claims 9 and 20 are directed to methods as depending from claims 8 and 19, respectively, and claim 18 is directed to a system as depending from claim 17, thus the analysis for patent eligibilities of claims 8, 17 and 19, intervening claims 2-7 and 11-16, and of base claims 1 and 10 are incorporated herein.
Step 2A Prong 1: See, claims 8, 17 and 19.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application.
In particular, the claims recite these additional elements:
retraining, via the at least one processor, the supervised learning module, the reinforced learning module or both, wherein the supervised learning module, the reinforced learning module or both is trained by minimizing a distance between the sampled policy action and the partial labels.
The above-noted additional elements in the claims amount to recitation of the words "apply it" (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer, which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f). In particular, “retraining, via the at least one processor, the supervised learning module, the reinforced learning module or both, wherein the supervised learning module, the reinforced learning module or both is trained by minimizing a distance between the sampled policy action and the partial labels” is simply generic training to perform the abstract idea of processing received data by learning/training modules (i.e., software modules/units/functions) to create a generically-recited neural network and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Merely asserting that a judicial exception is to be carried out on a generic computer (i.e., with the generically-recited “the at least one processor”, learning modules and the “neural network” of base claims 1, 10 and 19, and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) cannot meaningfully integrate the judicial exception into a practical application, and mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. See MPEP § 2106.05(f).
Step 2B Analysis: The claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of:
“retraining, via the at least one processor, the supervised learning module, the reinforced learning module or both, wherein the supervised learning module, the reinforced learning module or both is trained by minimizing a distance between the sampled policy action and the partial labels” is generic training to perform the abstract idea and amounts to no more than mere instructions to apply the exception (MPEP 2106.05(f)).
Mere instructions to apply the mental process electronically (i.e., with the generically-recited “the at least one processor”, learning modules and the “neural network” of base claims 1, 10 and 19, and “a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon” of base claim 10) do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. See MPEP § 2106.05(f).
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
These claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3 and 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over non-patent literature Das, et al. ("Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning." arXiv preprint arXiv:1711.05851 v2 (2018), cited in applicant’s IDS dated 6/03/2024, hereinafter “Das”) in view of Chatterjee et al. (U.S. Patent Application Pub. No. 2021/0097140 A1, hereinafter “Chatterjee”).
With respect to claim 1, Das discloses the invention as claimed including a method of finding reasoning pathways in a knowledge graph, in real-time10 (see, e.g., pages 1-2, Sect. 1, “Our goal is to automatically learn such reasoning paths in KBs. We frame the learning problem as one of query answering, that is to say, answering questions”, “the reasoning paths found by our agent automatically form an interpretable provenance for its predictions.”), the method comprising the steps of:
automatically generating, … partial labels for a dataset, wherein the partial labels are generated from a subset of the knowledge graph (see, e.g., page 2, Sects. 1-2, “Given a massive knowledge graph, we learn a policy, which, given the query (entity1; relation;?), starts from entity1 and learns to walk to the answer node by choosing to take a labeled relation edge at each step, conditioning on the query relation and entire path history”, “From the KB, a knowledge graph G can be constructed where the entities e1;e2 are represented as the nodes and relation r as labeled edge between them.” [i.e., generate partial labels for relation edges from a subset of the entire/massive knowledge graph]);
pretraining, via a supervised learning module, a reinforced learning module, or both, a neural network (see, e.g., pages 2-3, Sects. 1-2, “We present agent MINERVA, which learns to do query answering by walking on a knowledge graph conditioned on an input query, stopping when it reaches the answer node. The agent is trained using reinforcement learning”, “Using a training set of known facts, we train the agent using policy gradients more specifically by REINFORCE” and pages 5-6, Sect. 3.1, “We used the implementation or the best pre-trained models”, “we initialized MINERVA with pre-trained embeddings” [i.e., pretraining via supervised learning, a reinforcement/reinforced learning agent/module]); and
subsequent to generating partial labels, pretraining the neural network, or both, automatically selecting, … an optimal reasoning pathway from a start entity to a target entity for a question-and-answer query using the knowledge graph11 (see, e.g., pages 1-2, Sects. 1-2, “Given a massive knowledge graph, we learn a policy, which, given the query (entity1; relation;?), starts from entity1 and learns to walk to the answer node by choosing to take a labeled relation edge at each step”, “automatically learn such reasoning paths in KBs. We frame the learning problem as one of query answering”, “the reasoning paths found by our agent automatically form an interpretable provenance for its predictions.”, “formulates the query-answering task as a reinforcement learning (RL) problem where the goal is to take an optimal sequence of decisions (choices of relation edges) to maximize the expected reward (reaching the correct answer node)”, “a knowledge graph is a directed labeled multigraph G” and page 3, Sect. 2.1, “an agent at each state has option to select which outgoing edge it wishes to take having the knowledge of the label of the edge r and destination vertex v.” [i.e., after partial labelling or training, select an optimal reasoning path from a start entity/node to a target/answer entity/node]).
Although Das substantially discloses the claimed invention, Das is not relied on to explicitly disclose via at least one processor of a computing device, and via the at least one processor of the computing device.
In the same field, analogous art Chatterjee teaches via at least one processor of a computing device, and via the at least one processor of the computing device (see, e.g., paragraphs 7-8, “a system for generating a graph representing information about a group of related conversations, and includes a processor”, “a system that includes one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to receive a set of conversations”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Chatterjee to provide a “virtual agent [that] can include provisions for learning to converse with an end user in a natural manner. For example, virtual agent 100 may include a reinforcement learning module 210” where a “dialogue management system 114, which may be trained using reinforcement learning processes”, “where training data is collected by extracting the first few utterances of the customer and an intent label is used as an end class to build a mapping function. This step is called label propagation, since partially labelled data gives an estimate of different classes, this set may be used for training a mapping function from conversation to intent.” (See, e.g., Chatterjee, paragraphs 37 and 79). Doing so would have allowed Li to use Chatterjee’s virtual agent and partially labelled data for “automated generation of a conversation graph” where “the conversation graph can improve the performance of virtual and human agents in subsequent real- time conversations with customers, as well as inform the processing of other functions, such as automated conversations and process tree generation” and the “conversation graph can be also used to improve and/or facilitate conversation analysis.”, as suggested by Chatterjee (See, e.g., Chatterjee, paragraphs 5 and 103).
With respect to independent claim 10, claim 10 is substantially similar to claim 1 and therefore is rejected on the same ground as claim 1, discussed above. In particular, claim 10 is directed to a system with operations that largely correspond to the method steps of claim 1.
Although Li substantially discloses the claimed invention, Li is not relied on to explicitly disclose at least one processor; and a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon that, when executed by the at least one processor, cause a path-finding system to [perform operations].
In the same field, analogous art Chatterjee teaches at least one processor; and a non-transitory computer-readable medium operably coupled to the at least one processor, the computer-readable medium having computer-readable instructions stored thereon that, when executed by the at least one processor, cause a path-finding system to [perform operations] (see, e.g., paragraphs 7-8, “a system for generating a graph representing information about a group of related conversations, and includes a processor … the instructions cause the processor to generate a conversation graph for the first subset of conversations, where the conversation graph includes a plurality of nodes interconnected by a series of transitional paths.”, “a system that includes one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to receive a set of conversations” [i.e., a processor and a computer readable medium having executable instructions to cause the path generating/finding system to perform operations]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Chatterjee to provide a “virtual agent [that] can include provisions for learning to converse with an end user in a natural manner. For example, virtual agent 100 may include a reinforcement learning module 210” where a “dialogue management system 114, which may be trained using reinforcement learning processes”, “where training data is collected by extracting the first few utterances of the customer and an intent label is used as an end class to build a mapping function. This step is called label propagation, since partially labelled data gives an estimate of different classes, this set may be used for training a mapping function from conversation to intent.” (See, e.g., Chatterjee, paragraphs 37 and 79). Doing so would have allowed Li to use Chatterjee’s virtual agent and partially labelled data for “automated generation of a conversation graph” where “the conversation graph can improve the performance of virtual and human agents in subsequent real- time conversations with customers, as well as inform the processing of other functions, such as automated conversations and process tree generation” and the “conversation graph can be also used to improve and/or facilitate conversation analysis.”, as suggested by Chatterjee (See, e.g., Chatterjee, paragraphs 5 and 103).
Regarding claims 2 and 11, as discussed above, Li in view Chatterjee teaches the method of claim 1 and the system of claim 10.
Li further discloses wherein the step of automatically generating partial labels of a dataset further comprises the step of, determining the start entity and the target entity (see, e.g., page 1, Sect. 1, “Given a massive knowledge graph, we learn a policy, which, given the query (entity1; relation;?), starts from entity1 and learns to walk to the answer node by choosing to take a labeled relation edge at each step, conditioning on the query relation and entire path history. This formulates the query-answering task as a reinforcement learning (RL) problem where the goal is to take an optimal sequence of decisions (choices of relation edges) to maximize the expected reward (reaching the correct answer node).” and pages 7-8, Sects. 3.1.2 and 3.2.1, “MINERVA outputs the end points of the paths as target entities, it is sometimes possible that the particular target entity of the triple does not have a path from the source entity (however there are paths to other ‘correct’ answer entities).” and “starting from a source entity … We design a baseline model … Starting from ‘e1’” [i.e., generating labels of a dataset includes determining the start entity/entity 1/e1 and the target entity/answer node]).
Regarding claims 3 and 12, as discussed above, Li in view Chatterjee teaches the method of claim 2 and the system of claim 11.
Li further discloses wherein the step of automatically generating partial labels of a dataset further comprises the step of, calculating a correct path between the start entity and the target entity12 (see, e.g., pages 2-3, Sects. 1-2, “Given a massive knowledge graph, we learn a policy, which, given the query (entity1; relation;?), starts from entity1 and learns to walk to the answer node by choosing to take a labeled relation edge at each step, conditioning on the query relation and entire path history. This formulates the query-answering task as a reinforcement learning (RL) problem where the goal is to take an optimal sequence of decisions (choices of relation edges) to maximize the expected reward (reaching the correct answer node). We call the RL agent MINERVA”, “we present a query answering model, that learns to efficiently traverse the knowledge graph to find the correct answer to a query, eliminating the need to evaluate all entities.” and page 7, Sect. 3.1.2, “MINERVA outputs the end points of the paths as target entities, it is sometimes possible that the particular target entity of the triple does not have a path from the source entity (however there are paths to other ‘correct’ answer entities).” [i.e., determining/calculating a correct path between the start entity/entity 1 and the target entity/correct answer node]).
Conclusion
The prior art made of record, listed on form PTO-892, and not relied upon, is considered pertinent to applicant's disclosure.
The references listed on form PTO-892 are all generally related to techniques, methods and systems for building and using knowledge graphs for applications such as question-and-answer systems or query/question answering (QA).
For example, non-patent literature Zhang et al. ("Learning to walk with dual agents for knowledge graph reasoning." Proceedings of the AAAI Conference on artificial intelligence. Vol. 36. No. 5. 2022, hereinafter “Zhang”) discloses “we propose a dual-agent reinforcement learning framework, which trains two agents (GIANT and DWARF) to walk over a KG jointly and search for the answer collaboratively. Our approach tackles the reasoning challenge in long paths by assigning one of the agents (GIANT) searching on cluster-level paths quickly and providing stage-wise hints for another agent (DWARF). Finally, experimental results on several KG reasoning benchmarks show that our approach can search answers more accurately and efficiently, and outperforms existing RL-based methods for long path queries by a large margin.” (see, Abstract).
Also, for example, non-patent literature Li et al. ("Path reasoning over knowledge graph: A multi-agent and reinforcement learning based method." 2018 IEEE International conference on data mining workshops (ICDMW). IEEE, 2018, hereinafter “Li”) discloses “Relation reasoning over knowledge graphs is an important research problem in the fields of knowledge engineering and artificial intelligence, because of its extensive applications (e.g., knowledge graph completion and question answering). Recently, reinforcement learning has been successfully applied to multi-hop relation reasoning (i.e., path reasoning).” and “we propose a Multi-Agent and Reinforcement Learning based method for Path Reasoning, thus called MARLPaR, where two agents are employed to carry out relation selection and entity selection, respectively, in an iterative manner, so as to implement complex path reasoning.” (see, Abstract).
The examiner requests, in response to this office action, support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line no(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the reference cited or the objections made. He or she must also show how the amendments avoid such references or objections See 37 CFR 1.111 (c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RANDY K BALDWIN whose telephone number is (571)270-5222. The examiner can normally be reached on Mon - Fri 9:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RANDALL K. BALDWIN/Primary Examiner, Art Unit 2125
1 As indicated in the section 112(b) rejections of these claims above, an “optimal reasoning pathway from a start entity to a target entity for a question-and-answer query using the knowledge graph” has been interpreted any reasoning pathway from or between a start entity/node/unit to a target entity/node/unit for a question-and-answer query or question using a knowledge graph.
2 The preamble of the claim includes intended use language with no patentable weight (e.g., “in real-time”). The claim and its dependent claims do not positively recite any real-time processing, steps or operations.
3 As indicated above in the section 112(b) rejection of this claim, “a correct path between the start entity and the target entity” has been interpreted any suitable or usable path or pathway from or between a start entity/node/unit to a target entity/node/unit.
4 As indicated in the section 112(b) rejection of this claim above, an “optimal reasoning pathway from the start entity to the target entity for a question-and-answer query using the knowledge graph” has been interpreted any reasoning pathway from or between a start entity/node/unit to a target entity/node/unit for a question-and-answer query or question using a knowledge graph.
5 As indicated in the section 112(b) rejection of this claim above, “performing” a “supervised learning module” has been interpreted as performing the claimed steps/operations via or by a supervised learning/training module, agent, unit or function
6 As indicated in the section 112(b) rejection of this claim above “performing a reinforced learning module” has been interpreted as performing the claimed steps/operations via or by a reinforced learning/training module, agent, unit or function
7 As indicated above in the section 112(b) rejection of these claims, “a correct path between the start entity and the target entity” has been interpreted any suitable or usable path or pathway from or between a start entity/node/unit to a target entity/node/unit.
8 As indicated in the section 112(b) rejections of these claims above, “performing” a “supervised learning module” has been interpreted as performing the claimed steps/operations via or by a supervised learning/training module, agent unit or function
9 As indicated in the section 112(b) rejections of these claims above, “performing” a “reinforced learning module” has been interpreted as performing the claimed steps/operations via or by a reinforcement/reinforced learning/training module, agent, unit or function
10 The preamble of the claim includes intended use language with no patentable weight (e.g., “in real-time”). The claim and its dependent claims do not positively recite any real-time processing, steps or operations.
11 As indicated in the section 112(b) rejections of this claims above, an “optimal reasoning pathway from a start entity to a target entity for a question-and-answer query using the knowledge graph” has been interpreted any reasoning pathway from or between a start entity/node/unit to a target entity/node/unit for a question-and-answer query or question using a knowledge graph.
12 As indicated above in the section 112(b) rejection of these claims, “a correct path between the start entity and the target entity” has been interpreted any suitable or usable path or pathway from or between a start entity/node/unit to a target entity/node/unit.
Prosecution Insights