Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is sent in response to Applicant’s communication received on 01/28/2025 for the application number 19039474. The office hereby acknowledges receipt of the following placed of record in the file: Specification, Abstract, Oath/Declaration and claims.
Status of the claims
Claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/28/2025 was filed before the mailing date of the first office action. These submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because as explained below.
Claim 1 recites a method for modeling a portion of a communication network, comprising:
obtaining data concerning a configuration of a network,
passing the data to a KG (knowledge graph) creation engine;
by the KG creation engine, using a KG creation LLM (large language model) to extract knowledge from the data;
generating, by the KG creation LLM using the knowledge, a portion of a KG that represents a portion of the network;
by a judge LLM, evaluating the portion of the KG to determine an extent to which the portion of the KG accurately represents the portion of the network;
updating a ground truth KG with the extracted knowledge when the portion of the KG has a confidence score that exceeds a threshold confidence score.
Step (a) comprises a mental process. This step can be performed by a human as a person
can obtain data concerning the configuration of a network.
Step (b) comprises a mental process. This step can be performed by a human as a person can provide the obtained network configuration data and extract knowledge from the data.
Step (c) comprises a mental process. This step can be performed by a human as a person can review network configuration fata and extract knowledge from the data
Step (d) comprises a mental process. This step can be performed by a human as a person can organize the extracted knowledge into a structured representation of a portion of a network, such as a knowledge graph.
Step (e) comprises a mental process. This step can be performed by a human as a person can evaluate the knowledge contained in the knowledge graph and determine the extent to which the knowledge graph accurate represents the corresponding portion of the network.
Step (f) comprises a mental process. This step can be performed by a human as a person can compare the determined confidence score with a threshold and, when the score exceeds the threshold, add the extracted knowledge to a ground-truth knowledge graph.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any
statutory category. See MPEP 2106.03. The claim recites at least method. Thus, the claim is a process, which is one of the statutory categories of invention. (Step 1: YES). Step 2A, Prong
One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception.
As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the
judicial exception is “set forth” or “described” in the claim. As discussed above, the broadest
reasonable interpretation of steps (a)-(f) recites a mental process.
Specifically, step (a) can be performed by a human as a person can obtain data concerning the configuration of a network.
Step (b) can be performed by a human as a person can provide the obtained network configuration data and extract knowledge from the data.
Step (c) can be performed by a human as a person can review network configuration fata and extract knowledge from the data
Step (d) can be performed by a human as a person can organize the extracted knowledge into a structured representation of a portion of a network, such as a knowledge graph.
Step (e) can be performed by a human as a person can evaluate the knowledge contained in the knowledge graph and determine the extent to which the knowledge graph accurate represents the corresponding portion of the network.
Step (f) can be performed by a human as a person can compare the determined confidence score with a threshold and, when the score exceeds the threshold, add the extracted knowledge to a ground-truth knowledge graph.
Hence the claim encompasses mental processes practically performed in the human mind
by observation, evaluation, judgement, and opinion. See MPEP 2106.04(a)(2), subsection III.
(Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a
whole integrates the recited judicial exception into a practical application of the exception or
whether the claim is “directed to” the judicial exception. This evaluation is performed by (1)
identifying whether there are any additional elements recited in the claim beyond the judicial
exception, and (2) evaluating those additional elements individually and in combination to
determine whether the claim as a whole integrates the exception into a practical application. See
MPEP 2106.04(d).
The claim recited additional elements including a KG creation engine, a KG creation LLM, a judge LLM, a ground-truth KG, and a confidence score. However, these elements are recited at a high level of generality and perform generic computer functions, such as receiving data, processing data, generating content, and providing output. The use of these elements to obtain network configuration data, extract knowledge from the data, generate a knowledge graph, evaluate the accuracy of the knowledge graph, and update the ground truth knowledge graph merely automates the mental processes described above using generic computer components. Such implementation does not impose any meaningful limit on the judicial exception. Accordingly, these elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. (Step 2A, Prong Two: NO), and the claim is directed to the judicial exception. (Step 2A: YES)
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole
amounts to significantly more than the recited exception i.e., whether any additional element, or
combination of additional elements, adds an inventive concept to the claim. As discussed with
respect to Step 2A, Prong Two, a KG creation engine, a KG creation LLM, a judge LLM, a ground-truth KG, and a confidence score comprise additional elements that perform well-understood, routine, and conventional activities in the field such as receiving data, processing data, generating content, and providing output. See MPEP 2106.05(g). As known in the art these elements are well understood, routine, and conventional functions of a computing device. Even when considered in combination these additional elements merely implement the abstract idea using generic computer components and perform insignificant extra - solutional activity, which does not provide an inventive concept. The claim is not patent eligible.
Claim 2 recites a mental process as a human can identify the communication network as a radio access network.
Claim 3 recites a mental process as a human can identify entities in the network and relationships between the entities.
Claim 4 recites a mental process as a human can generate a confidence assessment regarding the accuracy of information.
Claim 5 recites a mental process as a human can use a confidence assessment to indicate the extent to which information accurately represents a network.
Claim 6 recites a mental process as a human can infer information having insufficient confidence to an expert for further evaluation.
Claim 7 recites a mental process as a human can coordinate the insertion of extracted knowledge into an existing ground-truth knowledge graph.
Claim 8 recites a mental process as a human can organize entities as nodes and relationships between the entities as edges connecting the nodes.
Claim 9 recites a mental process as a human can compare generated knowledge with corresponding ground-truth knowledge to evaluate its accuracy.
Claim 10 recites a mental process as a human can receive the result of an expert’s evaluation of low confidence information.
Claim 11 recites substantially the same limitations as claim 1. Accordingly, it is directed to the same abstract idea.
Claim 12 recites substantially the same limitations as claim 2. Accordingly, it is directed to the same abstract idea.
Claim 13 recites substantially the same limitations as claim 3. Accordingly, it is directed to the same abstract idea.
Claim 14 recites substantially the same limitations as claim 4. Accordingly, it is directed to the same abstract idea.
Claim 15 recites substantially the same limitations as claim 5. Accordingly, it is directed to the same abstract idea.
Claim 16 recites substantially the same limitations as claim 6. Accordingly, it is directed to the same abstract idea.
Claim 17 recites substantially the same limitations as claim 7. Accordingly, it is directed to the same abstract idea.
Claim 18 recites substantially the same limitations as claim 8. Accordingly, it is directed to the same abstract idea.
Claim 19 recites substantially the same limitations as claim 9. Accordingly, it is directed to the same abstract idea.
Claim 20 recites substantially the same limitations as claim 10. Accordingly, it is directed to the same abstract idea.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 15, 16, 17, 18, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ulasen et al. (US 20260187105 A1) in view of Duan et al. (CN 117033668 A) and further in view of Zhdanov (US 11501210 B1).
Regarding claim 1, Ulasen teaches obtaining data concerning a configuration of a network (Para 0037, “In some aspects, the knowledge graph module 110 may correspond to a computing device 101 or cloud network”, wherein a cloud network can comprise a configuration of a network), and passing the data to a KG (knowledge graph) creation engine (Para 0041, “The computing device may also execute the KG update MLM agent 114 prepared to analyze the LLM chat history 106, knowledge graph 108 database, and the graph schema”, wherein the KG update MLM agent is analogous to the KG creation engine); generating, by the KG creation LLM using the knowledge, a portion of a KG that represents a portion of the network (Para 0042, “the outputs may correspond to identification of missing data (e.g., nodes or properties) and/or identification of missing relationships (e.g., edges) in the knowledge graph 108” and para 0043, “the MLM outputs … may be combined to propose updates to the knowledge graph 108.”); updating the ground truth KG with the extracted KG along with a certain confidence (Para 0044, “proposing knowledge graph updates with a confidence score for evaluation.”).
Ulasen does not teach by a judge LLM, evaluating the portion of the KG to determine an extent to which the portion of the KG accurately represents the portion of the network.
However, Duan teaches by a judge LLM, evaluating the portion of the KG to determine an extent to which the portion of the KG accurately represents the portion of the network (Pg. 19, “each knowledge in the target knowledge map can be evaluated by a second large language model, and the accuracy of the target knowledge map can be given according to the evaluation result of all the knowledge” where the term knowledge map is being used as analogous to a knowledge graph as shown on Pg. 10, “In general, the knowledge graph exists in the form of graph data, including a number of nodes and edges between the nodes. The knowledge map may include entities and relationships between the entities, wherein each node represents the entity, and the connection between the node represents the relationship between the entities.”).
It would have been obvious to one of ordinary skill in the art before the effective filing data to modify Ulasen, incorporating the teachings of Duan in order to improve the accuracy and reliability of evaluating proposed updates (Pg. 19-20).
Ulasen modified by Duan does not teach proceeding only when the confidence score exceeds a threshold confidence score.
However, Zhdanov teaches proceeding only when the confidence score exceeds a threshold confidence score. (Col. 29, Ln. 37-39, “At 706, if the process 700 determines that the first confidence 704 is greater than the first threshold, the process 700 may follow the “YES” route and proceed” where when the confidence is over a certain threshold, the process continues without review).
It would have been obvious to one of ordinary skill in the art before the effective filing data to modify Ulasen, incorporating the teachings of Zhdanov in order to automatically proceed with sufficiently reliable KG updates while reserving uncertain updates for further review. (Col. 29).
Regarding claim 3, Ulasen teaches wherein the knowledge comprises information about entities in the network, and relationships between the entities. (Para 0045, “may also execute the optional component module 116 to extract nodes of a first type of node in the graph schema of nodes and relationships between the nodes and a second type of node in the graph schema of nodes and relationships between the nodes”).
Regarding claim 4, Ulasen does not teach wherein the confidence score is generated by the judge LLM.
However, Duan teaches wherein the confidence score is generated by the judge LLM (Pg. 19-20, “inputting the target knowledge map of the target field into the adjusted second large language model, … judging whether each knowledge contained in the type of knowledge is correct, and according to the judging result of each knowledge, determining the accuracy of the knowledge in the target knowledge map”, where said accuracy is a numerical value as seen later by the calculations involving it on Pg. 20, “the average value of the accuracy”. The numerical accuracy which Duan determines is analogous to a confidence score and is generated by an LLM).
It would have been obvious to one of ordinary skill in the art before the effective filing data to modify Ulasen, incorporating the teachings of Duan in order to provide an automated evaluation of the correctness of generated KG knowledge. (Pg. 20).
Regarding claim 5, Ulasen does not teach wherein the confidence score indicates the extent to which the portion of the KG accurately represents the portion of the network.
However, Duan teaches wherein the confidence score indicates the extent to which the portion of the KG accurately represents the portion of the network. (Pg. 20, “the average value of the accuracy of each kind of knowledge can be used as the final evaluation result of the target knowledge map to reflect the whole quality of the target knowledge map.”).
It would have been obvious to one of ordinary skill in the art before the effective filing data to modify Ulasen, incorporating the teachings of Duan in order to quantify the overall quality and correctness of the generated portion. (Pg. 20).
Regarding claim 6, Ulasen teaches updating the knowledge graph based on a confidence score. (Para 0044, “proposing knowledge graph updates with a confidence score for evaluation.”).
Ulasen does not teach wherein when the confidence score is below the threshold confidence score, the low confidence item is passed to a human expert for evaluation.
However, Zhdanov teaches wherein when the confidence score is below the threshold confidence score, the low confidence section is passed to a human expert for evaluation. (Col. 21, Ln 34-36, “those item(s) that the ML model was unable to identify, or identified below a threshold confidence, may be sent to a reviewer for review.”).
It would have been obvious to one of ordinary skill in the art before the effective filing data to modify Ulasen, incorporating the teachings of Zhdanov in order to obtain expert verification of items that don’t satisfy the required confidence threshold. (Col. 21).
Regarding claim 7, Ulasen teaches wherein the updating of the ground truth KG is performed by a creation module that synchronizes insertion of the extracted knowledge to the ground truth KG. (Para 0048, “The computing device may execute the KG updating engine 122 to update the knowledge graph 108 database with new knowledge data and/or new relationships between the knowledge data nodes.”, wherein the KG updating engine is the creation module and updating the existing knowledge graph corresponds to synchronizing insertion of the extracted knowledge. Specifically, the KG updating engine coordinates the addition of the newly extracted information into the KG so that the stored graph is aligned with the newly identified knowledge).
Regarding claim 8, Ulasen teaches wherein the ground truth KG comprises nodes that each correspond to a respective entity of the portion of the network (Para 0035, “The knowledge graph 108 uses nodes to represent entities and edges to represent the relationships between them.” where the KG in this application (including the reference / ground-truth KG) can be used to represent a network as in para 0037, “knowledge graph module 110 may correspond to a computing device 101 or cloud network”), and also comprises edges connecting the nodes, and the edges represent relationships between the nodes. (Para 0035, “The knowledge graph 108 uses nodes to represent entities and edges to represent the relationships between them.”).
Regarding claim 10, Ulasen teaches updating the knowledge graph based on a confidence score. (Para 0044, “proposing knowledge graph updates with a confidence score for evaluation.”).
Ulasen does not teach wherein when the confidence score is below the threshold confidence score, information is received, from a human expert, that indicates an outcome of an evaluation, by the human expert.
However, Zhdanov teaches wherein when the confidence score is below the threshold confidence score, information is received, from a human expert, that indicates an outcome of an evaluation, by the human expert. (Col. 21, Ln. 43-47, “At 310, the process 300 may receive the results of the review associated with the item(s). For example, the ML model may receive an indication indicating that the determined item(s) as output or predicted by the ML model(s) where cats or dogs.” Where the review is the information received from the human expert that indicates an outcome of an evaluation).
It would have been obvious to one of ordinary skill in the art before the effective filing data to modify Ulasen, incorporating the teachings of Zhdanov in order to use the expert’s evaluation outcome to confirm, reject, or correct a low-confidence KG update. (Col. 21).
Claim 11 is analogous to claim 1 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 13 is analogous to claim 3 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 14 is analogous to claim 4 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 15 is analogous to claim 5 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 16 is analogous to claim 6 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 17 is analogous to claim 7 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 18 is analogous to claim 8 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claim 20 is analogous to claim 10 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Ulasen (US 20260187105 A1), Duan (CN 117033668 A) and Zhdanov (US 11501210 B1) as above in claims 1, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 15, 16, 17, 18, 20 and further in view of Lu et al. (WO 2021098876 A1).
Regarding claim 2, Ulasen does not teach wherein the network comprises a RAN (radio access network).
However, Lu teaches wherein the network comprises a RAN (radio access network). (Pg. 22, “The communication network may be an Ethernet or a radio access network (RAN)”).
It would have been obvious to one of ordinary skill in the art before the effective filing data to modify Ulasen, incorporating the teachings of Lu in order to permit communication with devices through a known wireless access network architecture (Pg. 22).
Claim 12 is analogous to claim 2 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ulasen (US 20260187105 A1), Duan (CN 117033668 A) and Zhdanov (US 11501210 B1) as above in claims 1, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 15, 16, 17, 18, 20 and further in view of R et al. (US 20260127378 A1).
Regarding claim 9, Ulasen modified by Duan teaches a judge LLM which evaluates the KG for correctness after it is generated. (Duan, Pg. 19-20, “each knowledge in the target knowledge map can be evaluated by a second large language model, and the accuracy of the target knowledge map can be given according to the evaluation result of all the knowledge”).
Ulasen modified by Duan does not teach comparing the portion of the KG with a corresponding portion of the ground truth KG.
However, R teaches comparing the portion of the KG with a corresponding portion of the ground truth KG (Para 0031, “For each of the response triplets, one or more matching KG triplets may be identified from the knowledge graphs 250 via either exact or inexact matching. The semantic similarity between a pair of a response triplet and a matching KG triplet may be determined to indicate how accurate and consistent the response triplet is when compared with the matching ground truth triplet (KG triplet)”).
It would have been obvious to one of ordinary skill in the art before the effective filing data to modify Ulasen, incorporating the teachings of R in order to improve the accuracy of the KG evaluation by verifying generated knowledge against corresponding ground truth (Para 0031).
Claim 19 is analogous to claim 9 in that it recites substantially the same limitations. It is therefore rejected for the same reasons.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL ALAN FOSTER JR. whose telephone number is (571)272-8874. The examiner can normally be reached M - F 8:00am - 5:00pm, Alternate Fridays Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at (571) 272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL A FOSTER JR/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights