Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. Claims 1-21 are present for examination.
Information Disclosure Statement
3. The information disclosure statement (IDS) filed on 02/27/2025 was considered by the examiner.
Drawings
4. Regarding figures 1 and 2, these drawings are objected to because there are unlabeled circles or rectangular boxes in these figures. The unlabeled circles or rectangular boxes should be provided with descriptive text labels.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
5. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description:
(1) The “the context 5” disclosed in Specification paragraph 0018.
(2) The “Step 130” disclosed in Specification paragraph 0026.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
6. Claim 8 is objected to because of the following informalities:
Regarding claim 8, this claim recites the phrase of “the given query should have” [emphasis added] in line 3. It is suggested to avoid the use of a term “should” in claim recitations.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
7. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
8. Claims 1-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1 and 21, these claims recite the phrase of “in part on the confidence metrics” [emphasis added] in line 16 of claim 1 or line 18 of claim 21. However, it is unclear if this limitation refers to the limitations of “a confidence metric” in line 11 of claim 1 or line 13 of claim 21 and/or “one or more of the confidence metrics” [emphasis added] in lines 14-15 of claim 1 or lines 16-17 of claim 21. Clarification is required.
These claims additionally recite the phrase of “a propensity of the answer” [emphasis added] in line 16 of claim 1 or line 18 of claim 21. However, it is unclear if this limitation refers to the limitation of “a propensity of the answer” [emphasis added] in line 12 of claim 1 or line 14 of claim 21. Clarification is required.
Regarding claim 2, this claim recites the phrase of “the confidence metrics” [emphasis added] in line 4. However, it is unclear if this limitation refers to the limitation(s) of “a confidence metric” in line 11 of claim 1 or line 13 of claim 21, “one or more of the confidence metrics” in lines 14-15 of claim 1 or lines 16-17 of claim 21 and/or “in part on the confidence metrics” [emphasis added] in line 16 of claim 1 or line 18 of claim 21. Clarification is required.
Additionally, claim 2 recites the phrase of “so-obtained answers” [emphasis added] in line 5. It is unclear what exactly is meant by the phrase “so-obtained answers” because it is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Clarification is required.
Further, claim 2 recites the phrases of “a set of verification questions” in lines 1-2, “verification quarrions” in line 3, and “the verification questions” in line 5. It is unclear whether or not these phrases are related to one another. Clarification is required.
Regarding claim 5, this claim recites the phrase of “a verification question” and “the verification question” in lines 1-2. It is unclear whether or not these phrases are related to the phrases (“a set of verification questions”, “verification quarrions” and/or “the verification questions”) recited in claim 2. Clarification is required.
Also, this claim recites the limitation of “this given text” [emphasis added] in line 4. There is insufficient antecedent basis for this limitation in the claim.
Regarding claim 6, this claim recites the phrase of “a confidence metric” [emphasis added] in lines 4-5. However, it is unclear whether or not this phrase is related to “a confidence metric” in line 11, “one or more of the confidence metrics” and/or “in part on the confidence metrics” in line 16 of claim 1. Clarification is required.
Further, this claim recites the phrase of “so-generated question” [emphasis added] in line 4. Again, it is unclear what exactly is meant by the phrase “so-generated answers” because it is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Clarification is required. Also, claim 7 recites the similar limitation and is rejected due to the similar reasons as explained regarding claim 6.
Regarding claim 12, this claim recites the phrase of “a confidence metric” [emphasis added] in line 4. However, it is unclear whether or not this phrase is related to “a confidence metric” in line 11, “one or more of the confidence metrics” and/or “in part on the confidence metrics” in line 16 of claim 1. Clarification is required.
Regarding claim 16, this claim recites the phrase of “that is in direct physical interaction” [emphasis added] in line 2. However, it is unclear what exactly is meant by the phrase “direct physical interaction” because it is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Clarification is required.
Regarding claim 19, this claim recites the phrase of “an abnormal operating state” [emphasis added] in line 2. However, it appears that the term abnormal is indefinite since the intended scope of such a term is unclear since it is not defined by the claim, and the specification does not provide a standard for ascertaining the requisite degree. Thus, one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Clarification is required.
Claims not specifically mentioned above are rejected by virtue of their dependency on a rejected claim.
Claim Rejections - 35 USC § 103
9. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
10. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
11. Claims 1-21 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. 2024/0403341 (hereinafter Berglund) in view of U.S. 2024/0412720 (hereinafter Vasylyev), and further in view of U.S. 2023/0176557 (hereinafter Cella).
Regarding claim 1, as far as the claim is understood, Berglund discloses a computer-implemented method for retrieving information about at least one asset in an industrial plant, comprising:
providing, to a large language model (LLM) that is configured to take a text prompt as input and repeatedly predict portions of text, a given query for information, as well as [technical] context information about at least one [asset], thereby obtaining an answer to the given query ([0047]; “The prompt to the LLM 14 can further include context information of the query. For example, the query service 112 retrieves contextual information associated with the user who submitted the search query or the context in which the search query was generated, providing the contextual information to the LLM with the search query and the related content to enable the LLM to generate more customized answers…”);
setting up (i.e., trained or predict), based at least in part on the context information and one or both of the given query and the answer to this given query, a verification plan (i.e., programmatic rules), the verification plan comprising one or more actions (i.e., remove the lower-scored answer from the set of answers), wherein executing each action produces a confidence metric (i.e., confidence score) that is [indicative of a propensity of] the answer to the given query obtained from the LLM being correct ([0034-0035 and 0045]; “….Such predictions can be made based on programmatic rules (i.e., evaluating whether a query contains certain indicators), or by using a machine learning model that is trained to predict the intent of the query. Similarly, the query service 112 can ask the LLM 140 to evaluate the intent of the query”; and “…For example, if LLM generates the same answer to each of the submitted queries, the query service 112 may output only one answer to the user. Similarly, if one of the answers has a significantly lower confidence score than other answers, the query service 112 may remove the lower-scored answer from the set of answers output to the user”);
executing the verification plan, thereby obtaining one or more of the confidence metrics ([0051]; “The query service 112 can generate a confidence score for an answer, in some implementations… For example, the confidence score can indicate a degree of match between the answer generated by the LLM and the related content…”).
Berglund does not explicitly disclose the feature of producing a confidence metric that is indicative of a propensity of the answer to the given query obtained from the LLM being correct; and determining, based at least in part on the confidence metrics, a propensity of the answer to the given query obtained from the LLM being correct. However, Vasylyev discloses that “…Using this contextual information, assistant system 2 can query a weather API or knowledge base to obtain the relevant forecast data. It can then analyze the precipitation probability and intensity to determine whether an umbrella is recommended. Finally, the system can generate a natural language response that provides the requested information and advice, such as “It looks like there's a 60% chance of light rain showers this afternoon. It might be a good idea to bring an umbrella, just in case!” ([0271]). Vasylyev additionally discloses that “According to an aspect, this priority-augmented functionality may be advantageously utilized to enhance the security of assistant system 2 and its alignment with the intended guidelines. Also, various system-level inputs, such as those determining the prescribed behavior of assistant system 2 in response to use inputs or other events, may be provided with tags of higher priority to enforce the intended performance or at least significantly increase its probability. If a user subsequently provides an input that contradicts the system prompt but has a lower priority than the system prompt, assistant system 2 may be configured to generate response or perform other actions according to the system prompt rather than as instructed by the user. Similarly, the prioritization tags and respective LLM's parameter weighting may be configured to prevent lower-tier users to override the earlier or subsequent input of a higher-tier user…” ([0329]) and it would have been obvious for one with ordinary skill in the art to utilize the teachings of Vasylyev in the system of Berglund in view of the desire to enhance the search query and response generating system by utilizing the propensity metrics in the LLM resulting in improving the efficiency of generating the query results.
The references do not explicitly disclose the features of wherein a given query for information, as well as technical context information about at least one asset; and wherein the context information relates at least to one or more of: capabilities of the asset, requirements of the asset, how to interact with the asset, parameter values of the asset, and sensor data relating to the asset. However, Cella discloses that “In embodiments, the platform 100 may include the local data collection system 102 deployed in the environment 104 to monitor signals from additional large machines such as turbines, windmills, industrial vehicles, robots, and the like…” ([0325]). Cella also discloses that “ In one example, the set of properties of a digital twin of an industrial asset that may be updated by the digital twin dynamic model system 40008 using dynamic models 400100 may include the vibration characteristics of the asset, temperature(s) of the asset, the state of the asset (e.g., a solid, liquid, or gas), the location of the asset, the displacement of the asset, the velocity of the asset, the acceleration of the asset, probability of downtime values associated with the asset, cost of downtime values associated with the asset, probability of shutdown values associated with the asset…” ([2292]). In addition, Cella discloses that “…Additionally or alternatively, the digital twin system I/O system 60304 may periodically query and/or receive data from a connected data source 60030, such as a sensor system 60032 having sensors that sensor data from facilities (e.g., manufacturing facilities…” ([2436]) and it would have bene obvious for one with ordinary skill in the art to utilize the teachings of Cella in the modified system of Berglund in view of the desire to enhance the search query and response generating system by utilizing in the specific industrial plant components resulting in improving the efficiency of producing the technical task using the LLM schemes.
Additionally, Berglund discloses a non-transitory computer storage media containing machine-readable instructions that, when executed by one or more computers and/or compute instances ([0088 and 0090]).
Regarding claim 2, Berglund in view of Vasylyev and Cella disclose the method wherein the verification plan comprises at least a set of verification questions and expected answers;
wherein executing the verification plan comprises at least providing verification questions to the LLM to which the given query was provided, and/or to a different LLM; and wherein the confidence metrics comprise at least a measure for an extent to which the so-obtained answers to the verification questions are in agreement with the expected answers (Berglund: [0052 and 0055]) and (Vasylyev; [0148]). Therefore, the limitations of claim 2 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 3, Berglund in view of Vasylyev and Cella disclose the method wherein at least one verification question is chosen such that agreement of an answer to this question with an expected answer is indicative of whether the context information contains the answer to the given query; and/or the LLM is capable of understanding the given query; and/or the LLM can answer the given query given the context; and/or the LLM answers the given query in a logically correct way (Berglund: [0034 and 0053]).
Regarding claim 4, Berglund in view of Vasylyev and Cella disclose the method wherein at least one verification question is a paraphrase of the given query; and/or generated based at least in part on the context and optionally also the answer to the original query; and/or a question with an expected answer that is related to the given query (Berglund: [0033-0034]; the intent of the query).
Regarding claim 5, Berglund in view of Vasylyev and Cella disclose the method wherein at least one expected answer to a verification question is obtained by providing the context, and the verification question, to an extractive language model that is configured to extract information from given text in words and phrases from this given text (Berglund: [0034-0035 and 0038]) and (Vasylyev; [0148]). Therefore, the limitations of claim 5 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 6, Berglund in view of Vasylyev and Cella disclose the method wherein the verification plan further comprises generating, by the LLM to which the original query was provided, and/or by a different LLM, at least one question based on the context information, and also the answer to the given query; and determining a similarity of the so-generated question and the given query as a confidence metric (Berglund: [0033 and 0038-0039]; the similarity score).
Regarding claim 7, Berglund in view of Vasylyev and Cella disclose the method wherein the verification plan further comprises obtaining, in a manner different from the LLM to which the given query was provided, one or more further answers to the given query given the context information; and evaluating a confidence metric from the so-obtained further answers (Berglund: [0051-0053]; the same LLM or by different LLM).
Regarding claim 8, Berglund in view of Vasylyev and Cella disclose the method wherein the evaluating of the confidence metric comprises evaluating to which extent the original answer to the given query is reliable given the context information; and/or the given query should have an answer given the context information (Berglund: [0033 and 0051]).
Regarding claim 9, Berglund in view of Vasylyev and Cella disclose the method wherein the further answers are extracted from the context information by an extractive language model that is configured to extract information from given text in words and phrases from this given text (Berglund: [0014 and 0063]; the text chunk of the user query).
Regarding claim 10, Berglund in view of Vasylyev and Cella disclose the method wherein the verification plan further comprises converting the given query into an embedding that is a numerical encoding for inputting the given query into the LLM; comparing this embedding to embeddings of training examples used for training the LLM; and evaluating a confidence metric from the result of this comparison (Berglund: [0075, 0076-0077 and 0080]; the tokenized input by the LLM and utilizing the embedding vectors).
Regarding claim 11, Berglund in view of Vasylyev and Cella disclose the method wherein the comparing to the embeddings of training examples comprises determining a cluster of the embeddings of the training examples; and evaluating a distance of the embedding of the given query from this cluster (Berglund: [0073, 0076 and 0079]; the text segments and comparing the distances).
Regarding claim 12, Berglund in view of Vasylyev and Cella disclose the method wherein the verification plan further comprises determining one or more statistical quantities on the text of the answer to the given query on the one hand, and on the context information on the other hand; comparing the so-obtained values of the one or more statistical quantities; and
evaluating a confidence metric from the result of this comparison (Berglund: [0045 and 0073]) and (Vasylyev: [0079-0080 and 0372]; the TF-IDF feature for text data). Therefore, the limitations of claim 12 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 13, Berglund in view of Vasylyev and Cella disclose the method wherein the context information comprises a technical specification, a device description, and/or a manual of the asset, and/or a layout of the industrial plant as a whole (Berglund: [0040]) and (Cella: [0930]). Therefore, the limitations of claim 13 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 14, Berglund in view of Vasylyev and Cella disclose the method further comprising determining, from the answer to the given query, at least one action that changes the physical state and/or behavior of the asset to be performed on the at least one asset; and modifying the so-determined action based at least in part on the propensity of this answer being correct (Berglund: [0021]), (Vasylyev: [0271]) and (Cella: [2139]). Therefore, the limitations of claim 14 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 15, Berglund in view of Vasylyev and Cella disclose the method further comprising performing the modified action on the at least one asset (Berglund: [0021]) and (Cella: [2139]). Therefore, the limitations of claim 15 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 16, as far as the claim is understood, Berglund in view of Vasylyev and Cella disclose the method wherein the asset is a module of a modular industrial plant, or any other field device that is in direct physical interaction with an industrial process being executed on the industrial plant (Cella: [0472 and 0499]; “…Two modules may be constructed with substantially identical physical components, but may perform different functions in the industrial environment based on the program(s) loaded into programmable logic component(s) on the modules...”). Therefore, the limitations of claim 16 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 17, Berglund in view of Vasylyev and Cella disclose the method wherein the given query is chosen to relate to how to access a given functionality of the asset via a user interface of the asset (Berglund: [0040]) and (Cella: [0189 and 2658]). Therefore, the limitations of claim 17 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 18, Berglund in view of Vasylyev and Cella disclose the method further comprising modifying the user interface of the asset based at least in part on the given query and the obtained answer to this given query so as to make the given functionality better accessible in the user interface of the asset (Berglund: [0065]), (Vasylyev: [0051 and 0402]) and (Cella: [1659]; manipulating a user interface). Therefore, the limitations of claim 18 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 19, as far as the claim is understood, Berglund in view of Vasylyev and Cella disclose the method wherein the given query is chosen to relate to whether the at least one asset, and/or the industrial plant as a whole, is in an abnormal operating state (Berglund: [0013]) and (Cella: [0010 and 0998]; the malfunction, maintenance needed, faulty component operations). Therefore, the limitations of claim 19 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Regarding claim 20, Berglund in view of Vasylyev and Cella disclose the method wherein the propensity of the answer to the given query obtained from the LLM being correct is computed as an aggregate of individual confidence metrics, or a minimum of all individual confidence metrics (Berglund: [0013 and 0047]) and (Vasvlyev: [0732-0733]). Therefore, the limitations of claim 20 are also rejected in the analysis of claim 1, and the claims are rejected on that basis.
Conclusion
12. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MONICA M PYO whose telephone number is (571)272-8192. The examiner can normally be reached Monday-Friday 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, APU MOFIZ can be reached at 571-272-4080. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MONICA M PYO/ Primary Examiner, Art Unit 2161