DETAILED ACTION
Status of the Claims
Original claims 1-12 filed February 5, 2025, are pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on February 5, 2025, and April 13, 2026, are being considered by the examiner.
Claim Interpretation
Claims are given their broadest reasonable interpretation (BRI) during examination. MPEP 2111. Under BRI, the words of a claim are given their plain meaning, unless such meaning is inconsistent with the specification. MPEP 2111.01, Subsection I. The plain meaning of a term is the ordinary and customary meaning given to the term by those of ordinary skill in the art at the relevant time. Id.
Claim 5 recites “at least one of parsing, recognizing, transforming, and formalizing the image content”, which uses an at least one of: … and … format.
The plain meaning of a list in at least one of: … and … format is that it requires at least one of each listed element. For example, the plain meaning of a claim reciting at least one of A, B, and C is that it requires at least one of A, at least one of B, and at least one of C. SuperGuide Corp. v. DirecTV Enters., Inc., 358 F.3d 870, 885-888, 69 U.S.P.Q.2d 1865, 1876-1878 (Fed. Cir. 2004).
However, this plain meaning is inconsistent with the specification. Par. [0083] of the specification (as published) is reproduced below:
PNG
media_image1.png
200
400
media_image1.png
Greyscale
This clearly indicates that claims in an at least one of: A, B and C format are meant to require only one of all possible permutations (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C).
Therefore, the BRI of “at least one of parsing, recognizing, transforming, and formalizing the image content” in claim 5 is that it can be satisfied by only parsing, only recognizing, only transforming, only formalizing, or any other set/subset of these actions.
Unless otherwise noted, this interpretation is applied to all other claims that use an at least one of: … and … format, such as claims 3 and 9-12.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 10 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 10 recites the limitation "the lookup table" in in the last line. There is insufficient antecedent basis for this limitation in the claim.
Claim 10 recites a list including “image classification and/or segmentation, and proximity-based relation identification, …”. It is unclear whether “image classification and/or segmentation” and “proximity-based relation identification” are (a) different elements in the list, or (b) part of one combined element in the list. This ambiguity makes the scope of the claim unclear and renders the claim indefinite.
On the one hand, the claim uses the word “and” and both phrases are listed on the same line, which suggests that they are (b) part of one combined element in the list.
On the other hand, the claim delimits the other elements using commas and a comma separates “image classification and/or segmentation” from “proximity-based relation identification”, which suggests that they are (a) different elements in the list.
For purposes of practicing compact prosecution (MPEP 2173.06), the noted phrases from claim 10 are interpreted to be (a) different elements in the list.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more. See MPEP 2106 for an explanation of the patent subject matter eligibility (SME) analysis provided below.
Step 1
Yes, the claims are directed to one of the four statutory categories of inventions. Claims 1-12 are to a process.
Step 2A, Prong One
Yes, the claims do recite a mental process type of abstract idea judicial exception. See MPEP 2106.04(a)(2), Subsection III.
Claim 1 recites inputting a topology image, which can be performed mentally by viewing such an image. Claim 1 further recites identifying a symbol and/or an embedding representation indicative of a symbol in at least a segment of the topology image, which can be performed mentally by recognizing a symbol (e.g., recognizing text, a circle, etc.). Claim 1 further recites comparing a result of the identifying with predetermined symbols that are associated with predetermined characteristics, which can be performed mentally. For example, a person can mentally recognize that an identified symbol in a diagram topology image matches a predetermined pressure gauge symbol, which is associated with particular characteristics, such as the fact that it measures pressure and the range and units of the pressures it measures. A human can also mentally refer to a printed lookup table that associates predetermined symbols with certain characteristics and find an entry that matches an identified symbol. One example would be to use a dictionary (i.e., a lookup table) to find characteristics associated with a particular predetermined (by virtue of having been included in the dictionary) word symbol. Claim 1 further recites, based on a result of the comparing, determining at least one characteristic associated with the result of the identifying, which can be performed mentally by associating characteristics of a matched predetermined symbol with an identified symbol. Claim 1 further recites, based on a result of the determining, deriving a structured representation from an image content of the topology image, which can also be performed mentally, especially at the level of generality recited in the claimed invention. For example, a human could write a table of identified symbols and their corresponding characteristics on a piece of paper.
Examiner notes that claim 1 is to a process of collecting information in the form of a topology image and analyzing it to derive a structured representation of the image’s contents. This is analogous (albeit generic given the lack of a requirement to display or otherwise output the derived structured representation) to a claim to “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, which is an example of a claim that has been found by the courts to recite a mental process. MPEP 2106.04(a)(2), Subsection III, A, citing to Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016).
The scope of claim 2 can be satisfied by using a lookup table, which can be performed mentally for substantially the same reasons discussed above.
The scope of claim 3 can be satisfied by identifying a predetermined symbol related to the identified symbol, which can be performed mentally for substantially the same reasons discussed above.
Claim 4 recites identifying at least two symbols, which can be performed mentally for substantially the same reasons discussed above. Claim 4 further recites determining a proximity of the symbols and determining a relation based on the proximity, which can also be performed mentally. For example, a person can mentally read a diagram and understand that a temperature sensor symbol close to a tank symbol indicates that the temperature sensor is measuring the temperature of the tank.
Claim 5 generally recites functions of parsing, recognizing, transforming, and/or formalizing image content, which are all recited at such a high level of generality that they encompass functions that can be performed mentally. For example, a human can look at a topology image and recognize a depicted structure.
Claim 6 recites receiving feedback at such a high level of generality that it encompasses functions that can be performed mentally.
The scope of claim 7 can be satisfied by drawing a picture [i.e., a control graphic] and using it to help review the image content, which can be performed mentally.
The scope of claim 9 can be satisfied by writing a sentence on a piece of paper that describes the input topology image, which can be performed mentally.
The scope of claim 10 can be satisfied by classifying or segmenting an image, which can be performed mentally.
Claim 12 recites various predetermined characteristics that can be recognized and determined mentally for substantially the same reasons discussed above.
Step 2A, Prong Two
None of the additional elements recited in the claims integrate the abstract idea into a practical application because none of the additional elements go beyond:
Merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f);
Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g); or
Generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h).
As an initial matter, Examiner notes that many of the claim elements are directed to the abstract idea judicial exception itself (see above) and therefore are not additional elements. Furthermore, many of the dependent claims are quite broad in that they require only one of a list of options.
Several claims recite the use/training of an AI/ML model (e.g., claims 1, 2, 6 and 10). These additional elements amount to no more than reciting the words “apply it” (or an equivalent) or mere instructions to implement an abstract idea or other exception on a computer. The AI/ML model is recited at a high level of generality and is apparently invoked only as a tool to perform an existing process. For example, claim 1 requires inputting a topology image to an AI/ML model and claim 2 recites that the AI/ML model may be used for various steps of processing the topology image, but the specification acknowledges that such topology image processing is an existing process that is conventionally performed manually (e.g., [0004]).
Claim 8 specifies that “the topology image shows a brownfield topology and/or comprises brownfield topology information.” Claim 11 specifies that the symbol and/or predetermined symbol is associated with at least one of a valve, a tank, a sensor, a reactor, a boiler, a mixer, a separator, a controller, and an input/inlet/output/outlet. These elements do not amount to any more than generally linking the use of a judicial exception to a particular industrial, material handling, etc. technological environment or field of use. This is similar to how “[l]imiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid” was described as merely indicating a field of use or technological environment in which to apply a judicial exception “because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment.” MPEP 2106.05(h), citing to Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016).
Step 2B
The identification of additional elements from Step 2A, Prong Two is carried over. MPEP 2106.05, Subsection II. No additional elements were considered to be insignificant extra-solution activity. None of the additional elements, alone or in combination, amounts to significantly more than an abstract idea because, as explained above, and especially in view of the breadth imposed by the use of optional lists throughout the claims, none of the additional elements amounts to more than merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, or generally linking the use of a judicial exception to a particular technological environment or field of use, none of which are significantly more.
Conclusion
Claims 1-12 are patent-ineligible under 35 U.S.C. 101 because they are directed to an abstract idea judicial exception and do not recite any additional elements that amount to significantly more than the abstract idea judicial exception.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-12 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by ‘Poirier’ (US 2021/0319327 A1).
Regarding claim 1, Poirier discloses a method for processing topology images (e.g., Figures 1 and 5), comprising:
inputting a topology image (e.g., [0029]-[0030], a digital image of a schematic is generated by preprocessing 110) to an artificial intelligence/machine learning (AI/ML) model (e.g., [0031], Fig. 1, schematic image from preprocessing 110 is input to classification subsystem 120; Fig. 1, the classification subsystem includes detection module 122 and classification module 124, both of which use AI/ML models – e.g., [0032], [0039], [0043]);
identifying a symbol and/or an embedding representation indicative of a symbol in at least a segment of the topology image (e.g., [0034], areas within schematic/topology image are detected/identified as tag symbol, instrument symbol, or no symbol);
comparing a result of the identifying with at least one symbol among predetermined symbols and/or with at least one embedding representation indicative of a symbol among predetermined embedding representations indicative of predetermined symbols (e.g., [0039], Fig. 1, classification module 124 classifies the detected symbols as one of predetermined component symbols/embedding representations within a components library; [0043], the classification may be carried out by comparing embedding representations of the detected/identified symbols with embedding representations of predetermined symbols in a components library), wherein the predetermined symbols are associated with predetermined characteristics (e.g., [0048], components in the components library – i.e., the predetermined symbols – are associated with parameters/characteristics, such as “dimensions, function, inputs and outputs, throughput, operating variables, etc.”);
based on a result of the comparing, determining at least one characteristic associated with the result of the identifying (e.g., [0043], the comparing is used to determine what specific component in the components library matches the detected/identified symbol; Thus, the characteristics/parameters of that specific component are associated with the detected/identified symbol); and
based on a result of the determining, deriving a structured representation from an image content of the topology image (e.g., [0048], a digital representation of the physical system is generated from the content extracted from the topology image, the digital representation being a structured representation; e.g., [0048], the construction of the digital model/representation is based on the determined characteristics/parameters, which are retrieved “retrieved when the digital model is created”).
Regarding claim 2, Poirier discloses the method according to claim 1, wherein (Note: in view of “and/or” at the end of the third-to-last line, the scope of this claim can be satisfied by disclosure of at least one of the elements in the list):
the identifying comprises identifying the symbol and/or the embedding representation by use of the AI/ML model (e.g., [0032], detection module 122 may perform symbol identification/detection using a deep neural network, which is within the scope of an AI/ML model);
the identifying comprises identifying an embedding representation indicative of the symbol and identifying the symbol from the identified embedding representation;
the comparing comprises comparing the identified symbol and/or the identified embedding representation by use of at least one of nearest neighbor-search, similarity search, a lookup table in which the predetermined symbols and associated characteristics are stored, and a representation indicative of the predetermined embedding representations;
the determining comprises determining the at least one characteristic by use of the AI/ML model; and/or
the deriving comprises deriving the structured representation by use of the AI/ML model.
Regarding claim 3, Poirier discloses the method according to claim 1, wherein the determining comprises determining, as the at least one characteristic, at least one of inputs and/or outputs related to the identified symbol and/or the identified embedding representation (Note: this claim is in at least one of: … and … format – see Claim Interpretation above; e.g., [0048], identified symbol/component parameters/characteristics include “inputs and outputs”), and a predetermined symbol and/or a predetermined embedding representation related to the identified symbol and/or the identified embedding representation.
Regarding claim 4, Poirier discloses the method according to claim 1, wherein the identifying comprises identifying at least two symbols and/or two embedding representations; and wherein the determining comprises determining, as the at least one characteristic, a proximity of the identified symbols and/or the identified embedding representations to each other and determining a relation among the identified symbols and/or the identified embedding representations based on the result of the comparing and on the determined proximity (e.g., [0045]-[0047], Text symbols may be identified and associated with drawn symbols [e.g., a circle] based on being proximate/adjacent/near one another, the text being used to look up further characteristics in a components table; Additionally or alternatively, e.g., [0032] and [0036] describe identifying/detecting multiple symbols, determining whether they fall within the same dashed line [which is a proximity determination], and characterizing them as being constituent parts of a same assembly based on the determination).
Regarding claim 5, Poirier discloses the method according to claim 1, wherein the deriving comprises at least one of parsing (Note: this claim is in at least one of: … and … format – see Claim Interpretation above; e.g., [0048], image content of the topology/schematic image is parsed into a machine-readable bill of materials), recognizing, transforming, and formalizing the image content.
Regarding claim 6, Poirier discloses the method according to claim1, further comprising:
receiving feedback on the result of the identifying (e.g., [0032], a neural network is used for the identifying/detection; e.g., [0041], training a neural network includes comparing predicted outputs to expected outputs to calculate a cost, which is receiving feedback) and/or on the result of the comparing and/or on the result of the determining and/or on a result of the deriving; and
training and/or retraining the AI/ML model based on the feedback (e.g., [0041], the neural network is trained based on the noted feedback, specifically by iteratively adjusting its weights and biases to minimize the cost).
Regarding claim 7, Poirier discloses the method according to claim 1, further comprising:
generating code and/or a control graphic based on the structured representation (e.g., [0048], “The digital model may be computer code that the [sic] defines the components in the physical system, the connections between the components, and the state of the physical system”); and
processing the image content based on the code and/or the control graphic (As explained in the mapping provided above, the image content [i.e., the “image content of the topology image” referenced at the end of claim 1] is processed to determine the components, connections, etc. represented in the digital model/code, which may be time varying).
Regarding claim 8, Poirier discloses the method according to claim 1, wherein the topology image shows a brownfield topology and/or comprises brownfield topology information (Note: Labeling the topology image as a “brownfield” topology image does not significantly limit the scope of the claim, as a topology image will be processed the same whether it depicts an existing [i.e., brownfield] or a future/new-build [i.e., greenfield] topology; Regardless, [0028] describes that the engineering documents [i.e., the topology image] are of a “physical system” [as opposed to a planned/theoretical/greenfield system], so it falls within the BRI of showing a brownfield topology).
Regarding claim 9, Poirier discloses the method according to claim 1, wherein the structured representation is at least one of (Note: this claim is in at least one of: … and … format – see Claim Interpretation above)
a text description indicative of the image content of the topology image (e.g., [0048], structured representation may be a machine-readable bill of materials that is a table including “text associated with the components”),
an ontological format indicative of the image content, and
a graph format indicative of the image content.
Regarding claim 10, Poirier discloses the method according to claim 1, wherein the AI/ML model is based on further using at least one of:
joint embeddings and/or nearest neighbor-search (e.g., [0043], autoencoder encodes symbols in to a joint feature embedding space),
image classification and/or segmentation (e.g., [0039], [0043]),
and proximity-based relation identification (e.g., [0043], proximity in feature space),
transformer-based neural networks, and
an analytics pipeline comprising the lookup table and similarity search.
Regarding claim 11, Poirier discloses the method according to claim 1, wherein the symbol and/or predetermined symbol is associated with at least one of (Note: this claim is in at least one of: … and … format – see Claim Interpretation above):
a valve,
a tank,
a sensor (e.g., [0034]),
a reactor,
a boiler,
a mixer,
a separator,
a controller, and
an input/inlet/output/outlet.
Regarding claim 12, Poirier discloses the method according to claim 1 wherein the at least one characteristic and/or predetermined characteristic comprises at Least one of (Note: this claim is in at least one of: … and … format – see Claim Interpretation above):
an input to the symbol and/or predetermined symbol (e.g., [0048], identified symbol/component parameters/characteristics include “inputs and outputs”),
an output from the symbol and/or predetermined symbol (e.g., [0048], identified symbol/component parameters/characteristics include “inputs and outputs”),
a relation of the symbol and/or predetermined symbol to another symbol and/or predetermined symbol,
a connection of the symbol and/or predetermined symbol to another symbol and/or predetermined symbol,
an indication about whether the relation is one-directional or bi-directional, and
an indication about whether the relation is labelled with a text representing the type of relation and/or giving further details and/or attributes.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
The following references describe further examples of using AI/ML models to process topology images:
‘Ganesh’ (WO 2025/146434 A1)
‘Paliwal’ (US 2023/0045646 A1)
‘Sinha’ (US 2022/0171891 A1)
‘Gope’ (EP 3933653 A1)
‘Sharma’ (US 2020/0175372 A1)
‘Wang’ (US 2018/0373961 A1)
‘Nurminen’ (“Object Detection in Design Diagrams with Machine Learning,” 2020)
‘Su’ (“Image format pipeline and instrument diagram recognition method based on deep learning,” 8 Dec 2023)
‘Theisen’ (“Digitization of chemical process flow diagrams using deep convolutional neural networks,” 8 Dec 2022)
‘Yu’ (“Features Recognition from Piping and Instrumentation Diagrams in Image Format Using a Deep Learning Network,” 2019)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEOFFREY E SUMMERS whose telephone number is (571)272-9915. The examiner can normally be reached Monday-Friday, 7:00 AM to 3:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEOFFREY E SUMMERS/Examiner, Art Unit 2669