DETAILED ACTION
Status of Claims
This is a non-final action in reply to the application filed on July 25, 2024.
Claims 1-20 are currently pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The Information Disclosure Statements filed on 7/25/2024 has been considered. Initialed copies of the Form 1449 are enclosed herewith.
Claim Rejections- 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Per MPEP 2106.03 Eligibility Step 1: The Four Categories of Statutory Subject Matter [R-07.2022]. Step 1 is directed to determining whether or not the claims fall within a statutory class. Herein, claims 1-7 falls within statutory class of a process claims 8-14 falls within statutory class of a machine and claims 15-20 falls within statutory class of an article of manufacturing. Hence, the claims qualify as potentially eligible subject matter under 35 U.S.C §101. With Step 1 being directed to a statutory category, per MPEP 2106.04 Eligibility Step 2A: Whether a Claim is Directed to a Judicial Exception [R-07.2022]. Step 2 is the two-part analysis from Alice Corp. (also called the Mayo test). The 2019 PEG makes two changes in Step 2A: It sets forth new procedure for Step 2A (called “revised Step 2A”) under which a claim is not “directed to” a judicial exception unless the claim satisfies a two-prong inquiry. The two-prong inquiry is as follows: Prong One: evaluate whether the claim recites a judicial exception. If claim recites an exception, then Prong Two: evaluate whether the claim recites additional elements that integrate the exception into a practical application of the exception. The claim(s) recite(s) the following abstract idea indicated by non-boldface font and additional limitations indicated by boldface font:
Claim 1:
retrieving from one or more data sources, the one or more documents associated with the one or more assets;
processing at least one document of the one or more documents using a first processing technique of one or more processing techniques;
extracting first data from the at least one document based on the first processing technique, wherein the first data comprises one or more instrument tags associated with corresponding assets;
validating the first data using one or more validation techniques;
processing the at least one document of the one or more documents using a second processing technique of the one or more processing techniques;
extracting second data from the at least one document based on the second processing technique, wherein the second data is different from the first data;
configuring one or more data templates based on consolidation of the validated first data and the extracted second data; and
rendering, on a display, the one or more data templates for the corresponding assets.
Claim 8:
a processor;
a memory communicatively coupled to the processor, wherein the memory comprises one or more instructions which when executed by the processor, cause the processor to:
retrieve from one or more data sources, the one or more documents associated with the one or more assets;
process at least one document of the one or more documents using a first processing technique of one or more processing techniques;
extract first data from the at least one document based on the first processing technique, wherein the first data comprises one or more instrument tags associated with corresponding assets;
validate the first data using one or more validation techniques;
process the at least one document of the one or more documents using a second processing technique of the one or more processing techniques;
extract second data from the at least one document based on the second processing technique, wherein the second data is different from the first data;
configure one or more data templates based on consolidation of the validated first data and the extracted second data; and
render, on a display, the one or more data templates for the corresponding assets.
Claim 15:
retrieve from one or more data sources, the one or more documents associated with the one or more assets;
process at least one document of the one or more documents using a first processing technique of one or more processing techniques;
extract first data from the at least one document based on the first processing technique, wherein the first data comprises one or more instrument tags associated with corresponding assets;
validate the first data using one or more validation techniques;
process the at least one document of the one or more documents using a second processing technique of the one or more processing techniques;
extract second data from the at least one document based on the second processing technique, wherein the second data is different from the first data;
configure one or more data templates based on consolidation of the validated first data and the extracted second data; and
render, on a display, the one or more data templates for the corresponding assets.
Per Prong One of Step 2A, the identified recitation of an abstract idea falls within at least one of the Abstract Idea Groupings consisting of: Mathematical Concepts, Mental Processes, or Certain Methods of Organizing Human Activity. Particularly, the identified recitation falls within Mental Processes, concepts performed in the human mind including observations, evaluation, judgement and opinion. Per Prong Two of Step 2A, this judicial exception is not integrated into a practical application because the claim as a whole does not integrate the identified abstract idea into a practical application. The processor, memory and display is recited at a high level of generality, i.e., as a generic computing and processing system. This processor, memory and display is no more than mere instructions to apply the exception using a generic computing devices each comprising at least a processor, memory and display device. Further, processor configured to cause receiving/determining/transmitting data is mere instruction to apply an exception using a generic computer component which cannot integrate a judicial exception into a practical application. Accordingly, this/these additional element(s) does/do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Thus, since the claims are directed to the determined judicial exception in view of the two prongs of Step 2A, MPEP 2106.05 Eligibility Step 2B: Whether a Claim Amounts to Significantly More [R-07.2022] is directed to Step 2B. Therein, per Step 2B the additional elements and combinations therewith are examined in the claims to determine whether the claims as a whole amounts to significantly more than the judicial exception. It is noted here that the additional elements are to be considered both individually and as an ordered combination. In this case, the claims each at most comprise additional elements of a processor, memory and display. Taken individually, the additional limitations each are generically recited and thus does not add significantly more to the respective limitations. Further, executing all the steps/functions by a user/service subsystem is mere instruction to apply an exception using a generic computer component which cannot provide an inventive concept in Step 2B (or, looking back to Step 2A, cannot integrate a judicial exception into a practical application). For further support, the Applicant’s specification supports the claims being directed to use of a generic processor, memory and display type structure at paragraphs 0035: “As illustrated in FIG. 2 , the controller 200 may include a processor 202, e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 202 may be a component in a variety of systems. For example, the processor 202 may be part of a standard computer. The processor 202 may be one or more general processors, ” Paragraph 0036: “The controller 200 may include a memory 204 that can communicate via a bus 218. The memory 204 may be a main memory, a static memory, or a dynamic memory.” And paragraph 0037: “the controller 200 may further include a display 208, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid-state display, a cathode ray tube (CRT), a projector, a printer or other now known or later developed display device for outputting determined information. The display 208 may act as an interface for the user to see the functioning of the processor 202, or specifically as an interface with the software stored in the memory 204 or in the drive unit 206.” See also figure 2.
Taken as an ordered combination, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the limitations are directed to limitations referenced in Alice Corp. that are not enough to qualify as significantly more when recited in a claim with an abstract idea include, as a non-limiting or non-exclusive examples: i. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 134 S. Ct. at 2360, 110 USPQ2d at 1984 (see MPEP § 2106.05(f)); ii. Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known to the industry, as discussed in Alice Corp., 134 S. Ct. at 2359-60, 110 USPQ2d at 1984 (see MPEP § 2106.05(d)); iii. Adding insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea such as a step of obtaining information about credit card transactions so that the information can be analyzed by an abstract mental process, as discussed in CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011) (see MPEP § 2106.05(g)); or v. Generally linking the use of the judicial exception to a particular technological environment or field of use, e.g., a claim describing how the abstract idea of hedging could be used in the commodities and energy markets, as discussed in Bilski v. Kappos, 561 U.S. 593, 595, 95 USPQ2d 1001, 1010 (2010) or a claim limiting the use of a mathematical formula to the petrochemical and oil-refining fields, as discussed in Parker v. Flook. The courts have recognized the following computer functions inter alia to be well-understood, routine, and conventional functions when they are claimed in a merely generic manner: performing repetitive calculations; receiving, processing, and storing data (e.g., the present claims); electronically scanning or extracting data; electronic recordkeeping; automating mental tasks (e.g., process/machine for performing the present claims); and receiving or transmitting data (e.g., the present claims). The dependent claims 2-7, 9-14 and 16-20 do not cure the above stated deficiencies, and in particular, the dependent claims further narrow the abstract idea without reciting additional elements that integrate the exception into a practical application of the exception or providing significantly more than the abstract idea. Claims 2, 9 and 16 further limit the abstract idea by retrieving at least one of: one or more engineering documents, one or more process flow diagrams (PFDs), one or more piping and instrumentation diagrams (P&IDs), and one or more datasheets from the one or more data sources (a more detailed abstract idea remains an abstract idea). Claims 3, 10 and 17 further limit the abstract idea by analyzing one or more textual representations in proximity to one or more symbolic representations in the at least one document using the first processing technique, wherein the first processing technique corresponds to one or more image processing techniques based on machine learning and natural language processing; determining if the proximity of the one or more textual representations satisfies one or more first thresholds, wherein the one or more first thresholds are defined relative to size of the one or more symbolic representations, a type of the one or more symbolic representations, co-ordinates of a corresponding textual representation in the at least one document, and a distance between at least one symbolic representation and the corresponding textual representation; and identifying the one or more textual representations to be the one or more instrument tags if the proximity of the one or more textual representations satisfies one or more first thresholds (a more detailed abstract idea remains an abstract idea). Claims 3, 10 and 17 uses the image processing techniques based on machine learning and natural language processing as a tool, in its ordinary capacity, to carry out the abstract idea. Claims 4, 11 and 18 further limit the abstract idea by validating the first data using a first validation technique of the one or more validation techniques, wherein the first validation technique comprises: comparing at least one instrument tag of the one or more instrument tags with one or more tags provided by one or more users; and determining a first validation score for the at least one instrument tag; validating the first data using a second validation technique of the one or more validation techniques, wherein the second validation technique comprises: verification of the at least one instrument tag using one or more rules, co-ordinates of corresponding textual representation in the at least one document, and one or more material flow directions; and determining a second validation score for the at least one instrument tag; and determining a validation score for the at least one instrument tag based on the first validation score and the second validation score (a more detailed abstract idea remains an abstract idea). Claims 5, 12 and 19 further limit the abstract idea by analyzing one or more textual representations associated with one or more predefined shapes in the at least one document by the second processing technique, wherein the second processing technique corresponds to optical character recognition (OCR); determining if the one or more textual representations associated with the one or more predefined shapes satisfies one or more second thresholds, wherein the one or more second thresholds are defined relative to size of the one or more predefined shapes, a type of the one or more predefined shapes, and co-ordinates of a corresponding textual representation in the at least one document; and identifying the one or more textual representations to be the second data if the one or more textual representations satisfies one or more second thresholds (a more detailed abstract idea remains an abstract idea). Claims 5, 12 and 19 uses the optical character recognition technique as a tool, in its ordinary capacity, to carry out the abstract idea. Claims 6 and 13 further limit the abstract idea that configuring the one or more data templates comprises filling one or more fields of corresponding data templates using consolidation of the validated first data and the extracted second data (a more detailed abstract idea remains an abstract idea). And claims 7, 14 and 20 further limit the abstract idea by deriving one or more units of measurements and one or more operating limits associated with the corresponding assets based at least on the second data (a more detailed abstract idea remains an abstract idea). The identified recitation of the dependents claims falls within the Mental Processes, concepts performed in the human mind including observations, evaluation, judgement and opinion. Since there are no elements or ordered combination of elements that amount to significantly more than the judicial exception, the claims are not eligible subject matter under 35 USC §101. Thus, viewed as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. Therefore, the claim(s) are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 6-10, 13-17 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Sinha et al., (US 2022/0171891 A1) hereinafter “Sinha”.
Claim 1:
Sinha as shown discloses a method for extracting data from one or more documents associated with one or more assets in a facility, the method comprising:
retrieving from one or more data sources, the one or more documents associated with the one or more assets (¶ 0073: “The HMI application 328 generally begins at block 504 with inputting data from various plant engineering data sources, including plant engineering diagrams (e.g., PI&D, PFD, etc.), into the industrial plant control system 300.”);
processing at least one document of the one or more documents using a first processing technique of one or more processing techniques; extracting first data from the at least one document based on the first processing technique, wherein the first data comprises one or more instrument tags associated with corresponding assets (¶ 0073: “At block 506, the data is processed to extract relevant information and assets, such as names, numbers, symbols, lines, loops, and the like from the data. The asset extraction is done using machine learning via intelligent process 316 and the ML models therein.”);
validating the first data using one or more validation techniques (¶ 0068: “Rules engine 322 is configured to verify extracted tags from the compute image based on one or more rules.”);
processing the at least one document of the one or more documents using a second processing technique of the one or more processing techniques (¶ 0074: “ The asset relationship establishing is also done with the aid of intelligent process 316 and the ML models therein.”);
extracting second data from the at least one document based on the second processing technique, wherein the second data is different from the first data (¶ 0074: “At block 508, extracted assets and other relevant information are used as input to an asset relationship establishing process to build an asset hierarchy”);
configuring one or more data templates based on consolidation of the validated first data and the extracted second data; and (¶ 0153: “the HMI asset model includes the plant assets and relationships (asset hierarchy), ontological knowledge base, and various plant engineering data sources discussed previously.” ¶ 0154: “Once the HMI asset model 3500 (or rather the content thereof) has been established, the model may be used in developing an HMI according to embodiments of the present disclosure. Developing an HMI based on the HMI asset model can involve creating templates 3514 for the various assets (e.g., instruments, equipment, composites and combinations thereof, etc.). These templates may include, for example, level symbols (e.g., Level 2, Level 3, etc.), detailed displays of instruments and equipment, instrument attributes and actions, animated scripts, control engineering templates, consolidated details of pumps, valves, and the like.”);
rendering, on a display, the one or more data templates for the corresponding assets (¶ 0156: “The HMI developed from the HMI asset model can then be used by a user to monitor various displays and easily find the root cause of alarm” see Figures 36 and 37 ¶ 0157: “Referring next to FIG. 36, in some embodiments, the HMI according to the present disclosure can process alarms raised by various devices and automatically identify the root causes of the alarms.”);
Claims 8 and 15:
The limitations of claims 8 and 15 (¶ 0174) encompass substantially the same scope as claim 1. Accordingly, those similar limitations are rejected in substantially the same manner as claim 1, as described above. The following are the limitations of claim 8 that differ from claim 1.
Sinha as shown discloses a system for extracting data from one or more documents associated with one or more assets in a facility, the system comprising:
a processor, a memory communicatively coupled to the processor, wherein the memory comprises one or more instructions which when executed by the processor, cause the processor to: (¶ 0170: “a special purpose computing device in the form of a conventional computer, including a processing unit, a system memory, and a system bus that couples various system components including the system memory to the processing unit.”);
Claims 2, 9 and 16:
Sinha as shown discloses the following limitations:
wherein retrieving the one or more documents from the one or more data sources comprises: retrieving at least one of: one or more engineering documents, one or more process flow diagrams (PFDs), one or more piping and instrumentation diagrams (P&IDs), and one or more datasheets from the one or more data sources (¶ 0073: The HMI application 328 generally begins at block 504 with inputting data from various plant engineering data sources, including plant engineering diagrams (e.g., PI&D, PFD, etc.), into the industrial plant control system 300.” See also figures 19-29, ¶ 0134: “FIGS. 19-29 show exemplary plant engineering data sources that may be used to generate an asset hierarchy and/or a knowledge graph.”);
Claims 3, 10 and 17:
Sinha as shown discloses the following limitations:
wherein processing the at least one document using the first processing technique comprises: analyzing one or more textual representations in proximity to one or more symbolic representations in the at least one document using the first processing technique (¶ 0123: “For the symbol at 1702, for example, the nearest tag inside the symbol is “TIC 1803.” For a given symbol, all words having a distance greater than, say, 1.2 times the width of the symbol (i.e., words associated with other symbols) can be ignored. Then, the words are checked (e.g., using a rules engine) against a list of words from an industry standard like ISA to determine whether the words represent, for example, an alarm, a signal, a tag, or the like. If the word is a tag, then the tag number of the nearest tag is assigned to that word. For the symbol at 1702, for example, the tag number “1803” is assigned to the words “TE” and “TT” to produce “TE 1803” and “TT 1803.” These tagged words are then added to the list of tags for the symbol. The tags for a given symbol include all the tags inside and outside the symbol.” See also figure 17);
wherein the first processing technique corresponds to one or more image processing techniques based on machine learning and natural language processing (¶ 0063: Image converter 312 is configured to convert diagrams 200 to an image format. In some embodiments, image converter 312 obtains diagrams 200 in a Portable Document File (PDF) or other electronic data format and converts the diagrams to another image format, such as Portable Network Graphics (PNG), Joint Photographic Experts Group (JPEG), Graphics Interchange Format (GIF), and the like.” ¶ 0064: “Filtering algorithms 314 are configured to process the compute image to obtain an approximate size of the symbols therein. […] Obtaining the approximate size of the symbols helps normalize the symbols for machine learning purposes (via intelligent processing 326),” ¶ 0065: “Symbol extractor 316 is configured to detect the symbols extracted from the images.” See also ¶ 0137: “ the system uses Named Entity Recognition (NER) techniques to extract domain entities, such as pumps, valves, locations, alarm conditions, and the like, from the process narratives.”);
determining if the proximity of the one or more textual representations satisfies one or more first thresholds, wherein the one or more first thresholds are defined relative to size of the one or more symbolic representations, (¶ 0064: Filtering algorithms 314 are configured to process the compute image to obtain an approximate size of the symbols therein.”);
a type of the one or more symbolic representations, (¶ 0065: “symbol extractor 316 applies image processing algorithms to identify probable regions of symbols in the images, then detects the symbol types and locations in the images via a gross symbol identification technique.”);
co-ordinates of a corresponding textual representation in the at least one document, and a distance between at least one symbolic representation and the corresponding textual representation (¶ 0096: “To identify information associated with connector 902, a bounding box (not expressly shown) may be generated around text 910 inside connector 902 to determine if it is a connection identifier. If the bounding box is contained by the bounding box of connector 902, the text may be identified as connection identifier 910. Text situated above the bounding box of connector 902 within, for example, four times the distance between parallel lines of connector 902 (e.g., the two parallel lines connected to the arrowhead), may be identified as destination identifier contained in 908. In some embodiments, instead or in addition, text before and after the word “To” may be identified as a material name and destination identifier 908, respectively. In this context, the material name may identify the material flowing through a pipe or other material carrier.”);
and identifying the one or more textual representations to be the one or more instrument tags if the proximity of the one or more textual representations satisfies one or more first thresholds (¶ 0123: “For the symbol at 1702, for example, the nearest tag inside the symbol is “TIC 1803.” For a given symbol, all words having a distance greater than, say, 1.2 times the width of the symbol (i.e., words associated with other symbols) can be ignored. Then, the words are checked (e.g., using a rules engine) against a list of words from an industry standard like ISA to determine whether the words represent, for example, an alarm, a signal, a tag, or the like. If the word is a tag, then the tag number of the nearest tag is assigned to that word. For the symbol at 1702, for example, the tag number “1803” is assigned to the words “TE” and “TT” to produce “TE 1803” and “TT 1803.” These tagged words are then added to the list of tags for the symbol. The tags for a given symbol include all the tags inside and outside the symbol.” See also figure 17);
Claims 6 and 13:
Sinha as shown discloses the following limitations:
wherein configuring the one or more data templates comprises filling one or more fields of corresponding data templates using consolidation of the validated first data and the extracted second data (Figures 36-40B illustrates the HMI screen);
Claims 7, 14 and 20:
Sinha as shown discloses the following limitations:
deriving one or more units of measurements and one or more operating limits associated with the corresponding assets based at least on the second data (Figure 39, note the unit of measurement TI 30°C);
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 11 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sinha et al., (US 2022/0171891 A1) hereinafter “Sinha” as applied to claim 1, further in view of Boman et al., (US 9,495,614 B1) hereinafter “Boman”.
Claims 4, 11 and 18:
Sinha as shown discloses the following limitations:
wherein validating the first data comprises: validating the first data using a first validation technique of the one or more validation techniques, wherein the first validation technique comprises: comparing at least one instrument tag of the one or more instrument tags with one or more tags provided by one or more users and (Engineering data sources provided by one or more users as shown in Figure 19 illustrates Alarm data with tags and Figure 23 illustrates Field Wiring index with tags that are used to generate an asset hierarchy and/or knowledge graph, see also ¶0068 “the rules are based on ISA symbol standards”);
validating the first data using a second validation technique of the one or more validation techniques, wherein the second validation technique comprises: verification of the at least one instrument tag using one or more rules (¶ 0068: “Rules engine 322 is configured to verify extracted tags from the compute image based on one or more rules. […] Exemplary major compliance checks include, but are not limited to, verifying that the symbol is one of the valid types (e.g., field device, control room display, etc.) and verifying that the tag name has one or more identification letters. Exemplary minor compliance checks include, but are not limited to, verifying that identification letters in a tag name do not contain any numerical digits and the tag number in a tag name does not contain any alphabet characters except at the end.”);
co-ordinates of corresponding textual representation in the at least one document, (¶ 0096: “To identify information associated with connector 902, a bounding box (not expressly shown) may be generated around text 910 inside connector 902 to determine if it is a connection identifier. “)
and one or more material flow directions; and (¶ 0085: “The contents of squares 708, 710 may be analyzed using a machine learning model or algorithm trained to determine the presence of an arrowhead 712 and the direction of the arrowhead.”);
Sinha is silent with regard to the following limitations. However, Boman in an analogous art of data verification management for the purpose of providing the following limitations as shown does:
determining a first validation score for the at least one instrument tag (col. 16, lines 26-33: “can provide a similarity score or other measure indicating how similar the descriptor labels are to the recognized labels (or recognized image features). For example, if only portions of the descriptor labels are the same as recognized labels (e.g., portions of names, terms, or other words/phrases), then an appropriate similarly score can be assigned indicating the amount of similarity (e.g., 50%, 30%, etc.).”);
determining a second validation score for the at least one instrument tag (col. 18, lines 40-50: “can examine weights or other indicators which were previously associated with descriptor labels and which can be taken into consideration as a signal assisting determination of confidence of matches between descriptor labels and recognized labels, and/or for determination of a type of verification for a descriptor label. Matches may be able to be disregarded and/or weighted for accuracy based on such indicators. For example, an associated distance indicator can indicate the physical distance of a feature described in the descriptor label to the associated location,”);
and determining a validation score for the at least one instrument tag based on the first validation score and the second validation score (col. 13, lines 63-67 to col. 14, lines 1-3: “Some object recognition techniques can assign confidence scores or measures to indicate how confidently that features (e.g., objects) in the image have been recognized, how similar the feature is to known objects or other patterns, etc. In some implementations, an image feature can be categorized as recognized/identified (e.g., satisfies (e.g., is above) a predetermined confidence threshold), or unrecognized/unidentified (e.g., does not satisfy the confidence threshold).”);
Both Sinha and Boman teach data extraction management. Sinha teaches in ¶ 0142 “Semantic validation refers to the process of verifying that the data elements are logically valid.” Boman teaches in the Abstract “The verified labels are determined to describe at least one of the one or more recognized image features depicted in the image based on the comparing of the recognized image features with the descriptor labels.” Thus, they are deemed to be analogous references as they are reasonably pertinent to each other and are directed towards solving similar problems within the same environment. One of ordinary skill in the art would have recognized that applying the known technique of Boman would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Boman to the teaching of Sinha would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such as determining a first validation score for the at least one instrument tag and determining a second validation score for the at least one instrument tag; and determining a validation score for the at least one instrument tag based on the first validation score and the second validation score into similar systems. Further, as noted by Boman “a technical effect of one or more described implementations is that search, organization, and access of images based on associated image labels is reduced in time and resources expended to obtain accurate results.” (Boman, col. 4, lines 11-15).
Claims 5, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Sinha et al., (US 2022/0171891 A1) hereinafter “Sinha” as applied to claim 1, further in view of Kang et al., (US 11,756,323 B2) hereinafter “Kang”.
Claims 5, 12 and 19:
Sinha teaches in ¶ 0067: “Tag extractor 320 is configured to extract the tag component of a symbol in the compute image, such as a tag name and tag number. […] tag extractor 320 performs character recognition using machine learning techniques”. Sinha is silent with regard to the following limitations. However, Kang in an analogous art of data extraction management for the purpose of providing the following limitations as shown does:
wherein processing the at least one document using the second processing technique comprises: analyzing one or more textual representations associated with one or more predefined shapes in the at least one document by the second processing technique, wherein the second processing technique corresponds to optical character recognition (OCR) (col. 8 lines: 52-58: “a region where text is present is calculated by calculating an aspect ratio in the imaged P&ID drawing from which the symbol is removed, and text is recognized and extracted from the corresponding region (S230). In order to recognize the text, an existing text recognition program such as optical character reader (OCR) may be used.”);
determining if the one or more textual representations associated with the one or more predefined shapes satisfies one or more second thresholds, wherein the one or more second thresholds are defined relative to size of the one or more predefined shapes, a type of the one or more predefined shapes, and co-ordinates of a corresponding textual representation in the at least one document; and identifying the one or more textual representations to be the second data if the one or more textual representations satisfies one or more second thresholds (col. 9, lines 4-20: “if the recognized portion deviates a predetermined aspect ratio of a bounding box, the corresponding portion is removed, and if the recognized portion is within the range of the predetermined aspect ratio, the portion is left as a text region. The portion recognized as the text region is dilated to a predetermined threshold value and if a next recognized portion is determined as a text region, the recognized portion is left and, in this manner, a contour bounding box of the entire text region is generated to extract the entire text. The reason for setting and extracting the corresponding text region by a design information unit is to recognize the corresponding text region as a design information unit and to classify and associate the attribute information extracted from the text […] the text is recognized by applying the OCR to the text in the image of the extracted region.” Col. 9, lines 46-48: “Properties of the text detected in the drawing region is divided into line number, size, tag number, instrument type, serial number, P&ID name, and the like.”);
Both Sinha and Kang teach data extraction management. Sinha teaches in ¶ 0006 “ automatically, and through the use of machine learning (ML) models and algorithms, extract plant assets from engineering diagrams and other plant engineering data sources.” Kang teaches in the Abstract “ a method of recognizing and classifying design information by automatically digitizing an imaged P&ID drawing to digitize design information by totaling design elements with high accuracy within a short time.” Thus, they are deemed to be analogous references as they are reasonably pertinent to each other and are directed towards solving similar problems within the same environment. One of ordinary skill in the art would have recognized that applying the known technique of Kang would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Kang to the teaching of Sinha would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such as wherein processing the at least one document using the second processing technique comprises: analyzing one or more textual representations associated with one or more predefined shapes in the at least one document by the second processing technique, wherein the second processing technique corresponds to optical character recognition (OCR); determining if the one or more textual representations associated with the one or more predefined shapes satisfies one or more second thresholds, wherein the one or more second thresholds are defined relative to size of the one or more predefined shapes, a type of the one or more predefined shapes, and co-ordinates of a corresponding textual representation in the at least one document; and identifying the one or more textual representations to be the second data if the one or more textual representations satisfies one or more second thresholds into similar systems. Further, as noted by Kang “If a drawing is automatically digitized through the above method, it is possible to automatically generate most work such as the generation of a drawing, material calculation, equipment list as basic design information, a line list, instrument list calculation, and the like by totaling design elements with high accuracy within a short time, productivity at work may be improved by excluding simple and repetitive work of calculating design elements manually by high-quality engineers. […] if a drawing is automatically generated using data generated by the above method, design product compatibility may be maintained to improve design quality, as compared to the existing method of directly creating a drawing in the imaged P&ID. This solves problems such as wasting of time, missing items, misdescription, and the like of a plant engineering company, which may occur from a drawing drafted by checking one by one with naked eyes in the related art.” (Kang, col. 4, lines 38-55).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NADJA CHONG whose telephone number is (571)270-3939. The examiner can normally be reached on Monday-Friday 8:00 am - 2:00 pm ET, Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, RUTAO WU can be reached on 571.272.6045. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NADJA N CHONG CRUZ/
Primary Examiner, Art Unit 3623
Prosecution Insights