DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103, which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over US 20110060584 A1 (Ferrucc), in view of Yehia, E., Boshnak, H., AbdelGaber, S., Abdo, A. and Elzanfaly, D.S., 2019. Ontology-based clinical information extraction from physician’s free-text notes. Journal of biomedical informatics, 98, p.103276 (Yehia).
Regarding Claim 1 and 14:
A page processing method, comprising: obtaining a picture of a page; and acquiring checked-and-corrected character information in the picture of the page; and outputting the checked-and-corrected character information (Ferrucci: [0002], "computers are used to convert scanned images of text into text. Examples of such processing include optical character recognition (OCR) that converts paper documents into digital form by scanning"; [0018], "digital text 100... produced by one or more of the devices such as an OCR device 101” and "generates corrected text 150"),
wherein acquiring checked-and-corrected character information in the picture of the page comprising: acquiring a picture and extracting at least one piece of character information in the picture; and checking-and-correcting the at least one piece of character information based on a knowledge graph (Ferrucci: extracting text strings and automatically checking and correcting them based on a knowledge graph (formal ontology/fact repository), e.g., [0012], "Many OCR errors such as misinterpreted names and locations... may be automatically corrected by using facts"; [0018], "transformed into a corrected text 150 using a collection of facts, for instance, contained in one or more various repositories"; Ferrucci further specifies this graph structure, disclosing in [0025], "In one embodiment, a formal ontology may be utilized").
Ferrucci does not teach explicitly on using the claimed method in medical image. However. Yehia teaches (Yehia: teaches confirming medical images (radiology) from free text via a medical knowledge graph (ontology) and adding them to a structured combination (Electronic Health Record), e.g., Section 3, "An Ontology-based Clinical information extraction system (OB-CIE) is proposed to extract the clinical information from free-text clinical notes and convert them into a structured information". Yehia confirms medical image entities via the ontology, disclosing: Section 3.3, "Diagnostic tests class consists of radiology and laboratory tests". Yehia adds this to a structured combination, Section 3.4, "a set of rules were generated manually for mapping PCD ontology structure to the EHR database structure").
It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention was made to modify Ferrucci with Yehia's clinical ontology methodologies to identify and structure unstructured medical images/radiology data into an organized Electronic Health Record database environment.
Regarding Claim 2, Ferrucci as modified further teaches:
The page processing method according to claim 1, wherein the checking-and-correcting the at least one piece of character information based on a knowledge graph comprises: identifying character information having an error in the at least one piece of character information based on the knowledge graph; and correcting the character information having an error based on the knowledge graph (Ferrucci: teaches verifying extracted text against the knowledge graph to locate errors and correcting them based on the graph, e.g., [0026]-[0027], "factual consistency is checked by applying a model of factual consistency in order to identify those recognized facts which are inconsistent with the body of repository facts"; [0028], "Potential corrections are automatically evaluated to identify the most probable correction").
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over US 20110060584 A1 (Ferrucc), in view of Yehia, E., Boshnak, H., AbdelGaber, S., Abdo, A. and Elzanfaly, D.S., 2019. Ontology-based clinical information extraction from physician’s free-text notes. Journal of biomedical informatics, 98, p.103276 (Yehia) and in further view of US 20110052063 A1 (McAuley).
Regarding Claim 3, Ferrucci as modified does not teaches explicitly on hierarchical tree evaluation. However, McAuley teaches:
The page processing method according to claim 2, wherein the at least one piece of character information comprises a plurality of pieces of character information; and the identifying character information having an error in the at least one piece of character information based on the knowledge graph comprises: obtaining a plurality of entities respectively based on the plurality of pieces of character information in the picture, and selecting an entity from the plurality of entities as a to-be-checked-and-corrected entity for a process of determining whether character information corresponding to the to-be-checked-and-corrected entity has an error; and determining whether the character information corresponding to the to-be-checked-and-corrected entity has an error according to a hierarchical structure of the knowledge graph, and identifying the character information corresponding to the to-be-checked-and-corrected entity as the character information having an error when the character information corresponding to the to-be-checked-and-corrected entity has an error (McAuley: teaches determining errors and validating image text entities according to a hierarchical graph structure, e.g., [0008], "recursively partitioning an image into a tree of image regions"; [0027], "class labels are selected from a hierarchy of classes 30... applied via pairwise (edge) potentials along the edges of the tree structure to ensure that nodes on higher levels of the tree structure... will always be assigned to a class that is at least as generic as its child nodes at lower levels"; [0028], "In the mathematical notation used herein, class inheritance is represented by the notation a<b where class a is less specific than class b... For the hierarchy of FIG. 4, in all cases the inheritance c<background holds, where c denotes any class other than 'background'”; [0030], "The inheritance constraints can be understood as imposing on the class c_{k+1} of an image region x_{k+1} and a class c_k of a larger image region x_k that subsumes the image region x_{k+1} the inheritance constraint that either c_k < c_{k+1} or c_k = c_{k+1}"; in addition, Ferrucci: [0022]-[0026], "Facts extraction module 122 extracts facts represented in the text 100... fact extraction (also referred to as information extraction) retrieves information from natural language text, that is, automatically extracts structured information."; and "a mapping is produced by recognizing the components of the ontology (entities, relations, etc.) appearing in both F(T) and F(T,R)... factual consistency is checked by applying a model of factual consistency in order to identify those recognized facts which are inconsistent with the body of repository facts."; and Yehia: Sec 3.3.1, "Named Entity Recognition identifies specific words or phrases ('entities') and categorizes them as persons, locations, diseases, or medication. The common NER tasks are detection of entities and determining entities types.", among others).
It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention was made to modify Ferrucci with the hierarchical tree evaluation taught by McAuley to the ontology correction method of Ferrucci to improve the accuracy of classifying OCR entities based on established parent/child semantic structures.
Claims 4-6 rejected under 35 U.S.C. 103 as being unpatentable over US 20110060584 A1 (Ferrucc), in view of Yehia, E., Boshnak, H., AbdelGaber, S., Abdo, A. and Elzanfaly, D.S., 2019. Ontology-based clinical information extraction from physician’s free-text notes. Journal of biomedical informatics, 98, p.103276 (Yehia) and in further view of US 20110052063 A1 (McAuley) and US 20160019430 A1 (Stella).
Regarding Claim 4, Ferrucci as modified does not teaches explicitly on the explicit mathematical similarity and error thresholds. However, Stella teaches:
The page processing method according to claim 3, wherein the determining whether the character information corresponding to the to-be-checked-and-corrected entity has an error according to a hierarchical structure of the knowledge graph comprises: grading the plurality of entities according to the hierarchical structure of the knowledge graph; determining a level of the to-be-checked-and-corrected entity in the hierarchical structure of the knowledge graph; calculating a similarity between the to-be-checked-and-corrected entity and each entity that is at a same level and has a same relationship as the to-be-checked-and-corrected entity in the knowledge graph, to obtain a plurality of entity similarities related to the to-be-checked-and-corrected entity; and when a maximum value of the plurality of entity similarities is smaller than a predetermined entity similarity threshold, determining that the to-be-checked-and-corrected entity is a to-be-checked-and-corrected entity having an error and that the character information corresponding to the to-be-checked-and-corrected entity has an error (Stella: teaches utilizing distance/similarity algorithms and error thresholds to discard incorrect OCR matches, e.g., [0011], "calculating a set of errors between the first and second set of candidate words and the OCR machine code... and selecting a preferred candidate word... with the smallest set of errors"; [0020], [0025], "The minimum error may exclude a character shape class deletion..."; "The calculating/determining a set of errors between the first set of candidate words and the registered OCR machine code comprises a Levenshtein-Distance (LD) measure"; [0090], LD calculation; McAuley: teaches grading entities by level and tracking pairwise node relationships, [0023]-[0024], "tree structure including edges and nodes... pairwise potentials between a pair of nodes connected by an edge are used to encode inheritance constraints defined by a hierarchy of classes").
It would have been obvious for one of ordinary skill in the art before the effective filling date of the claimed invention was made to modify Ferrucci with the explicit mathematical similarity and error thresholds to to provide a precise, quantified metric for identifying non-matching OCR errors.
Regarding Claim 5, Ferrucci as modified further teaches:
The page processing method according to claim 4, wherein the correcting the character information having an error based on the knowledge graph comprises: determining a number of all entities that are at the same level and have the same relationship as the to-be-checked-and-corrected entity in the knowledge graph, to obtain an entity number; when the entity number is equal to 1, directly replacing character information corresponding to the to-be-checked-and-corrected entity having an error with character information corresponding to the entity that is at the same level and has the same relationship as the to-be-checked-and-corrected entity in the knowledge graph, or calculating a probability that the entity that is at the same level and has the same relationship as the to-be-checked-and-corrected entity is the to-be-checked-and-corrected entity having an error, to obtain an entity probability, and when the entity probability is larger than a predetermined entity probability, replacing character information corresponding to the to-be-checked-and-corrected entity having an error with character information corresponding to the entity that is at the same level and has the same relationship as the to-be-checked-and-corrected entity; and when the entity number is larger than 1, performing a following method comprising: determining at least two candidate entities based on the plurality of entity similarities; calculating a probability that each of the at least two candidate entities is the to-be-checked-and-corrected entity having an error, to obtain a candidate probability for the each of the at least two candidate entities; and replacing character information corresponding to the to-be-checked-and-corrected entity having an error with character information corresponding to a candidate entity corresponding to a maximum candidate probability (Stella: teaches generating multiple candidates, calculating probability (minimum set of errors), and replacing the text with the candidate holding the maximum probability, e.g., [0011], "selecting—from a dictionary—a first set of candidate words... selecting... a second set of candidate words... calculating a set of errors... and selecting a preferred candidate word... with the smallest set of errors."; Stella further calculates the probabilities (set of errors) to find the maximum probable match, [0123]-[0132], "c1=‘imitation’ errors=1... c2=‘immigration’ errors=2... c4=‘irritation’ errors=3").
Regarding Claim 6, Ferrucci as modified further teaches:
The page processing method according to claim 5, wherein the determining at least two candidate entities based on the plurality of entity similarities comprises: sorting all entities that are at the same level and have the same relationship as the to-be-checked-and-corrected entity in the knowledge graph based on the plurality of entity similarities in a descending order to obtain a sequence, selecting a predetermined number of entities at a beginning of the sequence as the at least two candidate entities; or sorting all entities that are at the same level and have the same relationship as the to-be-checked-and-corrected entity in the knowledge graph based on the plurality of entity similarities in an ascending order to obtain a sequence, and selecting a predetermined number of entities at an end of the sequence as the at least two candidate entities (Stella: teaches sorting candidate entities ascending/descending based on the calculated error (similarity) to rank them, e.g., [0123]-[0132], "(1) c1=‘imitation’ errors=1... (2) c2=‘immigration’ errors=2... (4) c4=‘irritation’ errors=3").
Allowable Subject Matter
The Claims 7 is objected to as being dependent upon a rejected base claim, but are potentially allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 8-13 depend on Claim 7, therefore, Claims 8-13 are objected for the same reasons as Claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHITONG CHEN whose telephone number is (571) 270-1936. The examiner can normally be reached on M-F 9:30am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Yuwen Pan can be reached on 571-272-7855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZHITONG CHEN/
Primary Examiner, Art Unit 2649
Prosecution Insights