DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 5 and 23 are objected to because of the following informalities:
Claim 5 does not end with a period.
Claim 23 does not end with a period.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 30 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the broadest reasonable interpretation of a “machine-storage medium” can encompass non-statutory transitory forms of signal transmission, such as a propagating electrical or electromagnetic signal per se.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 4 – 5, 7 – 11, 13 – 14, 19, 22 – 23 and 25 – 30 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Li (US Patent Application Publication No. 2020/0151591).
Regarding claim 1, Li discloses a system comprising:
at least one hardware processor (Paragraph 0062, lines 4-7, "System 200 can comprise a classification and extraction engine (CEE) 205, which in turn can comprise a CEE processor 210 in communication with a memory 215.");
and at least one memory storing instructions that cause the at least one hardware processor to perform operations (Paragraph 0062, lines 13-14, "Processor 210 may cooperate with the memory 215 to execute instructions.") comprising:
processing a current electronic document, using a set of machine-learning models, to extract a set of values for a set of data points based on a schema, the schema describing the set of data points to be extracted from electronic documents (Paragraph 0055, lines 1-8, "There can be many different types of documents. Even within a given document type, there may be variations due to versions of that document type and document and/or image processing such as Optical Character Recognition (OCR). Machine learning models (MLMs) can be used to process such diverse documents to classify the documents and/or to extract field values from the document fields in the documents."; Paragraph 0056, lines 1-8, "MLMs can be configured to receive a computer-readable or machine-readable input and in response produce a predicted output. For example, the input can comprise one or more computer-readable tokens corresponding to first document 105 and the predicted output can comprise a classification (i.e. the document type) of first document 105 and/or one or more field values 115 and 125 extracted from first document 105."; Paragraph 0092, lines 1-15, "In addition, in some examples, after classifying a document or a document page, MLM 220 can then be used to classify each token (or each character if the MLM operates at the character level) in the document into one of a pre-defined set of fields for this document's document class. There may be multiple non-overlapping instances of a field within a document. The most recently trained field prediction MLM for the current document class stored in memory, such as in memory 215, can be used. In some examples, MLM 220 can comprise a MLM configured to perform both classification and field value extraction. In other examples, MLM 220 can comprise more than one separate MLMs: one or more MLMs to perform the document classification and one or more other MLMs to perform field value extraction."; Processing documents to extract field values from the document fields in the documents reads on processing a current electronic document to extract a set of values for a set of data points based on a schema, one or more machine learning models to perform field value extraction reads on processing a current electronic document using a set of machine-learning models, and classifying each token in the document into one of a pre-defined set of fields for the document's document class reads on a schema describing the set of data points to be extracted from electronic documents.);
determining whether to select the current electronic document for human validation based on the schema (Paragraph 0110, lines 1-5, "In some examples, CEE 205 can determine whether a predicted output is to be communicated to review interface 245 for review by reviewer 250. This determination can be based on the confidence score associated with the predicted output."; Determining whether a predicted output is to be communicated for review by a reviewer based on the confidence score associated with the predicted output reads on determining whether to select the current electronic document for human validation based on a schema.);
and in response to determining to select the current electronic document for human validation based on the schema, adding the current electronic document to a human validation queue (Paragraph 0108, lines 1-9, "If a document classification prediction was altered by the reviewer, new field predictions for the document or its resulting sub-documents can be generated and then verified by the reviewer. In this case, the review interface can either require the reviewer to wait for new field predictions for verification, or queue the document for field verification after the field predictions have been generated while moving the reviewer to the next available document for field verification."; Paragraph 0110, lines 1-5, "In some examples, CEE 205 can determine whether a predicted output is to be communicated to review interface 245 for review by reviewer 250. This determination can be based on the confidence score associated with the predicted output."; Determining whether a predicted output is to be communicated for review by a reviewer based on the confidence score associated with the predicted output reads on determining to select the current electronic document for human validation based on the schema, and communicating a predicted output to a review interface for review by reviewer, where the document can be queued for verification, reads on adding the current electronic document to a human validation queue.).
Regarding claim 4, Li discloses the system as claimed in claim 1.
Li further discloses:
wherein the operations comprise: in response to determining to not select the current electronic document for human validation based on the schema, determining whether to select the current electronic document for human validation as a random sample based on a random sample value (Paragraph 0117, lines 1-7, "In addition, as part of DETT in some examples the following two methods can be used individually or together to help verify the validity or the accuracy predictions: first, a random sample of predictions that would have been automatically accepted can be instead sent for review and the error rate of these samples can be compared with the expected error rate."; Sending a random sample of predictions for review that would have been automatically accepted reads on determining whether to select the current electronic document for human validation as a random sample based on a random sample value in response to determining to not select the current electronic document for human validation based on the schema.).
Regarding claim 5, Li discloses the system as claimed in claim 4.
Li further discloses:
wherein the operations comprise: in response to determining to select the current electronic document for human validation as the random sample based on the random sample value, adding the current electronic document to the human validation queue (Paragraph 0108, lines 1-9, "If a document classification prediction was altered by the reviewer, new field predictions for the document or its resulting sub-documents can be generated and then verified by the reviewer. In this case, the review interface can either require the reviewer to wait for new field predictions for verification, or queue the document for field verification after the field predictions have been generated while moving the reviewer to the next available document for field verification."; Paragraph 0110, lines 1-5, "In some examples, CEE 205 can determine whether a predicted output is to be communicated to review interface 245 for review by reviewer 250. This determination can be based on the confidence score associated with the predicted output."; Paragraph 0117, lines 1-7, "In addition, as part of DETT in some examples the following two methods can be used individually or together to help verify the validity or the accuracy predictions: first, a random sample of predictions that would have been automatically accepted can be instead sent for review and the error rate of these samples can be compared with the expected error rate."; Sending a random sample of predictions for review that would have been automatically accepted reads on determining whether to select the current electronic document for human validation as a random sample based on a random sample value, and communicating a predicted output to a review interface for review by reviewer, where the document can be queued for verification, reads on adding the current electronic document to a human validation queue.).
Regarding claim 7, Li discloses the system as claimed in claim 4.
Li further discloses:
wherein the schema defines the random sample value (Paragraph 0117, lines 1-7, "In addition, as part of DETT in some examples the following two methods can be used individually or together to help verify the validity or the accuracy predictions: first, a random sample of predictions that would have been automatically accepted can be instead sent for review and the error rate of these samples can be compared with the expected error rate."; Sending a random sample of predictions for review that would have been automatically accepted reads on determining whether to select the current electronic document for human validation as a random sample based on a schema defined random sample value.).
Regarding claim 8, Li discloses the system as claimed in claim 1.
Li further discloses:
wherein the operations comprise: based on the current electronic document being added to the human validation queue, providing a user with access to a human validation interface (Paragraph 0065, lines 1-11, "System 200 can be in communication with a review interface 245. Review interface 245 can in turn be in communication a reviewer 250. In some examples, CEE 205 can be in communication with reviewer 250 via review interface 245. CEE 205 can send a predicted output to review interface 245 where the predicted output can be reviewed by reviewer 250. The review can comprise, for example, a confirmation/verification, a rejection, an alteration, and/or a correction of the predicted output. In some examples, upon review reviewer 250 can provide feedback on the predicted output."; Paragraph 0068, lines 1-4, "At box 305, first document 105 (shown in FIG. 1) from a set of documents 100 can be sent to a GUI. In other examples, document 105 can be sent to a type of review interface 245 other than a GUI."; Sending a predicted output to a review interface where the predicted output can be reviewed by a reviewer reads on providing a user with access to a human validation interface.).
Regarding claim 9, Li discloses the system as claimed in claim 8.
Li further discloses:
wherein one or more user interface elements of the human validation interface are defined by the schema (Paragraph 0065, lines 1-11, "System 200 can be in communication with a review interface 245. Review interface 245 can in turn be in communication a reviewer 250. In some examples, CEE 205 can be in communication with reviewer 250 via review interface 245. CEE 205 can send a predicted output to review interface 245 where the predicted output can be reviewed by reviewer 250. The review can comprise, for example, a confirmation/verification, a rejection, an alteration, and/or a correction of the predicted output. In some examples, upon review reviewer 250 can provide feedback on the predicted output."; Paragraph 0068, lines 1-4, "At box 305, first document 105 (shown in FIG. 1) from a set of documents 100 can be sent to a GUI. In other examples, document 105 can be sent to a type of review interface 245 other than a GUI."; A review interface reads on a human validation interface, and the review comprising a verification, a rejection, an alteration, a correction, or feedback on the predicted output reads on one or more user interface elements of the human validation interface being defined by the schema.).
Regarding claim 10, Li discloses the system as claimed in claim 1.
Li further discloses:
wherein the operations comprise: causing presentation, by a human validation interface, of an individual electronic document from the human validation queue, the individual electronic document being presented with a set of extracted data point values associated with the individual electronic document (Paragraph 0104, lines 1-10, "In some examples, field review/verification can occur as long as there is a classified document that is assigned to the reviewer to verify the extracted, i.e. predicted, field values. This can begin when initiated by the reviewer or immediately after one or more document classifications have been verified by the reviewer. For field verification, the GUI can present a single document at a time. Field predictions can be shown as a list of fields and predicted values and/or by highlighting the locations of the predicted field extractions on a preview of the document."; A graphical user interface presenting a document to a reviewer reads on causing presentation of an individual electronic document by a human validation interface, and field predictions being shown as a list of fields and predicted values by highlighting the locations of the predicted field extractions on a preview of the document reads on the individual electronic document being presented with a set of extracted data point values associated with the individual electronic document.).
Regarding claim 11, Li discloses the system as claimed in claim 10.
Li further discloses:
wherein the operations comprise: receiving, by the human validation interface, user feedback for the individual electronic document (Paragraph 0065, lines 1-11, "System 200 can be in communication with a review interface 245. Review interface 245 can in turn be in communication a reviewer 250. In some examples, CEE 205 can be in communication with reviewer 250 via review interface 245. CEE 205 can send a predicted output to review interface 245 where the predicted output can be reviewed by reviewer 250. The review can comprise, for example, a confirmation/verification, a rejection, an alteration, and/or a correction of the predicted output. In some examples, upon review reviewer 250 can provide feedback on the predicted output."; Paragraph 0068, lines 1-4, "At box 305, first document 105 (shown in FIG. 1) from a set of documents 100 can be sent to a GUI. In other examples, document 105 can be sent to a type of review interface 245 other than a GUI."; A reviewer providing feedback on the predicted output reads on receiving user feedback for the individual electronic document.).
Regarding claim 13, Li discloses the system as claimed in claim 11.
Li further discloses:
wherein the operations comprise: adjusting at least one machine-learning model in the set of machine-learning models based on the user feedback (Paragraph 0117, lines 1-11, "In addition, as part of DETT in some examples the following two methods can be used individually or together to help verify the validity or the accuracy predictions: first, a random sample of predictions that would have been automatically accepted can be instead sent for review and the error rate of these samples can be compared with the expected error rate. Second, where errors can be subsequently detected by a different downstream system or process, these errors can be reported back to the system. This information can be added to the accuracy training data for when the accuracy model is updated."; Paragraph 0166, lines 1-7, "Moreover, at box 815 the prediction can be sent from the CEE to a GUI. At box 820, in turn, feedback on the prediction can be received at the CEE from the GUI. The feedback can be used to form a reviewed prediction. Furthermore, at box 825 the reviewed prediction can be added to a training dataset. In some examples, the CEE can add the reviewed prediction to the training dataset."; Adding a reviewed prediction to a training dataset, where feedback is used to form the reviewed prediction, reads on adjusting at least one machine-learning model in the set of machine-learning models based on the user feedback.).
Regarding claim 14, Li discloses the system as claimed in claim 11.
Li further discloses:
wherein the user feedback comprises at least one of: acceptance of the individual electronic document; rejection of the individual electronic document; or one or more correct values for the set of extracted data point values (Paragraph 0065, lines 1-11, "System 200 can be in communication with a review interface 245. Review interface 245 can in turn be in communication a reviewer 250. In some examples, CEE 205 can be in communication with reviewer 250 via review interface 245. CEE 205 can send a predicted output to review interface 245 where the predicted output can be reviewed by reviewer 250. The review can comprise, for example, a confirmation/verification, a rejection, an alteration, and/or a correction of the predicted output. In some examples, upon review reviewer 250 can provide feedback on the predicted output."; A review comprising a verification, a rejection, an alteration, a correction, or feedback on the predicted output reads on user feedback, a verification reads on acceptance of the individual electronic document, a rejection reads on a rejection of the individual electronic document, and a correction reads on one or more correct values for the set of extracted data point values.).
Regarding claim 19, arguments analogous to claim 1 are applicable.
Regarding claim 22, arguments analogous to claim 4 are applicable.
Regarding claim 23, arguments analogous to claim 5 are applicable.
Regarding claim 25, arguments analogous to claim 7 are applicable.
Regarding claim 26, arguments analogous to claim 8 are applicable.
Regarding claim 27, arguments analogous to claim 9 are applicable.
Regarding claim 28, arguments analogous to claim 10 are applicable.
Regarding claim 29, arguments analogous to claim 11 are applicable.
Regarding claim 30, arguments analogous to claim 1 are applicable. In addition, Li discloses a machine-storage medium, the machine-storage medium including instructions that when executed by a machine (Paragraph 0024, lines 1-6, "According to another aspect of the present specification, there is provided a non-transitory computer-readable storage medium comprising instructions executable by a processor, the instructions configured to cause the processor to perform any one or more of the methods described herein."), cause the machine to perform the steps of claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2 – 3, 6, 12, 20 – 21 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Kato et al. (US Patent No. 12,481,669), hereinafter Kato.
Regarding claim 2, Li discloses the system as claimed in claim 1.
Li further discloses:
wherein the operations comprise: in response to determining to not select the current electronic document for human validation based on the schema, storing the set of values [in a table] (Paragraph 0113, lines 1-12, "Furthermore, in some examples Defined Error Tolerance Techniques (DETT) can be used by CEE 205 to determine whether a predicted output is sent to review interface 245 to be reviewed, and/or whether the output is designated for review by an expert or non-expert reviewer. In some examples, DETT can be used by CEE 205 to set the threshold for the confidence score, which threshold can then be used to decide whether a prediction/predicted output is to be reviewed, and/or whether the review is to be by an expert or non-expert reviewer. When CEE 205 determines, using DETT, that a review is not needed, a predicted output can be automatically accepted bypassing the review."; Automatically accepting a predicted output when a determination is made that a review is not needed reads on storing the set of values in response to determining to not select the current electronic document for human validation based on the schema.).
Li does not specifically disclose: storing the set of values in a table.
Kato teaches:
storing the set of values in a table (Column 2, lines 34-48, "FIG. 1 illustrates a logical block diagram illustrating extracting data from natural language communications to populate tables, according to some embodiments. Data collaboration system 110 may be a standalone system or implemented as part of a set of service offering from a provider network, such as data collaboration service 210 discussed below with regard to FIG. 2. One aspect of data collaboration system 100 may be the use of tables, such as table(s) 130, to store, track, record, or otherwise maintain state for various use-cases, operations, tasks, or other scenarios in which multiple participants are collaborating. These table(s) 130 may be defined through requests to create the table, as well as specify features of the table, such as the columns, including the column names and types of column data to be stored."; Extracting data from natural language communications to populate tables reads on storing the set of values in a table.).
Kato is considered to be analogous to the claimed invention because it is in the same field of extracting data from documents. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Kato to extract data from natural language communications to populate tables. Doing so would allow for avoiding error-prone and time-consuming copy techniques (Kato; Column 2, lines 17-33).
Regarding claim 3, Li in view of Kato discloses the system as claimed in claim 2.
Kato further teaches:
wherein the set of values is stored in the table according to the schema, the schema defining a mapping between one or more data points and one or more columns of the table (Column 2, lines 34-48, "FIG. 1 illustrates a logical block diagram illustrating extracting data from natural language communications to populate tables, according to some embodiments. Data collaboration system 110 may be a standalone system or implemented as part of a set of service offering from a provider network, such as data collaboration service 210 discussed below with regard to FIG. 2. One aspect of data collaboration system 100 may be the use of tables, such as table(s) 130, to store, track, record, or otherwise maintain state for various use-cases, operations, tasks, or other scenarios in which multiple participants are collaborating. These table(s) 130 may be defined through requests to create the table, as well as specify features of the table, such as the columns, including the column names and types of column data to be stored."; Column 12, lines 57-66, "As indicated at 710, a communication between different participants that includes natural language data and is associated with a table managed by the data collaboration system is obtained, according to some embodiments. For example, as discussed above with regard to FIG. 3, a table can be created with a specified schema (e.g., multiple columns with respective column names and data types). The association between the table and the communication may, for instance, be indicated in an identifier for the table (or endpoint, tag, or other that associates for capturing the communication)."; Creating a table with a specified schema containing multiple columns with respective column names and data types reads on a schema defining a mapping between one or more data points and one or more columns of a table.).
Kato is considered to be analogous to the claimed invention because it is in the same field of extracting data from documents. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li in view of Kato to further incorporate the teachings of Kato to create a table with a specified schema containing multiple columns with respective column names and data types. Doing so would allow for avoiding error-prone and time-consuming copy techniques (Kato; Column 2, lines 17-33).
Regarding claim 6, Li discloses the system as claimed in claim 4.
Li further discloses:
wherein the operations comprise: in response to determining to not select the current electronic document for human validation as the random sample based on the random sample value, storing the set of values [in a table] (Paragraph 0113, lines 1-12, "Furthermore, in some examples Defined Error Tolerance Techniques (DETT) can be used by CEE 205 to determine whether a predicted output is sent to review interface 245 to be reviewed, and/or whether the output is designated for review by an expert or non-expert reviewer. In some examples, DETT can be used by CEE 205 to set the threshold for the confidence score, which threshold can then be used to decide whether a prediction/predicted output is to be reviewed, and/or whether the review is to be by an expert or non-expert reviewer. When CEE 205 determines, using DETT, that a review is not needed, a predicted output can be automatically accepted bypassing the review."; Automatically accepting a predicted output when a determination is made that a review is not needed reads on storing the set of values in response to determining to not select the current electronic document for human validation as the random sample based on the random sample value.).
Li does not specifically disclose: storing the set of values in a table.
Kato teaches:
storing the set of values in a table (Column 2, lines 34-48, "FIG. 1 illustrates a logical block diagram illustrating extracting data from natural language communications to populate tables, according to some embodiments. Data collaboration system 110 may be a standalone system or implemented as part of a set of service offering from a provider network, such as data collaboration service 210 discussed below with regard to FIG. 2. One aspect of data collaboration system 100 may be the use of tables, such as table(s) 130, to store, track, record, or otherwise maintain state for various use-cases, operations, tasks, or other scenarios in which multiple participants are collaborating. These table(s) 130 may be defined through requests to create the table, as well as specify features of the table, such as the columns, including the column names and types of column data to be stored."; Extracting data from natural language communications to populate tables reads on storing the set of values in a table.).
Kato is considered to be analogous to the claimed invention because it is in the same field of extracting data from documents. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Kato to extract data from natural language communications to populate tables. Doing so would allow for avoiding error-prone and time-consuming copy techniques (Kato; Column 2, lines 17-33).
Regarding claim 12, Li discloses the system as claimed in claim 11.
Li further discloses:
wherein the operations comprise: storing the set of extracted data point values [in a table] based on the user feedback (Paragraph 0104, lines 1-16, "In some examples, field review/verification can occur as long as there is a classified document that is assigned to the reviewer to verify the extracted, i.e. predicted, field values. This can begin when initiated by the reviewer or immediately after one or more document classifications have been verified by the reviewer. For field verification, the GUI can present a single document at a time. Field predictions can be shown as a list of fields and predicted values and/or by highlighting the locations of the predicted field extractions on a preview of the document. The reviewer can add an instance of a field for extraction that was not predicted by selecting the field from a field list and selecting the tokens on the appropriate page(s) of the document using the GUI. The GUI can show the textual value of the selected tokens and the reviewer can then make corrections to this text if needed."; The reviewer making corrections to the textual value of selected tokens reads on storing the set of extracted data point values based on the user feedback.).
Li does not specifically disclose: storing the set of extracted data point values in a table.
Kato teaches:
storing the set of extracted data point values in a table (Column 2, lines 34-48, "FIG. 1 illustrates a logical block diagram illustrating extracting data from natural language communications to populate tables, according to some embodiments. Data collaboration system 110 may be a standalone system or implemented as part of a set of service offering from a provider network, such as data collaboration service 210 discussed below with regard to FIG. 2. One aspect of data collaboration system 100 may be the use of tables, such as table(s) 130, to store, track, record, or otherwise maintain state for various use-cases, operations, tasks, or other scenarios in which multiple participants are collaborating. These table(s) 130 may be defined through requests to create the table, as well as specify features of the table, such as the columns, including the column names and types of column data to be stored."; Extracting data from natural language communications to populate tables reads on storing the set of extracted data point values in a table.).
Kato is considered to be analogous to the claimed invention because it is in the same field of extracting data from documents. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Kato to extract data from natural language communications to populate tables. Doing so would allow for avoiding error-prone and time-consuming copy techniques (Kato; Column 2, lines 17-33).
Regarding claim 20, arguments analogous to claim 2 are applicable.
Regarding claim 21, arguments analogous to claim 3 are applicable.
Regarding claim 24, arguments analogous to claim 6 are applicable.
Claims 15 – 17 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Wyle et al. (US Patent No. 11,860,950), hereinafter Wyle.
Regarding claim 15, Li discloses the system as claimed in claim 1.
Li further discloses:
wherein the determining whether to select the current electronic document for human validation based on the schema comprises: determining whether one or more values in the set of values do not match one or more related parameters described by the schema (Paragraph 0094, lines 1-6, "The output of this step can comprise zero, one or multiple instances of a set of ordered tokens (or characters) for each field defined for this document class and a (typically unit-less) metric for the prediction confidence of each token/character in each instance of each field. This metric can also be referred to as a confidence score."; Paragraph 0110, lines 1-5, "In some examples, CEE 205 can determine whether a predicted output is to be communicated to review interface 245 for review by reviewer 250. This determination can be based on the confidence score associated with the predicted output."; Determining a confidence score for the prediction confidence of each token in each instance of each field reads on determining whether one or more values in the set of values do not match one or more related parameters described by the schema.);
in response to determining that at least one value in the set of values does not match a parameter described by the schema for the at least one value, determining to select the current electronic document for human validation (Paragraph 0094, lines 1-6, "The output of this step can comprise zero, one or multiple instances of a set of ordered tokens (or characters) for each field defined for this document class and a (typically unit-less) metric for the prediction confidence of each token/character in each instance of each field. This metric can also be referred to as a confidence score."; Paragraph 0110, lines 1-5, "In some examples, CEE 205 can determine whether a predicted output is to be communicated to review interface 245 for review by reviewer 250. This determination can be based on the confidence score associated with the predicted output."; Determining whether a predicted output is to be communicated to a review interface for review by a reviewer based on a confidence score associated with the predicted output reads on determining to select the current electronic document for human validation in response to determining that at least one value in the set of values does not match a parameter described by the schema for the at least one value.).
Li does not specifically disclose: in response to determining that all values in the set of values match related parameters described by the schema, determining to not select the current electronic document for human validation.
Wyle teaches:
in response to determining that all values in the set of values match related parameters described by the schema, determining to not select the current electronic document for human validation (Column 2, lines 21-25, "The method may further comprise transmitting, by the processor, the new document with the second content to a human for verification, in response to the checking the integrity of the second content in the new document being a failure."; Column 3, lines 19-45, "The system directly removes the need for human maintained and rigid OCR templates by creating a dynamic, data extraction platform based around the ability to recognize regions of interest between similar documents. In various embodiments, the system is configured to create a generalized document automation framework that captures relevant data from documents based upon replicating historical human actions associated with a document. In general, in various embodiments and with respect to FIG. 5, the system may receive a new document having second content and regions of interest (step 505). The system may use machine vision and natural language processing to compare the new document (e.g., current year tax form) to an historical document (e.g., prior year tax form) (step 510). The historical document may have already been subject to human data extraction in an existing corpus. This is accomplished by comparing visual elements (e.g., logos, fonts, lines, and layout), and textual elements (e.g., text contained in headers, values, and tables on the document). The system may create match metrics for each of the one or more of a plurality of historical documents, wherein the match metrics are based on the comparing (step 515). The system reviews the historical documents and maps the extractions of first content to similar regions in the new document based upon both visual and text commonalities between documents. As used herein, a “match” may include an identical match, a partial match, a correlation, similar content, similar layout, etc."; Capturing relevant data from documents and determining if the extractions match extractions of content of similar regions of historical documents reads on determining that all values in the set of values match related parameters described by the schema, and transmitting the new document to a human for verification in response to the checking the integrity of the content in the new document being a failure reads on determining to not select the current electronic document for human validation when all values in the set of values match related parameters described by the schema, where checking the integrity of the content in the new document being successful reads on all values in the set of values matching related parameters described by the schema.).
Wyle is considered to be analogous to the claimed invention because it is in the same field of extracting data from documents. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Wyle to capture relevant data from documents, determine if the extractions match extractions of content of similar regions of historical documents, and transmit the new document to a human for verification only in response to the checking the integrity of the content in the new document being a failure. Doing so would allow for removing the need for human maintained and rigid optical character recognition (OCR) templates by creating a data extraction platform based around the ability to recognize regions of interest between similar documents (Wyle; Column 3, lines 19-22).
Regarding claim 16, Li in view of Wyle discloses the system as claimed in claim 15.
Li further discloses:
wherein the one or more related parameters comprise a parameter that defines a threshold for an acceptable confidence level provided by a select machine-learning model in connection with a select value extracted from a select electronic document (Paragraph 0094, lines 1-6, "The output of this step can comprise zero, one or multiple instances of a set of ordered tokens (or characters) for each field defined for this document class and a (typically unit-less) metric for the prediction confidence of each token/character in each instance of each field. This metric can also be referred to as a confidence score."; Paragraph 0110, lines 1-5, "In some examples, CEE 205 can determine whether a predicted output is to be communicated to review interface 245 for review by reviewer 250. This determination can be based on the confidence score associated with the predicted output."; Determining whether a predicted output is to be communicated to a review interface for review by a reviewer based on the confidence score associated with the predicted output reads on a parameter that defines a threshold for an acceptable confidence level provided by a select machine-learning model in connection with a select value extracted from a select electronic document.).
Regarding claim 17, Li in view of Wyle discloses the system as claimed in claim 15.
Li further discloses:
wherein the one or more related parameters comprise a parameter that defines a rule for a select value extracted from a select electronic document for a select data point (Paragraph 0083, lines 1-4, "Tokens can then be enriched with additional features by analyzing the characters in each token and the tokens before and after it using various pre-defined rules or natural language processing (NLP) techniques."; Analyzing the characters in each token and the tokens before and after it using various pre-defined rules reads on a parameter that defines a rule for a select value extracted from a select electronic document for a select data point.).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Benincasa et al. (US Patent No 11,768,884), hereinafter Benincasa.
Regarding claim 18, Li discloses the system as claimed in claim 1, but does not specifically disclose: wherein the operations comprise: performing a training process to define at least a portion of the schema, the training process comprising a user asking one or more questions relating to a training set of electronic documents.
Benincasa teaches:
wherein the operations comprise: performing a training process to define at least a portion of the schema, the training process comprising a user asking one or more questions relating to a training set of electronic documents (Column 2, lines 36-44, "According to a first aspect of the present invention, there is provided a computer system for extracting structured data from unstructured or semi-structured text in an electronic document, the system comprising: a. a graphical user interface configured to present to a user a graphical view of a document for use in training multiple data extraction models for the document, each data extraction model associated with a user defined question"; Column 16, lines 11-16, "With multiple user-defined questions, a separate I/O label sequence is determined for each question, and used to independently train a separate CRF on that specific question (not shown in FIG. 4). Training is instigated by the end-user who is defining the question(s), as described in further detail below."; Column 18, line 66 - Column 19, line 8, "The highlighted part 104 of the document 100 is associated with a question for which a model is to be trained. When a user highlights a part 104 of the document 100, a question definer 112 is provided to the user. The question defined 112 is a graphical element of the GUI comprising a drop-down menu which provides a field into which a user can type the name of the question in human readable text or select a name of a question from the drop-down menu which has already been used for defining answers in the document 100 or any other documents in the same project."; Training a data extraction model using user-defined questions reads on a training process comprising a user asking one or more questions relating to a training set of electronic documents.).
Benincasa is considered to be analogous to the claimed invention because it is in the same field of extracting data from documents. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Benincasa to train a data extraction model using user-defined questions. Doing so would allow for a data extraction model being trained on a relatively small number of documents (Benincasa; Column 2, lines 25-35).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Jayaram et al. (US Patent No. 12,400,465) teaches a method of data extraction from input data objects using a data extraction schema.
Neelamana (US Patent No. 11,734,579) teaches a method for extracting data from formatted and non-formatted documents using machine learning algorithms.
Yaramada et al. (US Patent No. 11,630,956) teaches a method for automatically extracting data from documents using multiple deep learning models.
Ko et al. (US Patent No. 11,341,354) teaches a method of receiving a document having a plurality of first text strings, extracting the plurality of first text strings from the document, and providing the extracted plurality of first text strings to machine learning models to process the strings and generate an output.
Nelson et al. (“Processing PDF documents with a human loop using Amazon Textract and Amazon Augmented AI”) teaches a method for automatically extracting text and data from documents and triggering human review of certain fields when the fields are determined to be below a certain confidence threshold.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James Boggs whose telephone number is (571)272-2968. The examiner can normally be reached M-F 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571)272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES BOGGS/Examiner, Art Unit 2657
Prosecution Insights