DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sigal (PGPUB: 20240071121 A1) in view of HOEHNE (PGPUB: 20200302208 A1), and in view of Wang (PGPUB: 20230206675 A1).
Regarding claim 1 Sigal teaches a system comprising:
a memory storing program code (see Fig. 6, items 610 and 614); and
one or more processing units to execute the program code (see Fig. 6, item 602) to cause the system to:
acquire text data extracted from an image (see Fig. 1, paragraph 33, upon receiving the image, document manager 115 extracts the set of text from the document captured in the image using OCR. Next, document manager 115 sends the extracted text to machine learning model manager 125 for further processing);
generate a prompt including the extracted text data (see Fig. 1, paragraph 26, machine learning model manager 125 handles the management of machine learning models. For instance, machine learning model manager 125 may receive a request (e.g., from document manager 115, client device 105, an application, a service, another computing device, etc.) to classify a particular document along with text extracted from the particular document. In response to such a request, machine learning model manager 125 accesses machine learning models storage 135 to retrieve a trained machine learning model configured to classify documents. Then, machine learning model manager 125 provides the text extracted from the particular document as inputs to the machine learning model);
input the prompt to a text generation model (see Fig. 1, paragraph 26, in response to such a request, machine learning model manager 125 accesses machine learning models storage 135 to retrieve a trained machine learning model configured to classify documents. Then, machine learning model manager 125 provides the text extracted from the particular document as inputs to the machine learning model); and
receive corrected text data from the text generation model in response to the prompt (see Fig. 1, paragraph 26, machine learning model manager 125 provides the text extracted from the particular document as inputs to the machine learning model. Based on the text, the machine learning model generates a set of outputs (e.g., a set of scores for a set of types of documents). Based on the set of outputs, machine learning model manager 125 determines a type of document for the particular document. Machine learning model manager 125 stores the determination of the type of document in documents storage 140. In some cases, machine learning model manager 130 also sends the determination of the type of document to the requestor).
However, Sigal does not expressly teach that instructions to correct the text data and the text data includes handwritten text.
HOEHNE teaches that document 120 may be a computer file, document, image, and/or other digital file or object including text information that may be extracted. Document 120 may include handwritten and/or typewritten text. Examples of document 120 may include a webpage, printed papers, publications, an invoice, an instruction manual, a slideshow presentation, hand-written notes, and/or other images including text characters, to name just some examples. An embodiment of document 120 is described with reference to FIG. 2A (see Fig. 1,2, and 5, paragraph 20); document 120 may include handwritten and typewritten text. OCR system 110 may correlate identified characters in both handwritten and typewritten characters to the same index value. For example, if document 120 include the letter “E” in both the handwritten and typewritten form, OCR system 110 may use the same index value to identify each instance of the letter. In this manner, OCR system 110 may recognize both types of characters and map them to the same index value (see Fig. 1, paragraph 23); CNN 140 may receive document 120. To receive document 120, a user may supply a command to OCR system 110 to perform an OCR process on document 120. Using CNN 140, OCR system 110 may identify characters to generate one or more segmentation masks and/or identify words to generate bounding boxes (see Fig. 1, paragraph 25); OCR system 110 may receive the training package from a user configuring OCR system 110. In this manner, OCR system 110 may be customized depending on the application and the types of documents 120 analyzed. This customization may yield more accurate results and/or may improve character recognition times because the training data may be more focused. Training CNN 140 to identify particular patterns of importance may yield faster pattern recognition and/or indexing of characters (see Fig. 1, paragraph 76).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sigal by HOEHNE to obtain document 120 may be a computer file, document, image, and/or other digital file or object including text information that may be extracted. Document 120 may include handwritten and/or typewritten text. Examples of document 120 may include a webpage, printed papers, publications, an invoice, an instruction manual, a slideshow presentation, hand-written notes, and/or other images including text characters, to name just some examples, in order to provide the text data includes handwritten text; and further to obtain CNN 140 may receive document 120. To receive document 120, a user may supply a command to OCR system 110 to perform an OCR process on document 120 and OCR system 110 may receive the training package from a user configuring OCR system 110. In this manner, OCR system 110 may be customized depending on the application and the types of documents 120 analyzed. This customization may yield more accurate results and/or may improve character recognition times because the training data may be more focused, in order to further provide instructions to correct the text data and the text data includes handwritten text. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
However, the combination does not expressly teach to generate a prompt.
Wang teaches that mixed documents are often in a form-like format (as illustrated in FIG. 4) with machine-printed questions (e.g., “When did the incident occur?”) or sentences (e.g., “Please outline below the diagnosis associated with your patent's primary condition”), and handwritten responses. The questions or sentences will be referred to herein as “prompts.” One of ordinary skill in the art will recognize that each prompt may be less than a complete question or sentence, including, e.g., single words (e.g., “Name”, “Age”, etc.), numbers or letters followed by zero or more punctuation marks (e.g., “1)”, “a:”, “A”, etc.) (see Fig. 4 and 14, paragraph 141).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by Wang to obtain each prompt may be less than a complete question or sentence, including, e.g., single words (e.g., “Name”, “Age”, etc.), numbers or letters followed by zero or more punctuation marks (e.g., “1)”, “a:”, “A”, etc.), in order to provide to generate a prompt. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Regarding claims 9 and 17. Sigal teaches a method comprising:
extracting text data from an image (see Fig. 1, paragraph 33, upon receiving the image, document manager 115 extracts the set of text from the document captured in the image using OCR. Next, document manager 115 sends the extracted text to machine learning model manager 125 for further processing);
generating a request including the extracted text data (see Fig. 1, paragraph 26, machine learning model manager 125 handles the management of machine learning models. For instance, machine learning model manager 125 may receive a request (e.g., from document manager 115, client device 105, an application, a service, another computing device, etc.) to classify a particular document along with text extracted from the particular document. In response to such a request, machine learning model manager 125 accesses machine learning models storage 135 to retrieve a trained machine learning model configured to classify documents. Then, machine learning model manager 125 provides the text extracted from the particular document as inputs to the machine learning model);
inputting the request to a text generation model (see Fig. 1, paragraph 26, in response to such a request, machine learning model manager 125 accesses machine learning models storage 135 to retrieve a trained machine learning model configured to classify documents. Then, machine learning model manager 125 provides the text extracted from the particular document as inputs to the machine learning model); and
receiving corrected text data from the text generation model in response to the request (see Fig. 1, paragraph 26, machine learning model manager 125 provides the text extracted from the particular document as inputs to the machine learning model. Based on the text, the machine learning model generates a set of outputs (e.g., a set of scores for a set of types of documents). Based on the set of outputs, machine learning model manager 125 determines a type of document for the particular document. Machine learning model manager 125 stores the determination of the type of document in documents storage 140. In some cases, machine learning model manager 130 also sends the determination of the type of document to the requestor).
However, Sigal does not expressly teach that indicating one or more portions of the extracted text data which represent handwritten text.
HOEHNE teaches that document 120 may be a computer file, document, image, and/or other digital file or object including text information that may be extracted. Document 120 may include handwritten and/or typewritten text. Examples of document 120 may include a webpage, printed papers, publications, an invoice, an instruction manual, a slideshow presentation, hand-written notes, and/or other images including text characters, to name just some examples. An embodiment of document 120 is described with reference to FIG. 2A (see Fig. 1,2, and 5, paragraph 20); document 120 may include handwritten and typewritten text. OCR system 110 may correlate identified characters in both handwritten and typewritten characters to the same index value. For example, if document 120 include the letter “E” in both the handwritten and typewritten form, OCR system 110 may use the same index value to identify each instance of the letter. In this manner, OCR system 110 may recognize both types of characters and map them to the same index value (see Fig. 1, paragraph 23)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Sigal by HOEHNE to obtain document 120 may be a computer file, document, image, and/or other digital file or object including text information that may be extracted. Document 120 may include handwritten and/or typewritten text. Examples of document 120 may include a webpage, printed papers, publications, an invoice, an instruction manual, a slideshow presentation, hand-written notes, and/or other images including text characters, to name just some examples, in order to provide indicating one or more portions of the extracted text data which represent handwritten text.
However, the combination does not expressly teach to generate a prompt.
Wang teaches that mixed documents are often in a form-like format (as illustrated in FIG. 4) with machine-printed questions (e.g., “When did the incident occur?”) or sentences (e.g., “Please outline below the diagnosis associated with your patent's primary condition”), and handwritten responses. The questions or sentences will be referred to herein as “prompts.” One of ordinary skill in the art will recognize that each prompt may be less than a complete question or sentence, including, e.g., single words (e.g., “Name”, “Age”, etc.), numbers or letters followed by zero or more punctuation marks (e.g., “1)”, “a:”, “A”, etc.) (see Fig. 4 and 14, paragraph 141).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by Wang to obtain each prompt may be less than a complete question or sentence, including, e.g., single words (e.g., “Name”, “Age”, etc.), numbers or letters followed by zero or more punctuation marks (e.g., “1)”, “a:”, “A”, etc.), in order to provide to generate a prompt. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results
Regarding claims 5 and 13. The system of Claim 1, the one or more processing units to execute the program code to cause the system to:
receive a schema comprising a plurality of fields (see Wang, Fig. 4),
wherein the prompt includes the plurality of fields (see Wang, Fig. 4), and
wherein reception of the corrected text data comprises reception of one or more field, text data pairs (see Wang, paragraph 47, a low confidence score can also be associated with an indication of the reason for the low confidence (e.g., 1) incomplete or missing information; 2) inconsistent information; 3) unclear information; 4) calculation verification required, etc.), allowing the person to identify and address the specific reason for the low confidence score. Any human input to the information extraction engine in response to a low confidence score can be used as a labeled data point to re-train the engine and improve its accuracy).
Regarding claims 6 and 14. The system of Claim 5, the one or more processing units to execute the program code to cause the system to:
create a database table row based on the one or more field, text data pairs (see Wang, Fig. 2, 4 and 14, paragraph 28 and 141, data storage engine 215 is configured to store the documents received by document receiving engine 210 into database 120. Data storage engine 215 is also configured to store documents converted by document conversion engine 220 and outputs from information retrieved and extracted from documents by information retrieval engine 225 and information extraction engine 230 into database 120. Data storage engine 215 can also be configured to store data in either structured or unstructured format, or both, depending on the type of documents and data received from document receiving engine 210; mixed documents are often in a form-like format (as illustrated in FIG. 4) with machine-printed questions (e.g., “When did the incident , 8, 10occur?”) or sentences (e.g., “Please outline below the diagnosis associated with your patent's primary condition”), and handwritten responses. The questions or sentences will be referred to herein as “prompts.” One of ordinary skill in the art will recognize that each prompt may be less than a complete question or sentence, including, e.g., single words (e.g., “Name”, “Age”, etc.), numbers or letters followed by zero or more punctuation marks (e.g., “1)”, “a:”, “A”, etc.)).
Regarding claims 2, 8, 10, and 16. The system of claim 1, wherein the image is an image of a document (see Sigal, paragraph 5, receives an image of a document, the document comprising a set of text), and
wherein the prompt includes a description of the document (see Sigal, paragraph 3, the set of training data comprises a set of text in a set of documents and a set of labels indicating a set of types of documents for the set of documents).
Regarding claims 3-4, 7, 11-12, 15, and 18. The system of claim 2, the one or more processing units to execute the program code to cause the system to:
classify one or more portions of the text data as handwritten (see HOEHNE, Fig. 1, paragraph 59, OCR system 110 may then categorize different handwriting styles in type grid segmentation mask 200C),
wherein the prompt indicates the one or more portions which are classified as handwritten (see Fig. 1,2, and 5, paragraph 20, document 120 may be a computer file, document, image, and/or other digital file or object including text information that may be extracted. Document 120 may include handwritten and/or typewritten text. Examples of document 120 may include a webpage, printed papers, publications, an invoice, an instruction manual, a slideshow presentation, hand-written notes, and/or other images including text characters, to name just some examples. An embodiment of document 120 is described with reference to FIG. 2A; see HOEHNE, Fig. 1, paragraph 76, OCR system 110 may receive the training package from a user configuring OCR system 110. In this manner, OCR system 110 may be customized depending on the application and the types of documents 120 analyzed. This customization may yield more accurate results and/or may improve character recognition times because the training data may be more focused. Training CNN 140 to identify particular patterns of importance may yield faster pattern recognition and/or indexing of characters).
Regarding claims 19 and 20. The one or more non-transitory media of Claim 18, the program code executable by one or more processing units of a computing system to cause the computing system to:
receive a schema comprising a plurality of fields (see Wang, Fig. 4),
the prompt including the plurality of fields (see Wang, Fig. 4), and
receipt of the corrected text data comprising receipt of one or more field, text data pairs (see Wang, paragraph 47, a low confidence score can also be associated with an indication of the reason for the low confidence (e.g., 1) incomplete or missing information; 2) inconsistent information; 3) unclear information; 4) calculation verification required, etc.), allowing the person to identify and address the specific reason for the low confidence score. Any human input to the information extraction engine in response to a low confidence score can be used as a labeled data point to re-train the engine and improve its accuracy); and
create a database table row based on the one or more field, text data pairs (see Wang, Fig. 2, 4 and 14, paragraph 28 and 141, data storage engine 215 is configured to store the documents received by document receiving engine 210 into database 120. Data storage engine 215 is also configured to store documents converted by document conversion engine 220 and outputs from information retrieved and extracted from documents by information retrieval engine 225 and information extraction engine 230 into database 120. Data storage engine 215 can also be configured to store data in either structured or unstructured format, or both, depending on the type of documents and data received from document receiving engine 210; mixed documents are often in a form-like format (as illustrated in FIG. 4) with machine-printed questions (e.g., “When did the incident , 8, 10occur?”) or sentences (e.g., “Please outline below the diagnosis associated with your patent's primary condition”), and handwritten responses. The questions or sentences will be referred to herein as “prompts.” One of ordinary skill in the art will recognize that each prompt may be less than a complete question or sentence, including, e.g., single words (e.g., “Name”, “Age”, etc.), numbers or letters followed by zero or more punctuation marks (e.g., “1)”, “a:”, “A”, etc.)).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN JIA whose telephone number is (571)270-5536. The examiner can normally be reached 9:00 am-7:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIN JIA/Primary Examiner, Art Unit 2663
Prosecution Insights