DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application claims priority to foreign application with application number JAPAN 2023-141269 dated 08/31/2023. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. The following title is suggested: INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND STORAGE MEDIUM FOR AUTOMATED OPTICAL CHARACTER RECOGNITION CORRECTION.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
"a character recognition unit configured to" in claim 1, in the specification “a character recognition unit” is defined as “ a character recognition unit configured to perform a character recognition process on an image of a processing target document.” ¶0005.
"a generation unit configured to" in claim 1, in the specification “a generation unit” is defined as “ The instruction message generation unit 155 generates an instruction message by inserting an item name and the like into an instruction message template prepared in advance. The instruction message generation unit 155 transmits the instruction message through the network 104 so that the instruction message will be input into the external information processing server 105.” ¶0042 and “the instruction message generation unit 155 generates an instruction message by using item names and the item values extracted in S314. Details of the instruction message will be described later” ¶0076.
"a transmission unit configured to" in claim 1, in the specification “a transmission unit” is defined as “ a transmission unit configured to transmit the instruction message in order to obtain a reply to the instruction message from the large language model” ¶0005.
"a reception unit configured to" in claim 1, in the specification “a transmission unit” is defined as “ a reception unit configured to receive the reply to the instruction message from the large language model.” ¶0005.
"an extraction unit configured to" in claim 2, and claim 3, in the specification “an extraction unit” is defined as “The item value extractor 115 is a trained model trained to output information of character strings (item values) corresponding to extraction target items from among a group of character strings included in a processing target document image in response to receiving data of feature amounts of that group of character strings.” ¶0086.
"a display control unit configured to" in claim 11, claim 15, claim 16, and claim 17, in the specification “a display control unit” is defined as “The display control unit 159 displays information received from the information processing server 103 on a display of a display device 210 (see Fig. 2A). The display control unit 159 displays an item value confirmation screen 1000 (see Figs. 10A to 10C) to be described later, for example.” ¶0035 and “The display control unit 158 performs control for displaying the item values extracted by the document image analysis unit 154 and the reply to the instruction message obtained from the large language model 116 to the user. The display control unit 158 generates information for displaying the later-described item value confirmation screen 1000 (see Figs. 10A to 10C) to display it on a display unit, for example. In a case where the user designates correction of the displayed item values shown the display control unit 158 accepts that correction and generates an item value confirmation screen 1000 corresponding to the user's correction.” ¶0045.
"a setting unit configured to" in claim 13 and 18, in the specification “a setting unit” is defined as “ The CPU 261 functions also as a setting unit that sets whether to perform auto-correction on the item values obtained by the item value extraction process.” ¶0185 and “ The CPU 261 functions also as a setting unit that sets the large language models into which to input an instruction message” in ¶0178.
"a plurality of correction units each configured to" in claim 14, in the specification “a correction unit” is defined as “In a column 1801, correction units that output candidate character strings for correcting item values obtained by the item value extraction process are displayed. For example, the large language models selected on the large language model setting screen 1700 are displayed as correction units. Incidentally, the correction units may include correction rules each of which outputs a candidate character string(s) by performing a predetermined determination. Thus, in a case where there is a correction rule, it will be displayed as a correction unit in the column 1801.” ¶0186 and “ for multiple correction units and the character strings output from those multiple correction units match each other, auto-correction may be performed with the matched character string to display the matched character string in the item value display region 1002 by default.” in ¶0201.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
The claims in this application are given their broadest reasonable interpretation using the
plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claim 7 recite “or” then listing “the predetermined item is an item whose item value could not be obtained by the extraction unit among the extraction target items or an item for which an incorrect item value was extracted by the extraction unit among the extraction target items”. Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim 11 recite “at least one of” then listing “of the second character string extracted by the extraction unit or the first character string received as the reply from the large language model”. Since “at least one of” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 UFR 3.73(b). The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are provisionally rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims of co-pending Application No. 18/809,516. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are narrower in every aspect than the claims in the above-listed reference application and are therefore obvious variants thereof.
This is a provisional nonstatutory obviousness-type double patenting rejection because the patentably indistinct claims have not in fact been patented.
For example, the following is a chart comparing claim 1, claim 19, claim 20 of the instant application to the claim 1, claim 17, and claim 18 of the co-pending application number 18/809,516:
Instant application: 18/807,868
U.S. Application No: 18/809,516
Claim 1. An information processing apparatus comprising:
a character recognition unit configured to perform a character recognition process on an image of a processing target document;
a generation unit configured to generate an instruction message based on a result of the character recognition process, the instruction message being a message for causing a large language model to reply a first character string corresponding to a predetermined item which is included in the document;
a transmission unit configured to transmit the instruction message in order to obtain a reply to the instruction message from the large language model; and
a reception unit configured to receive the reply to the instruction message from the large language model.
Claim 19: An information processing method comprising:
performing a character recognition process on an image of a processing target document; generating an instruction message based on a result of the character recognition process, the instruction message being a message for causing a large language model to reply a first character string corresponding to a predetermined item which is included in the document; transmitting the instruction message in order to obtain a reply to the instruction message from the large language model; and
receiving the reply to the instruction message from the large language model.
Claim 20: A non-transitory computer readable storage medium storing a program which causes a computer to:
perform a character recognition process on an image of a processing target document; generate an instruction message based on a result of the character recognition process, the instruction message being a message for causing a large language model to reply a first character string corresponding to a predetermined item which is included in the document;
transmit the instruction message in order to obtain a reply to the instruction message from the large language model; and
receive the reply to the instruction message from the large language model.
Claim 1: An information processing apparatus comprising:
a character recognition unit configured to perform a character recognition process on an image of a processing target document;
a generation unit configured to generate an instruction message based on a result of the character recognition process, the instruction message being a message for causing a large language model to reply a document type of the processing target document;
a transmission unit configured to transmit the instruction message in order to obtain a reply to the instruction message from the large language model; and
a reception unit configured to receive the reply to the instruction message from the large language model.
Claim 17: An information processing method comprising:
performing a character recognition process on an image of a processing target document; generating an instruction message based on a result of the character recognition process, the instruction message being a message for causing a large language model to reply a document type of the processing target document; transmitting the instruction message in order to obtain a reply to the instruction message from the large language model; and
receiving the reply to the instruction message from the large language model.
Claim 18: A non-transitory computer readable storage medium storing a program which causes a computer to:
perform a character recognition process on an image of a processing target document; generate an instruction message based on a result of the character recognition process, the instruction message being a message for causing a large language model to reply a document type of the processing target document;
transmit the instruction message in order to obtain a reply to the instruction message from the large language model; and
receive the reply to the instruction message from the large language model.
Although co-pending application 18/809,516 discloses “An information processing apparatus comprising: a character recognition unit configured to perform a character recognition process on an image of a processing target document; a generation unit configured to generate an instruction message based on a result of the character recognition process, the instruction message being a message for causing a large language model to reply a” and “a transmission unit configured to transmit the instruction message in order to obtain a reply to the instruction message from the large language model; and a reception unit configured to receive the reply to the instruction message from the large language model.” it does not explicitly disclose “a first character string corresponding to a predetermined item which is included in the document”. However, in an analogous field of endeavor Cardozo discloses “a first character string corresponding to a predetermined item which is included in the document (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string)”. Accordingly, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the limitations of claim 1, 17 and 18 of the co-pending application 18/809,516 with the teachings of Cardozo to implement the identification of a first predetermined character string. One of ordinary skill in the art would be motivated to combine the limitations of claim 1, 17 and 18 of the co-pending application 18/809,516 with the Cardozo reference in order to produce “methods and system that limit the need for manual review” as disclosed by Cardozo in ¶0003. Therefore, it would have been obvious to combine the limitations of claim 1, 17, and 18 of the co-pending application 18/809,516 and Cardozo to obtain the invention of the instant claim 1, 19 and 20.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 9-13, and 19-20 are rejected under 35 U.S.C. 103 as unpatentable over Zhang et al (Zhang, Yanzhe, et al. "Llavar: Enhanced visual instruction tuning for text-rich image understanding." arXiv preprint arXiv:2306.17107 (2023), hereafter referred to as Zhang) in view of Cardozo et al (US Patent Publication US 2023/0034513 A1, hereafter referred to as Cardozo).
Regarding Claim 1, Zhang teaches a character recognition unit (Zhang Pg 4 ¶02 discloses using PaddleOCR to OCR all images) configured to perform a character recognition process on an image of a processing target document (Zhang Pg 4 ¶02 and Pg 2 ¶02 discloses performing OCR (optical character recognition) on the desired image);
a generation unit configured (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) to generate an instruction message based on a result of the character recognition process (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following), the instruction message being a message for causing a large language model to reply (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image)
configured to transmit the instruction message (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, as high-quality instruction-following examples. This process requires GPT--4 to denoise the OCR results and develop specific questions to create complex instructions based on the input) in order to obtain a reply to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image); and
configured to receive the reply (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, the examiner is interpreting the answer to be the reply) to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image).
Zhang does not explicitly disclose an information processing apparatus comprising:
a first character string corresponding to a predetermined item which is included in the document; a transmission unit, a reception unit.
Cardozo is in the same field of image analysis in which character recognition is performed for document interpretation. Further, Cardozo teaches an information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) comprising:
a first character string corresponding to a predetermined item which is included in the document (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string);
a transmission unit (Cardozo ¶0027, ¶ discloses control circuity to send and receive commands and requests),
a reception unit (Cardozo Fig 4 416, ¶0006, ¶0027 discloses processing the user submission and receiving content and data).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the computer structure and hardware to implement the method including the identification of a character string corresponding to a predetermined item identified in the document as taught by Cardozo to make an invention that can automatically identify and display the predetermined item of importance to a user in real time; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to create methods and systems that are particularly relevant in real-time environments as the methods and system limit the need for manual review. (Cardozo, ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 2, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 1, further comprising
an extraction unit configured to extract (Zhang Pg 4 ¶02 discloses an extracting the text) a second character string (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) corresponding to the predetermined item (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) from among a group of character strings (Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of test strings in the images) obtained by the character recognition process (Zhang Pg 4 ¶02 and Pg 2 ¶02 discloses performing OCR (optical character recognition) on the desired image). See Claim 1 for rationale, its parent claim.
Regarding Claim 3, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 2,
wherein the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text ) extracts the second character string (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) by inputting the character strings obtained by the character recognition process (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) into a trained model (Zhang Pg 5 ¶02 discloses the character strings being input instructions fed to the model). See Claim 1 for rationale, its parent claim.
Regarding Claim 4, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 2,
wherein the generation unit (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) generates the instruction message (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) that includes the predetermined item and (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) the second character string (Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of test strings in the images which the examiner is interpreting to include a second text string) extracted by the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text ). See Claim 1 for rationale, its parent claim.
Regarding Claim 9, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 1,
wherein the generated instruction message (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) includes the predetermined item (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) and a group of character strings obtained by the character recognition process (Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of text strings). See Claim 1 for rationale, its parent claim.
Regarding Claim 10, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 9,
wherein the generated instruction message (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) is a message for causing the large language model to reply the first character string (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) corresponding to the predetermined item (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) from among the group of character strings obtained by the character recognition process (Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of text strings). See Claim 1 for rationale, its parent claim.
Regarding Claim 11, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 2,
further comprising a display control unit configured to display (Cardozo ¶0027-¶0028 discloses a display for receiving and displaying data), on a display unit (Cardozo ¶0026, ¶0028 and Fig 3 324 discloses a display on a computer or similar user interface), at least one of the second character string (Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of text strings) extracted by the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text ) or the first character string received as the reply from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on). See Claim 1 for rationale, its parent claim.
Regarding Claim 12, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 11,
wherein the display control unit (Cardozo ¶0027-¶0028 discloses a display for receiving and displaying data) displays a difference between (Cardozo ¶0054-¶0056 Fig 4 416-418 discloses validating the first target string with the plurality of text strings and if they are different recommending a recommendation to the user to confirm their difference as it does then not validate the target string) the second character string ( Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of test strings in the images which the examiner is interpreting to include a second text string) extracted by the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text ) and the first character string received from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) in such a manner as to exaggerate the difference (Cardozo ¶0054-¶0056 Fig 4 416-418 discloses validating the first target string with the plurality of text strings and if they are different recommending a recommendation to the user to confirm their difference as it does then not validate the target string). See Claim 1 for rationale, its parent claim.
Regarding Claim 13, Zhang in view of Cardozo teaches the information processing apparatus(Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 1,
further comprising a setting unit configured to set (Zhang Pg 3 ¶02 discloses selecting the model) a plurality of large language models (Zhang Pg 3 ¶02 discloses the multiples types of large language models and the model that is selected being the LLaVA model) to be caused to reply to the instruction message (Zhang Pg 10 ¶2 discloses LLaVA answering the questions),
wherein the transmission unit (Cardozo ¶0027, ¶ discloses control circuity to send and receive commands and requests) transmits the instruction message (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, as high-quality instruction-following examples. This process requires GPT--4 to denoise the OCR results and develop specific questions to create complex instructions based on the input) to the set plurality of large language models (Zhang Pg 3 ¶02 discloses the multiples types of large language models and the model that is selected being the LLaVA model). See Claim 1 for rationale, its parent claim.
Regarding Claim 19, Zhang teaches performing a character recognition process (Zhang Pg 4 ¶02 discloses using PaddleOCR to OCR all images) on an image of a processing target document (Zhang Pg 4 ¶02 and Pg 2 ¶02 discloses performing OCR (optical character recognition) on the desired image);
generating an instruction message (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) based on a result of the character recognition process (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) , the instruction message being a message for causing a large language model to reply (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image)
the instruction message (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, as high-quality instruction-following examples. This process requires GPT--4 to denoise the OCR results and develop specific questions to create complex instructions based on the input) in order to obtain a reply to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image); and
the reply (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, the examiner is interpreting the answer to be the reply) to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image).
Zhang does not explicitly disclose an information processing method comprising:
a first character string corresponding to a predetermined item which is included in the document, transmitting, receiving.
Cardozo is in the same field of image analysis in which character recognition is performed for document interpretation. Further, Cardozo teaches an information processing method (Cardozo ¶0003, ¶0006 discloses methods for automatically extracting and processing data) comprising:
a first character string corresponding to a predetermined item which is included in the document (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string);
transmitting (Cardozo ¶0027, ¶ discloses control circuity to send and receive commands and requests)
receiving (Cardozo Fig 4 416, ¶0006, ¶0027 discloses processing the user submission and receiving content and data).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the computer structure and hardware to implement the method including the identification of a character string corresponding to a predetermined item identified in the document as taught by Cardozo to make an invention that can automatically identify and display the predetermined item of importance to a user in real time; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to create methods and systems that are particularly relevant in real-time environments as the methods and system limit the need for manual review. (Cardozo, ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 20, Zhang teaches perform a character recognition process (Zhang Pg 4 ¶02 discloses using PaddleOCR to OCR all images) on an image of a processing target document (Zhang Pg 4 ¶02 and Pg 2 ¶02 discloses performing OCR (optical character recognition) on the desired image);
generate an instruction message (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) based on a result of the character recognition process (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following), the instruction message being a message for causing a large language model to reply (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image)
the instruction message (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, as high-quality instruction-following examples. This process requires GPT--4 to denoise the OCR results and develop specific questions to create complex instructions based on the input) in order to obtain a reply to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image);
the reply (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, the examiner is interpreting the answer to be the reply) to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image).
Zhang does not explicitly disclose anon-transitory computer readable storage medium storing a program which causes a computer to:
a first character string corresponding to a predetermined item which is included in the document; transmit, receive.
Cardozo is in the same field of image analysis in which character recognition is performed for document interpretation. Further, Cardozo teaches a non-transitory computer readable storage medium (Cardozo ¶0029 discloses a non-transitory storage media storing instructions) storing a program which causes a computer (Cardozo ¶0028 discloses a program and processors to perform operations) to:
a first character string corresponding to a predetermined item which is included in the document (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string);
transmit (Cardozo ¶0027, ¶ discloses control circuity to send and receive commands and requests)
Receive (Cardozo Fig 4 416, ¶0006, ¶0027 discloses processing the user submission and receiving content and data).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the computer structure and hardware to implement the method including the identification of a character string corresponding to a predetermined item identified in the document as taught by Cardozo to make an invention that can automatically identify and display the predetermined item of importance to a user in real time; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to create methods and systems that are particularly relevant in real-time environments as the methods and system limit the need for manual review. (Cardozo, ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 5-8, and 14-18 are rejected under 35 U.S.C. 103 as unpatentable over Zhang in view of Cardozo in further view of Somech et al (US Patent Pub US 2023/0034513 A1, hereafter referred to as Somech).
Regarding Claim 5, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 4,
wherein the generation unit (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) generates the instruction message ( Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) for causing the large language model to reply (Zhang Fig 1 discloses a message asking what type of book and the model replying with the types of book) whether the second character string ( Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of test strings in the images which the examiner is interpreting to include a second text string) extracted by the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text ).
Zhang in view of Cardozo does not explicitly disclose contains an error.
Somech is in the same field of image analysis in which characteristic recognition is performed for document interpretation. Further, Somech teaches contains an error (Somech ¶0003 discloses if characters are likely to be an error from OCR).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Cardozo by incorporating the OCR error detection and correction as taught by Somech to make an invention that can automatically detect and correct the errors resulting from the OCR; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to improve existing technologies by causing certain characters (for example, words) to be replaced at a document if such characters are likely to be an error. (Somech ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 6, Zhang in view of Cardozo in view of Somech teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 4,
wherein the generation unit (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) generates the instruction message ( Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) for causing the large language model to reply (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) the first character string corresponding to the predetermined item (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) as a correct character string (Cardozo ¶0051-¶0052 Fig 4 410 and 412 disclose comparing the target string to the character strings to determine if they match and are correct) in a case where the second character string( Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of test strings in the images which the examiner is interpreting to include a second text string) includes a character misrecognized in the character recognition process (Somech ¶0024 discloses that characters can be blurred or misspelled leading to inaccurate or incorrect representation of the characters). See rationale for Claim 5 its parent claim.
Regarding Claim 7, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 2,
wherein the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text) extracts item values representing character strings corresponding to extraction target items (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) including the predetermined item (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) from the group of character strings (Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of text strings), and
the predetermined item is an item (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string)
by the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text) among the extraction target items (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string)
was extracted by the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text) among the extraction target items (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string).
Zhang in view of Cardozo does not explicitly disclose whose item value could not be obtained, or an item for which an incorrect item value.
Somech is in the same field of image analysis in which characteristic recognition is performed for document interpretation. Further, Somech teaches whose item value could not be obtained (Somech ¶0024 discloses that characters can be blurred or misspelled leading to inaccurate or incorrect representation of the characters) or an item for which an incorrect item value (Somech ¶0003 discloses if characters are likely to be an error from OCR).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Cardozo by incorporating the OCR error detection and correction as taught by Somech to make an invention that can automatically detect and correct the errors resulting from the OCR; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to improve existing technologies by causing certain characters (for example, words) to be replaced at a document if such characters are likely to be an error. (Somech ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 8, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 2,
wherein, in a case where the extraction unit (Zhang Pg 4 ¶02 discloses an extracting the text )
the generation unit (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) generates the instruction message ( Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) for causing the large language model to reply the first character string (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) corresponding to the predetermined item (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) from among the group of character strings obtained by the character recognition process ( Cardozo ¶0050 and Fig 4 408 discloses identifying a plurality of text strings).
Zhang in view of Cardozo does not explicitly disclose could not extract the second character string.
Somech is in the same field of image analysis in which characteristic recognition is performed for document interpretation. Further, Somech teaches could not extract the second character string (Somech ¶0024 discloses that characters can be blurred or misspelled leading to inaccurate or incorrect representation of the characters).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Cardozo by incorporating the OCR error detection and correction as taught by Somech to make an invention that can automatically detect and correct the errors resulting from the OCR; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to improve existing technologies by causing certain characters (for example, words) to be replaced at a document if such characters are likely to be an error. (Somech ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 14, Zhang in view of Cardozo teaches the information processing apparatus(Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) the second character string (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) outputs the first character string obtained from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on).
Zhang in view of Cardozo does not explicitly disclose according to claim 2, further comprising a plurality of correction units each configured to output a candidate character string for correcting wherein at least one of the plurality of correction units.
Somech is in the same field of image analysis in which characteristic recognition is performed for document interpretation. Further, Somech teaches according to claim 2, further comprising a plurality of correction units (Somech ¶0059, ¶0062 Fig 2 212 discloses multiple units as part of the character replacement suggestion component) each configured to output a candidate character string for correcting (Somech ¶0043, Fig 2 212 discloses a character replacement suggestion component while ¶0062 and Fig 2 208 discloses a wrong character detector) wherein at least one of the plurality of correction units (Somech ¶0059, ¶0062 Fig 2 212 discloses multiple units as part of the character replacement suggestion component).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Cardozo by incorporating the OCR error detection and correction as taught by Somech to make an invention that can automatically detect and correct the errors resulting from the OCR; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to improve existing technologies by causing certain characters (for example, words) to be replaced at a document if such characters are likely to be an error. (Somech ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 15, Zhang in view of Cardozo in view of Somech teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 14,
further comprising a display control unit (Somech Fig 2 , 220 and ¶0080, ¶0083, ¶0085 discloses a presentation unit to display the outcome of the character predictor to the user) configured to, in a case where the candidate character strings output by the plurality of correction units (Somech ¶0043, Fig 2 212 discloses a character replacement suggestion component while ¶0062 and Fig 2 208 discloses a wrong character detector) match each other, display the matched character string (Somech ¶0153 and Fig 6A, and Fig 10 1006 discloses that if there is a matched character string then the string is returned) as the character string (Somech Fig 2 , 220 and ¶0080, ¶0083, ¶0085 discloses a presentation unit to display the outcome of the character predictor to the user) corresponding to the predetermined item (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) on a display unit (Somech Fig 2 , 220 and ¶0080, ¶0083, ¶0085 discloses a presentation unit to display the outcome of the character predictor to the user). See rationale for Claim 14 its parent claim.
Regarding Claim 16, Zhang in view of Cardozo in view of Somech teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 14,
further comprising a display control unit (Somech Fig 2 , 220 and ¶0080, ¶0083, ¶0085 discloses a presentation unit to display the outcome of the character predictor to the user) configured to display the candidate character strings output by the plurality of correction units (Somech ¶0043, Fig 2 212 discloses a character replacement suggestion component while ¶0062 and Fig 2 208 discloses a wrong character detector) in list form on a display unit (Somech ¶0124 and Fig 7 discloses displaying a list of correct term candidates). See rationale for Claim 14 its parent claim.
Regarding Claim 17, Zhang in view of Cardozo in view of Somech teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 14,
further comprising a display control unit (Somech Fig 2 , 220 and ¶0080, ¶0083, ¶0085 discloses a presentation unit to display the outcome of the character predictor to the user) configured to preferentially display a matched character string (Somech ¶0153 and Fig 6A, and Fig 10 1006 discloses that if there is a matched character string then the string is returned) on a display unit among the candidate character strings output by the plurality of correction units (Somech ¶0043, Fig 2 212 discloses a character replacement suggestion component while ¶0062 and Fig 2 208 discloses a wrong character detector ¶0124 and Fig 6, 610 discloses displaying a list of correct term candidates). See rationale for Claim 14 its parent claim.
Regarding Claim 18, Zhang in view of Cardozo teaches the information processing apparatus (Cardozo ¶0026- ¶0028, ¶0030 and Fig 3 discloses data and information processing devices that have hardware to complete the character recognition of a document) according to claim 2, the second character string (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) extracted by the extraction unit(Zhang Pg 4 ¶02 discloses an extracting the text) with the first character string (Cardozo ¶0044 and Fig 4 404 discloses a text string based on a value present in the document, this being the target string) received from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image).
Zhang in view of Cardozo does not explicitly disclose further comprising a setting unit configured to set whether to replace, and to output the replaced character string.
Somech is in the same field of image analysis in which characteristic recognition is performed for document interpretation. Further, Somech teaches according to claim 2,
further comprising a setting unit configured to set whether to replace (Somech Fig 6A 612 and ¶0124, ¶0150 discloses a unit for selecting the option to replace the incorrect term candidates) and to output the replaced character string (Somech ¶0150 discloses the user selecting the correct replacement string to be displayed).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Cardozo by incorporating the OCR error detection and correction as taught by Somech to make an invention that can automatically detect and correct the errors resulting from the OCR; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to improve existing technologies by causing certain characters (for example, words) to be replaced at a document if such characters are likely to be an error. (Somech ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US Patent Publication US-20230063374-A1 to Muramatsu et al discloses a method for extracting character strings from a document using OCR.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL ROBERTS whose telephone number is (571)272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL L ROBERTS/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights