CTNF 18/809,516 CTNF 100468 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Priority Receipt is acknowledged that application claims priority to foreign application with application number JAPAN 2023-140773 dated 08/31/2023. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78. Specification The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. The following title is suggested: INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND STORAGE MEDIUM FOR DOCUMENT PROCESSING. 07-30-03-h AIA Claim Interpretation 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: "a determination unit configured to" in claim 10, in the specification “a determination unit” is not defined. "a setting unit configured to" in claim 14 and 16, in the specification “a setting unit” is defined as “ The CPU 261 functions also as a setting unit that sets the large language models into which to input an instruction message” ¶0198 and “ The CPU 261 functions also as a setting unit that document type options” in ¶0228. Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification. Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives , the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009). Claim 11 recite “or” then listing “configured to display at least one of the document type determined by the determination unit or the document type received from the large language mode”. Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim . While citations have been provided for completeness and rapid prosecution, only one element is required . Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. Claims 10-13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. The Examiner strongly suggested that appropriate corrections be made to clarify the claim scope. 07-34-23 Claim limitation “a determination unit” in Claim 10 invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. There is no disclosure in the specification of any determination unit or what hardware is used to represent the determination unit. There is disclosure of determining the document type in [0077] and [0078] as well as Fig 3B. But this does not link the determination unit to its action of inputting a group of character strings that is claimed in Claim 10; therefore it is indefinite. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. Applicant may: (a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph; (b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or (c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)). If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either: (a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or (b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181. Claim(s) 11-13 depend either directly or indirectly from the rejection of Claim(s) 10, therefore they are also rejected. Appropriate correction is required. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. Claims 1-11 and 14-18 are rejected under 35 U.S.C. 103 as unpatentable over Zhang et al (Zhang, Yanzhe, et al. "Llavar: Enhanced visual instruction tuning for text-rich image understanding." arXiv preprint arXiv:2306.17107 (2023) , hereafter referred to as Zhang) in view of Kubo et al (US Patent Publication US 2022/0269898 A1, hereafter referred to as Kubo). Regarding Claim 1, Zhang teaches a character recognition unit (Zhang Pg 4 ¶02 discloses using PaddleOCR to OCR all images) configured to perform a character recognition process on an image of a processing target document (Zhang Pg 4 ¶02 and Pg 2 ¶02 discloses performing OCR (optical character recognition) on the desired image) ; a generation unit configured (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) to generate an instruction message based on a result of the character recognition process (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following), the instruction message being a message for causing a large language model to reply a document type (Zhang Fig 1 discloses identifying the object as a book and asking what the name of the book in the image is) of the processing target document (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) ; configured to transmit the instruction message (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, as high-quality instruction-following examples. This process requires GPT--4 to denoise the OCR results and develop specific questions to create complex instructions based on the input) in order to obtain a reply to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) ; and configured to receive the reply (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, the examiner is interpreting the answer to be the reply) to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image). Zhang does not explicitly disclose an information processing apparatus comprising: a transmission unit, a reception unit. Kubo is in the same field of image analysis in which character recognition is performed for document interpretation. Further, Kubo teaches an information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) comprising: a transmission unit (Kubo ¶0025 and Fig 2 12 discloses a transmission and reception controller) a reception unit (Kubo ¶0025 and Fig 2 12 discloses a transmission and reception controller). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the computer structure and hardware to implement the method including the displaying of the documents as taught by Kubo to make an invention that can automatically display the document type and title to a user in real time a incorporate user input into the document display; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to achieve both a processing speed and character recognition accuracy as compared to a case where single image conversion processing is uniformly executed for an entire document as preprocessing prior to character recognition. (Kubo, ¶0005). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Regarding Claim 2, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 1, wherein the generated instruction message (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) is a message for causing the large language model to reply the document type of the document (Zhang Fig 1 discloses a message asking what type of book and the model replying with the types of book) based on character strings obtained by the character recognition process (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on). See Claim 1 for rationale, its parent claim. Regarding Claim 3, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 1, further comprising an extraction unit configured to extract at least a character string (Zhang Pg 4 ¶02 discloses an extracting the text) corresponding to a predetermined item (Zhang Fig 1 discloses the OCR character strings detailing the description, title, type and author from the source image) from among character strings obtained by the character recognition process (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) , wherein the generation unit (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) generates the instruction message ( Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) for causing the large language model to reply the document type (Zhang Fig 1 discloses a message asking what type of book and the model replying with the types of book) of the processing target document based on the character string (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) corresponding to the predetermined item (Zhang Fig 1 discloses the OCR character strings detailing the description, title, type and author from the source image) . See Claim 1 for rationale, its parent claim. Regarding Claim 4, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 3, wherein the predetermined item is a title (Zhang Fig 1 discloses the character strings including the Title) . See Claim 1 for rationale, its parent claim. Regarding Claim 5, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 3, wherein the predetermined item includes a plurality of items (Zhang Fig 1 discloses the character strings including the description, cover description, and Title and Author of the book) . See Claim 1 for rationale, its parent claim. Regarding Claim 6, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 3, wherein the extraction unit extracts a character string (Zhang Pg 4 ¶02 discloses an extracting the text) corresponding to the predetermined item (Zhang Fig 1 discloses the OCR character strings detailing the description, title, type and author from the source image) by inputting the character strings obtained by the character recognition process (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) into a trained model (Zhang Pg 5 ¶02 discloses the character strings being input instructions fed to the model), and the generation unit (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) generates the instruction message ( Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) for causing the large language model to reply the document type (Zhang Fig 1 discloses a message asking what type of book and the model replying with the types of book) of the processing target document based on the character string (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) corresponding to the predetermined item (Zhang Fig 1 discloses the OCR character strings detailing the description, title, type and author from the source image) . See Claim 1 for rationale, its parent claim. Regarding Claim 7, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 1, wherein the instruction message includes a first question (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and a first question and instruction following) for causing the large language model to answer whether the document type (Zhang Fig 1 discloses asking what the name of the book in the image is) of the processing target document (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) is one of a plurality of document types set in advance (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs) . See Claim 1 for rationale, its parent claim. Regarding Claim 8, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 7, wherein the instruction message includes a second question (Zhang Fig 1 discloses a second question being asked) for causing the large language model to reply a document type (Zhang Pg 15 -16 Appendix A discloses the image classifier can classify the type as other) other than the plurality of document types as the document type of the processing target document (Zhang Pg 15 -16 Appendix A discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) in a case where the document type of the processing target document is none of the plurality of document types (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs and also discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other). See Claim 1 for rationale, its parent claim. Regarding Claim 9, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 8, wherein the instruction message includes a third question for causing the large language model to reply (Zhang Fig 1 discloses a first, second, and third question being asked and the model replying) a reason (Zhang Pg 15 -16 Appendix A discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other, therefore the reason the document is labeled as other is due to the similarity threshold) that for the replies to the first question and the second question (Zhang Fig 1 discloses a first, second, and third question being asked and the model replying) . See Claim 1 for rationale, its parent claim. Regarding Claim 10, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 8, further comprising a determination unit configured to input (Zhang Pg 6 ¶02 discloses training the model with the input instructions which are character strings) a group of character strings obtained by the character recognition process (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses OCR1 and OCR2 being the character strings being obtained by the OCR model and this being the information used to base the model reply on) into a trained model (Zhang Pg 3 ¶03 discloses training the image classifier) to determine the document type of the processing target document (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs and also discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) , wherein the trained model is a trained model trained (Zhang Pg 3 ¶03 discloses training the image classifier) to output information indicating one of the plurality of document types (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs and also discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) . See Claim 1 for rationale, its parent claim. Regarding Claim 11, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 10, further comprising a display control unit (Kubo ¶0024, Fig 2, 28 discloses a display controller) configured to display at least one of the document type (Kubo ¶0024, discloses the display displaying a various number of operation screens) determined by the determination unit (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs and also discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) or the document type received from the large language model (Zhang Pg 15 -16 Appendix A discloses the image classifier can classify the type as other) on a display unit (Kubo ¶0024, Fig 2, 28 discloses a display controller) as the document type of the processing target document (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs and also discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) . See Claim 1 for rationale, its parent claim. Regarding Claim 14, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 8, further comprising a setting unit configured to set document types (Zhang Pg 14-15 Appendix A discloses selecting clusters of images into categories of document types) to be included as the plurality of document types (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs) . See Claim 1 for rationale, its parent claim. Regarding Claim 15, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 14, wherein in a case where the large language model outputs a first document type being a document type other than the plurality of document types (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs and also discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) as the reply to the second question (Zhang Fig 1 discloses a second question being asked) and a user selects the first document type as the document type of the processing target document (Kubo ¶0038-¶0039 discloses the user selecting the document type and the document type being received and displayed) , the setting unit sets the first document type (Zhang Pg 14-15 Appendix A discloses selecting clusters of images into categories of document types) as a document type included in the plurality of document types (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs) . See Claim 1 for rationale, its parent claim. Regarding Claim 16, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 1, further comprising a setting unit configured to set one (Zhang Pg 3 ¶02 discloses selecting the model) or more large language models (Zhang Pg 3 ¶02 discloses the multiples types of large language models and the model that is selected being the LLaVA model) to be caused to reply to the instruction message (Zhang Pg 10 ¶2 discloses LLaVA answering the questions) , wherein the transmission unit (Kubo ¶0025 and Fig 2 12 discloses a transmission and reception controller) transmits the instruction message (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, as high-quality instruction-following examples. This process requires GPT--4 to denoise the OCR results and develop specific questions to create complex instructions based on the input) to the set one or more large language models (Zhang Pg 3 ¶02 discloses the multiples types of large language models and the model that is selected being the LLaVA model) . See Claim 1 for rationale, its parent claim. Regarding Claim 17, Zhang teaches performing a character recognition process (Zhang Pg 4 ¶02 discloses using PaddleOCR to OCR all images) on an image of a processing target document (Zhang Pg 4 ¶02 and Pg 2 ¶02 discloses performing OCR (optical character recognition) on the desired image) ; generating an instruction message (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) based on a result of the character recognition process (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) , the instruction message being a message for causing a large language model to reply a document type (Zhang Fig 1 discloses asking what the name of the book in the image is) of the processing target document (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) ; transmitting the instruction message (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, as high-quality instruction-following examples. This process requires GPT--4 to denoise the OCR results and develop specific questions to create complex instructions based on the input) in order to obtain a reply to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) ; and receiving the reply (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, the examiner is interpreting the answer to be the reply) to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) . Zhang does not explicitly disclose an information processing method comprising. Kubo is in the same field of image analysis in which character recognition is performed for document interpretation. Further, Kubo teaches an information processing method (Kubo ¶0002 disclose an information processing method) comprising. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the computer structure and hardware to implement the method including the displaying of the documents as taught by Kubo to make an invention that can automatically display the document type and title to a user in real time a incorporate user input into the document display; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to achieve both a processing speed and character recognition accuracy as compared to a case where single image conversion processing is uniformly executed for an entire document as preprocessing prior to character recognition. (Kubo, ¶0005). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Regarding Claim 18, Zhang teaches perform a character recognition process (Zhang Pg 4 ¶02 discloses using PaddleOCR to OCR all images) on an image of a processing target document (Zhang Pg 4 ¶02 and Pg 2 ¶02 discloses performing OCR (optical character recognition) on the desired image) ; generate an instruction message (Zhang Figure 1 and Pg 2 ¶01 discloses using text only GPT 4 to generate the instruction message) based on a result of the character recognition process (Zhang Fig 1 and Pg 2 ¶02 discloses instruction messages and instruction following) , the instruction message being a message for causing a large language model to reply a document type (Zhang Fig 1 discloses asking what the name of the book in the image is) of the processing target document (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) ; transmit the instruction message (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, as high-quality instruction-following examples. This process requires GPT--4 to denoise the OCR results and develop specific questions to create complex instructions based on the input) in order to obtain a reply to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) ; and receive the reply (Zhang Fig 1 and Pg 2 ¶01 discloses where each conversation can be multiple turns of question and answer pairs, the examiner is interpreting the answer to be the reply) to the instruction message from the large language model (Zhang Fig 1 and Pg 2 ¶02-¶03 discloses the large language model to generate the conversations when asked about the input image) . Zhang does not explicitly disclose a non-transitory computer readable storage medium storing a program which causes a computer to. Kubo is in the same field of image analysis in which character recognition is performed for document interpretation. Further, Kubo teaches a non-transitory computer readable storage medium storing a program which causes a computer (Kubo ¶0005, ¶0023, ¶0029 discloses a non-transitory computer readable medium with a processor and memory to store a program an execute various programs) to. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by incorporating the computer structure and hardware to implement the method including the displaying of the documents as taught by Kubo to make an invention that can automatically display the document type and title to a user in real time a incorporate user input into the document display; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to achieve both a processing speed and character recognition accuracy as compared to a case where single image conversion processing is uniformly executed for an entire document as preprocessing prior to character recognition. (Kubo, ¶0005). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Claims 12-13 are rejected under 35 U.S.C. 103 as unpatentable over Zhang in view of Kubo in further view of Oka et al (JP 2017107445 A (using espace.net for description machine translation and google images for figure translation) hereafter referred to as Oka). Regarding Claim 12, Zhang in view of Kubo teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 11, wherein of the displayed document type (Kubo ¶0038-¶0039 discloses the user selecting the document type and the document type being received and displayed) , the display control unit accepts the correction (Kubo ¶0038-¶0039 discloses the user selecting the document type and the document type being received and displayed) . Zhang in view of Kubo does not explicitly disclose in a case where a user designates correction. Oka is in the same field of image analysis in which characteristic recognition is performed for document interpretation. Further, Oka teaches in a case where a user designates correction (Oka Pg 1 ¶08 and Pg 6 ¶04-¶05 and Fig 5 discloses in which an operator can easily designate a correct type in a case where the type of the form cannot be classified into one). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang in view of Kubo by incorporating user correction as taught by Kubo to make an invention that can automatically display the document type and title to a user in real time and incorporate user input correction into the model; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need for an apparatus in which an operator can easily designate a correct type in a case where the type of the form cannot be classified into one. (Oka, Pg 1 ¶07). Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Regarding Claim 13, Zhang in view of Kubo in further view of Oka teaches the information processing apparatus (Kubo ¶0005, ¶0007, ¶0020 disclose an information processing device) according to claim 12, wherein the display control unit displays (Kubo ¶0024, Fig 2, 28 discloses a display controller) the document type (Kubo ¶0024, discloses the display displaying a various number of operation screens) determined by the determination unit on the display unit (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs and also discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) as the document type of the processing target document (Zhang Pg 15 -16 Appendix A discloses the plurality of document categories that are used in the model including posters, books, and info graphs and also discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) , and in a case where the large language model outputs a first document type (Zhang Pg 15 -16 Appendix A discloses the image classifier can classify the type as other) being a document type other than the plurality of document types as the reply to the second question (Zhang Pg 15 -16 Appendix A discloses that if the similarity between the categories and the images is less than a threshold then it is classified as other) , displays the first document type (Zhang Pg 15 -16 Appendix A discloses the image classifier can classify the type as other) on the display unit (Kubo ¶0024, discloses the display displaying a various number of operation screens) as a candidate for a corrected character string (Oka Pg 1 ¶08 and Pg 6 ¶04-¶05 and Fig 5 discloses in which an operator can easily designate a correct type in a case where the type of the form cannot be classified into one) . See Claim 12 for rationale, its parent claim. Reference Cited 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Khandan, Nouna. "An intelligent hybrid model for identity document classification." arXiv preprint arXiv:2106.04345 (2021). (Year: 2021) discloses a method for using a hybrid model for image classification of personal documents . Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL ROBERTS whose telephone number is (571)272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /RACHEL L ROBERTS/Examiner, Art Unit 2674 /ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674 Application/Control Number: 18/809,516 Page 2 Art Unit: 2674 Application/Control Number: 18/809,516 Page 3 Art Unit: 2674 Application/Control Number: 18/809,516 Page 4 Art Unit: 2674 Application/Control Number: 18/809,516 Page 5 Art Unit: 2674 Application/Control Number: 18/809,516 Page 7 Art Unit: 2674 Application/Control Number: 18/809,516 Page 8 Art Unit: 2674 Application/Control Number: 18/809,516 Page 9 Art Unit: 2674 Application/Control Number: 18/809,516 Page 10 Art Unit: 2674 Application/Control Number: 18/809,516 Page 11 Art Unit: 2674 Application/Control Number: 18/809,516 Page 13 Art Unit: 2674 Application/Control Number: 18/809,516 Page 15 Art Unit: 2674 Application/Control Number: 18/809,516 Page 17 Art Unit: 2674 Application/Control Number: 18/809,516 Page 18 Art Unit: 2674 Application/Control Number: 18/809,516 Page 19 Art Unit: 2674 Application/Control Number: 18/809,516 Page 20 Art Unit: 2674 Application/Control Number: 18/809,516 Page 21 Art Unit: 2674 Application/Control Number: 18/809,516 Page 22 Art Unit: 2674 Application/Control Number: 18/809,516 Page 23 Art Unit: 2674 Application/Control Number: 18/809,516 Page 24 Art Unit: 2674 Application/Control Number: 18/809,516 Page 25 Art Unit: 2674
Prosecution Insights