DETAILED ACTION
Status of the Claims
Original claims 1-20 are pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on August 23, 2024, and December 3, 2024, are being considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: USER INTERFACE FOR CHARACTER RECOGNITION.
Claim Interpretation
Claims are given their broadest reasonable interpretation (BRI) during examination. MPEP 2111. Under BRI, the words of a claim are given their plain meaning, unless such meaning is inconsistent with the specification. MPEP 2111.01, Subsection I. The plain meaning of a term is the ordinary and customary meaning given to the term by those of ordinary skill in the art at the relevant time. Id.
Claim 1 recites a “document image” at line 5. Claims 19 and 20 recite the same term. The plain meaning of a “document image” is an image of a document. The plain meaning of a document is, for example, “Something written, inscribed, etc., which furnishes evidence or information upon any subject, as a manuscript, title-deed, tomb-stone, coin, picture, etc.” Oxford English Dictionary, definition 4.a. So, while some documents may be printed on paper, the scope of the term “document” is broad enough to cover other ways of providing text on other types of objects. This plain meaning is not inconsistent with the specification, which describes paper documents (e.g., [0024], as published) but does not specifically exclude other types of documents.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 2 and 5-18 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 recites “obtain a unit character” at line 3. It is unclear whether this obtained unit character is the same as, or different from, the “a unit character” extracted at line 4 of claim 1.
On the one hand, use of the article “a” suggests that a different, distinct element is being introduced to the claim. On the other hand, using the same term “unit character” suggests that both elements are the same. This ambiguity makes the scope of the claim unclear and renders the claim indefinite.
Examiner suggests amending claim 2 to either (a) clearly refer to the earlier unit character of claim 1 such as by reciting “the unit character” or (b) clearly differentiate the later unit character such as by reciting “a second unit character” and making corresponding amendments through the remainder of the claims.
For the purpose of examination with respect to the prior art, the claim is being interpreted to cover either possibility noted above. MPEP 2173.06.
Claims 5-18 are also indefinite at least because they include the limitations of claim 2.
Claim 6 further recites “a unit character region” at line 3, which is further indefinite for substantially the same reasons discussed above because it is unclear whether this character region is the same as, or different from, the “a unit character region” of claim 2.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 5-6 and 19-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by ‘Nanaumi’ (US 2017/0262722 A1).
Regarding claim 1, Nanaumi discloses an information processing apparatus (e.g., [0081], computer) comprising:
at least one memory that stores instructions (e.g., [0081], storage medium); and
at least one processor that executes the instructions (e.g., [0081], one or more processors) to:
extract a character string (e.g., [0053], Fig. 8, character string 803) and a unit character included in the character string (e.g., [0053], Fig. 8, any of unit characters 802) based on a result of character recognition processing (e.g., [0053], display in Fig. 8 is based on character recognition processing described in Fig. 7, such as at S701, S702 and S707) on a document image (e.g., [0043], the character recognition processing is performed based on a captured image; e.g., [0022]-[0023], in a non-limiting example, the subject 105 of the image is a tire including serial ID text, which falls within the BRI of a “document” – see Claim Interpretation above);
generate context information related to the unit character (Several aspects of Nanaumi’s disclosure fall within the BRI of such context information; In one example, the delineated character areas 802 in Fig. 8 are context information related to each unit character) based on the extracted character string (i.e., each individual character falls within the extracted character string) and attribute information indicating an attribute of the character string (e.g., Fig. 8, the rectangular area 801 defining the area where the character string was detected, which indicates a position or location attribute of the character string); and
display the extracted unit character and the context information corresponding to the unit character on a UI screen (e.g., Fig. 8, extracted characters and, for example, their delineated character areas 802 are displayed on a UI screen).
Regarding claim 2, Nanaumi discloses the information processing apparatus according to claim 1, wherein the at least one processor executes the instructions further to:
obtain a unit character from a character list including the unit character (Note the ‘112(b) rejection; If the unit characters are the same, the list could include the set of extracted characters shown in Fig. 10A [i.e., B, 5, L, 9, A, 0, 1], the alphabet (which is a list of possible characters), etc.; If the unit characters are different, the list could include the extracted character 1001 and similar characters 1003-1005 illustrated in Fig. 10A); and
highlight a unit character region (e.g., [0064], Fig. 10A, highlighted region 1001) and the context information corresponding to the obtained unit character (e.g., [0064], Fig. 10A, highlighted character area 1002) on a UI screen.
Regarding claim 5, Nanaumi discloses the information processing apparatus according to claim 2, wherein the UI screen includes a button (e.g., Fig. 10A, button 404) to switch displaying and hiding the context information of the obtained unit character (e.g., Figs. 10A-B, context information of selected unit character is displayed by highlighting box when button 404 is switched on; e.g., Fig. 13B, display of selected character is disabled and hidden when button 404 is switched off).
Regarding claim 6, Nanaumi discloses the information processing apparatus according to claim 2, wherein the at least one processor executes the instructions further to:
highlight a unit character region corresponding to the obtained unit character (e.g., [0064], Fig. 10A, selected character is highlighted at 1002) and a character string region including the obtained unit character on the UI screen (e.g., Fig. 10A, rectangular region 801 highlights the region of the string as a whole).
Regarding claim 19, Examiner notes that the claim recites a method that is substantially the same as the method performed by the apparatus of claim 1. Nanaumi discloses the apparatus of claim 1 (see above). Accordingly, claim 19 is also rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nanaumi for substantially the same reasons as claim 1.
Regarding claim 20, Examiner notes that the claim recites a non-transitory computer-readable storage medium storing a program which causes a computer to execute a method that is substantially the same as the method performed by the apparatus of claim 1. Nanaumi discloses the apparatus of claim 1 (see above). Nanaumi further discloses implementing its apparatus using a non-transitory computer-readable storage medium storing a program which causes a computer to execute the method of the apparatus (e.g., [0081]). Accordingly, claim 20 is also rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nanaumi for substantially the same reasons as claim 1.
Claim(s) 1-4, 8-9 and 19-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by ‘Todoroki’ (US 2022/0180121 A1).
Regarding claim 1, Todoroki discloses an information processing apparatus (e.g., Fig. 3) comprising:
at least one memory that stores instructions (e.g., [0044], Fig. 3, RAM 14C and/or ROM 14B); and
at least one processor that executes the instructions (e.g., [0044]-[0046], Fig. 3, CPU 14A) to:
extract a character string (e.g., Fig. 8, text reading “FUJI HANAKO”) and a unit character included in the character string (any of the characters within the string can be considered a “unit” character as they make up one part/unit of the string) based on a result of character recognition processing on a document image (e.g., [0049]-[0050], [0057]-[0058], document image is read and characters are recognized);
generate context information related to the unit character (e.g., Fig. 8 shows a variety of context information, such as the item name, a number of results, whether each entry has been checked, etc.) based on the extracted character string (e.g., the number of results [10 for the example in Fig. 8] is for a given extracted character string) and attribute information indicating an attribute of the character string (e.g., Fig. 8, the record ID and/or item name are attributes of the character string); and
display the extracted unit character (e.g., Fig. 8, CHECK RESULT column displays extracted characters) and the context information corresponding to the unit character (see the various context information noted above) on a UI screen (Fig. 8 illustrates a UI screen).
Regarding claim 2, Todoroki discloses the information processing apparatus according to claim 1, wherein the at least one processor executes the instructions further to:
obtain a unit character from a character list including the unit character (Note the ‘112(b) rejection; e.g., Figs. 6-8, character to be checked is selected/obtained from list at left); and
highlight a unit character region and the context information corresponding to the obtained unit character on a UI screen (e.g., Figs. 6-8 show display of unit character regions in SCAN IMAGE and they are highlighted with the context information by displaying them [instead of other, non-selected character information], checking or unchecking of the check box, etc.).
Regarding claim 3, Todoroki discloses the information processing apparatus according to claim 1, wherein the context information includes
the attribute information of the character string (e.g., Fig. 6-8, the record ID is an attribute of the character string),
information of an image region corresponding to the character string in the document image (e.g., Figs. 6-8, SCAN IMAGE),
attribute information of an adjacent character string having a specific relationship with the character string (e.g., Fig. 7, CHECK RESULT column, text strings such as “COMPANY WORKER” are adjacent to check mark characters and are associated in the same row with specific check mark characters, the association being attribute information),
information of an image region corresponding to the adjacent character string in the document image (e.g., Fig. 7, CHECK RESULT column, the text shows the label of the image region corresponding to the adjacent character, thus identifying the meaning of the checked or unchecked state of the checkmark),
attribute information of a character string indicating a type of the document image (e.g., Figs. 6-8, ITEM NAME is an attribute indicating a type of the document image, specifically a type of data included in the document image), and
information of an image region of the character string indicating the type in the document image (e.g., Figs. 6-8, the specific value of ITEM NAME, such as YEAR, CHECK MARK or NAME).
Regarding claim 4, Todoroki discloses the information processing apparatus according to claim 3, wherein the specific relationship is a relationship between a key and a value (e.g., Fig. 7, third row, “SELF-EMPLOYED” is the key and unchecked is the value).
Regarding claim 8, Todoroki discloses the information processing apparatus according to claim 2, wherein the at least one processor executes the instructions further to:
display a certainty factor of character recognition of the obtained unit character on the UI screen (e.g., Figs. 6-8, CERTAINTY column).
Regarding claim 9, Todoroki discloses the information processing apparatus according to claim 8, wherein the at least one processor executes the instructions further to:
display the context information of the obtained unit character on the UI screen in a case where the certainty factor of character recognition of the obtained unit character is smaller than a threshold (e.g., Fig. 8, context information of obtained unit characters – e.g., item name as explained above with respect to claim 1 – is displayed even in cases where the certainty factor is smaller than a threshold of HIGH; e.g., the context information is displayed for extracted characters with LOW certainty; Note that the claim recites only a positive requirement and does not specify what is/isn’t displayed in a case where the certainty factor is not smaller than the threshold).
Regarding claim 19, Examiner notes that the claim recites a method that is substantially the same as the method performed by the apparatus of claim 1. Todoroki discloses the apparatus of claim 1 (see above). Accordingly, claim 19 is also rejected under 35 U.S.C. 102(a)(1) as being anticipated by Todoroki for substantially the same reasons as claim 1.
Regarding claim 20, Examiner notes that the claim recites a non-transitory computer-readable storage medium storing a program which causes a computer to execute a method that is substantially the same as the method performed by the apparatus of claim 1. Todoroki discloses the apparatus of claim 1 (see above). Todoroki further discloses implementing its apparatus using a non-transitory computer-readable storage medium storing a program which causes a computer to execute the method of the apparatus (e.g., [0007], Fig. 3, [0043] et seq.). Accordingly, claim 20 is also rejected under 35 U.S.C. 102(a)(1) as being anticipated by Todoroki for substantially the same reasons as claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7 and 10-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nanaumi in view of ‘Marcus’ (“Principles of Effective Visual Communication for Graphical User Interface Design,” 1995).
Regarding claim 7, Nanaumi teaches the information processing apparatus according to claim 6.
Nanaumi teaches highlighting the unit character region corresponding to the obtained unit character and the character string region including the obtained unit character on the UI screen in different shapes (e.g., Fig. 10A, unit character region 1002 is highlighted with a plain rectangle shape, while character string region 801 is highlighted by a border shape that is rectangular with added circles at the corners).
Nanaumi’s graphical user interface (GUI) illustrations are in black and white (e.g., Fig. 10A) and the text of Nanaumi does not discuss color. Accordingly, Nanaumi does not explicitly teach highlighting in different colors.
However, Marcus does teach principles of GUI design, including that color can be used to providing highlighting (e.g., page 429, Color/Texture, 1st paragraph), that different colors can convey relationships between different elements (e.g., page 430, Color Organization), that different colors may be used redundantly with different shapes (e.g., page 431, par. above Color Emphasis), and that use of color provides various advantages including improved enjoyment and comprehensibility relative to black-and-white (e.g., page 430, left column, bulleted list and below).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the apparatus of Nanaumi to highlight different regions with not only different shapes but also different colors as taught by Marcus in order to improve the apparatus with the reasonable expectation that this would result in an apparatus that exhibited the various benefits of color-based highlighting, such as improved comprehensibility and enjoyment. This technique for improving the apparatus of Nanaumi was within the ordinary ability of one of ordinary skill in the art based on the teachings Marcus.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Nanaumi and Marcus to obtain the invention as specified in claim 7.
Regarding claim 10, Nanaumi in view of Marcus teaches the information processing apparatus according to claim 7, and Nanaumi further teaches that the at least one processor executes the instructions further to:
store a pair of the obtained unit character and a similar character that is similar to the obtained unit character in the memory (e.g., [0057]-[0061], Fig. 9, S902-904, a display-item list of alternative characters is created and stored, the alternative characters being “similar” at least because they are included in serial ID text strings that are similar to the extracted text string); and
highlight a character region of the similar character that is similar to the obtained unit character on the UI screen based on data stored in the memory (e.g., [0064], Fig. 10A, character regions of the similar characters are displayed and highlighted by being displayed in teardrop-shaped regions 1003-1005).
Regarding claim 11, Examiner notes that the claim requires further highlighting the character region of the similar character in a different color. This is obvious over Nanaumi in view of Marcus for substantially the same reasons provided above with respect to claim 7. I.e., the same reasons why it would be obvious to highlight the unit character region and the character string region in different colors apply to further highlighting the character region of the similar character in a different color.
Regarding claim 12, Nanaumi in view of Marcus teaches the information processing apparatus according to claim 10, and Nanaumi further teaches that
the UI screen further includes a region (e.g., Fig. 10A, region 404) to receive an input regarding whether to highlight the character region of the similar character (e.g., Fig. 9, the display-item list illustrated in Fig. 10A is only displayed at S908 if button 404 is determined to be enabled at S901; I.e., an input enabling button 404 is an input regarding whether to display the display-item list of Fig. 10A and thus to highlight the character region of the similar character), and
the at least one processor executes the instructions further to:
highlight the character region of the similar character on the UI screen (i.e., display the display-item list at S908) in a case where an input indicating highlighting of the character region of the similar character is received (i.e., YES path from S901).
Regarding claim 13, Nanaumi in view of Marcus teaches the information processing apparatus according to claim 10, and Nanaumi further teaches that the at least one processor executes the instructions further to:
display a screen showing a recognition result on a main screen of the UI screen (e.g., Fig. 10B, main screen shows image with text and recognition result) and switch (e.g., Fig. 9, S901 and S910 switch on keyboard display at S912) and display plural detail screens of the recognition result on a side screen of the UI screen (e.g., Fig. 10B, keyboard is a bottom side screen; Each key in the keyboard can be considered a detail screen and they are “of the recognition result” at least because the keys are used to input corrections to the recognition result – e.g., [0068]-[0069]).
Allowable Subject Matter
Claims 14-18 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
‘Ohira’ (US 2022/0198190 A1)
See various UIs illustrated in the drawings
‘Goodsitt’ (US 2022/0067288 A1)
Highlights different portions of OCR output with different colors corresponding to different levels of confidence – e.g., Figs. 4 and 6B
‘Kawara’ (US 2021/0019554 A1)
Example of using a closed dictionary for OCR processing of paper document images, similar to how Nanaumi uses a closed dictionary of tire codes for OCR processing of tire images
‘Aravamudan’ (US 2007/0288456 A1)
Example of modifying a user interface based on an operation history of a user – e.g., Fig. 8
‘Tada’ (US 2004/0255218 A1)
User interface at Fig. 24
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEOFFREY E SUMMERS whose telephone number is (571)272-9915. The examiner can normally be reached Monday-Friday, 7:00 AM to 3:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEOFFREY E SUMMERS/Examiner, Art Unit 2669
Prosecution Insights