DETAILED ACTION
Allowable Subject Matter
Claims 9-11, and 14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-7, 15, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Agarwal et al. (Pub. No. US 2019/0163971) in view of Chen et al. (Pub. No. US 2017/0351913) further in view of Chen et al. (Pub. No. US 2009/0028443 hereinafter “Chen2”).
Regarding claims 1, 15 and 20, Agarwal teaches a computing platform comprising: at least one processor [fig. 9 and related description]; at least one non-transitory computer-readable medium processor [fig. 9 and related description]; and program instructions stored on the at least one non-transitory computer-readable medium that are executable by the at least one processor such that the computing platform is processor [fig. 9 and related description]; configured to: receive, from a computing device associated with a given user, image data (input text document image) corresponding to an image of a physical artifact (handwritten documents) [Para. 61 “In one embodiment, such software in the memory 102 includes a program 100, which, when executed by the CPU 101, enables the computer 10 to perform a text line segmentation method described above.”; Para. 2 “Text line segmentation, i.e., segmenting lines of text from a document image (e.g. handwritten documents), is an important part of an intelligent character/word recognition (ICR/IWR) system” and 26 “The method starts with an input text document image (S100), which is a binary image including foreground pixels (i.e. black pixels) representing text content and background pixels (e.g. white pixels)”]; analyze the image data (input image) to identify contours (CCs) present in the image [Para. 26 “The method starts with an input text document image (S100), which is a binary image including foreground pixels (i.e. black pixels) representing text content and background pixels (e.g. white pixels)”]; sort the contours (CCs) into mutually exclusive groups (three subsets) [Para. 27 “Then, the CCs are divided into three subsets based on their sizes, using an average height of the CCs as the average character height to set the criteria for division (step S102). The three subsets are referred to as a first subset for normal size, a second subset for large size, and a third subset for small size”];
Agarwal also teaches having the first group (third subset) of contours [Para. 26 “Then, the CCs are divided into three subsets based on their sizes, using an average height of the CCs as the average character height to set the criteria for division (step S102). The three subsets are referred to as a first subset for normal size, a second subset for large size, and a third subset for small size.”].
However, Agarwal doesn’t explicitly teach the rest of the claim limitation.
Chen teaches wherein a first group (very small connected components) of contours (connected components) is discarded from further analysis [Para. 69 “FIG. 8 shows the detailed process of the image noise removal (6050). This process takes the connected components as input (6050.1). There are three sorts of connected components that are very likely to be noisy” and “Very small connected components”; Para. 73 “If the size of the connected component is smaller than a threshold (6050.2), then it is considered noise and will be discarded (6050.6)”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal’s connected-component categorization by incorporating Chen’s teaching of discarding very small connected component during noise removal before line processing. This medication improves Agarwal by eliminating noise removal before line processing. This medication improves Agarwal by eliminating noise components before line segmentation, thereby reducing false component assignment and unnecessary processing.
Agarwal teaches evaluate a second group (subset of normal size CCs) of contours (CCs) and thereby identify respective locations of lines in an array of lines in the image [Para. 28 “The subset of normal size CCs are used for line detection by applying Hough transform to the centroids of the CCs (step S103)” and Para. 38 “the result is a list of candidate lines and a list of remaining CCs (with their centroids and bounding boxes) (step S205)”], wherein each line comprises a respective set of contours (CCs) from the second group (subset of normal size CCS) of contours (CCs) [Para. 28 “The subset of normal size CCs are used for line detection by applying Hough transform to the centroids of the CCs” and 39 “For each candidate line, a line bounding box which bounds all CCs belonging to that line is obtained, and the height of the line bounding box is computed (step S206)”]
however, Agarwal doesn’t explicitly teach the rest of the claim limitation.
Chen teaches wherein each line comprises a respective set of contours (CCs) from the second group of contours that have a similar y-axis value (sufficient vertical overlapping) within the image [Para. 107 “With the assumption that the current character should have sufficient vertical overlapping with the text line it belongs to, if the overlapping length is less than a threshold (6180.08), the current character will not be a part of the current text line, and the flow goes back to 6180.04”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal’s connected-component categorization by incorporating Chen’s teaching of discarding very small connected component during noise removal before line processing. This medication improves Agarwal by eliminating noise removal before line processing. This medication improves Agarwal by eliminating noise components before line segmentation, thereby reducing false component assignment and unnecessary processing.
Agarwal teaches evaluating a third group (subset of large size CCs) of contours (CCs) and thereby add additional contours to the array of lines [Para. 31 “After text line detection, the subset of large size CCs are broken into smaller CCs (step S104)”; “Then each CC, including each of the normal size CCs, small size CCs, and the CCs resulting from breaking down large size CCs, is assigned to the nearest text line detected in step S103 (step S105) to generate the segmented text lines as output.”].
however, Agarwal doesn’t explicitly teach each contour in the third group intersects a given line in the array of lines.
Chen teaches teach each contour in the third group intersects (sufficient vertical overlapping) a given line in the array of lines [Para. 107 “With the assumption that the current character should have sufficient vertical overlapping with the text line it belongs to, if the overlapping length is less than a threshold (6180.08), the current character will not be a part of the current text line, and the flow goes back to 6180.04”; Para. 109 “If the current character passes all above checks, it belongs to the current text line, and will be added to the current text line (6180.10). The flow then goes back to 61080.03.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal’s connected-component categorization by incorporating Chen’s teaching of discarding very small connected component during noise removal before line processing. This medication improves Agarwal by eliminating noise removal before line processing. This medication improves Agarwal by eliminating noise components before line segmentation, thereby reducing false component assignment and unnecessary processing.
Agarwal also teaches grouping into individual characters (word/character segmentation) [Para. 60].
However, Agarwal in view of Chen doesn’t explicitly teach the rest of claim limitations.
Chen2 teaches for each line in the array of lines, group the line’s respective set of contours into individual characters [Para. 30 “A reliable layout analysis tool to locate text lines in a document image, and a reliable segmentation tool to segment text lines into individual characters.”; and Para. 52 “In step 42 the layout tool is used to locate the text lines, and in step 44 the segmentation tool is used to segment the text lines into individual characters.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal in view of Chen’s subsequent word/character segmentation stage by incorporating Chen2’s segmentation tool to segment the detected text lines into individual characters. This medication improves Agarwal by providing explicit per line character segmentation after line detection, thereby preparing cleaner inputs for OCR recognition. and
Agarwal teaches apply one or more character recognition techniques (recognition module) and thereby output a recognized character (output word or character) [Para. 3 “In the field of offline intelligent word/character recognition, a conventional ICR/IWR system typically includes the following stages for processing an input text document image: text line segmentation; word/character segmentation; and a recognition module (usually CNN of kNN), to generate output word or character”].
However, Agarwal in view of Chen doesn’t explicitly teach output a recognized character for each individual character in each line.
Chen2 teaches applying one of recognizing techniques (optical character recognition) and output a recognized character (label) for each individual character (individual character image) in each line [Para. 51 “Once the hybrid models of the present application have been trained, they may be used to undertake the prediction/classification operation for optical character recognition, such as shown in the flow diagram 40 of FIG. 4”; Para. 52 “In step 42 the layout tool is used to locate the text lines, and in step 44 the segmentation tool is used to segment the text lines into individual characters”; and Para. 53].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal in view of Chen’s subsequent word/character segmentation stage by incorporating Chen2’s segmentation tool to segment the detected text lines into individual characters. This medication improves Agarwal by providing explicit per line character segmentation after line detection, thereby preparing cleaner inputs for OCR recognition.
Regarding claims 5 and 19, Agarwal teaches wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to sort the contours into mutually exclusive groups comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: establish a contour size baseline (average character height) comprising a minimum size threshold (first predefined threshold size) and a maximum size threshold (second threshold predefined size) [Para. 27]; identify contours that fall within the minimum size threshold and the maximum size threshold as the second group of contours (first subset for normal size) [Para. 27]; identify contours (CCs) that exceed the maximum size threshold as the third group of contours (second subset for large size) [Para. 27].
Agarwal doesn’t explicitly teach the rest of claim limitations.
Chen teaches identify contours (connected components) that do not meet minimum size threshold as the first group of contours (very small connected components) [Para. 69, 70, and 73].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal’s connected-component categorization by incorporating Chen’s teaching of discarding very small connected component during noise removal before line processing. This medication improves Agarwal by eliminating noise removal before line processing. This medication improves Agarwal by eliminating noise components before line segmentation, thereby reducing false component assignment and unnecessary processing.
Regarding claim 6, Agarwal in view of Chen further Chen2 teaches all claim limitations above. Furthermore, Chen teaches wherein the minimum size threshold comprises a minimum possible size of any given contour for a given character font presented in the physical artifact [Para. 107].
Regarding claim 7, Agarwal teaches wherein the maximum size threshold comprises a maximum possible size of any given contour for a given character font presented in the physical artifact [Para. 27].
Claims 2-4, and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Agarwal et al. (Pub. No. US 2019/0163971) in view of Chen et al. (Pub. No. US 2017/0351913) further in view of Chen et al. (Pub. No. US 2009/0028443 hereinafter “Chen2”) and further in view of Nepomniachtchi (Pub. No. US 2009/0185241 hereinafter “Nepo”).
Regarding claims 2 and 16, Agarwal in view Chen further in view of Chen2 doesn’t explicitly teach the claim limitations.
However, Nepo teaches wherein the image was captured using a camera of the computing device (mobile device) [Para. 54 “In some embodiments, checks 100, payment coupons 200, or other documents might be imaged using a mobile device. This connection might be wired or wireless. In this way the mobile device may connect to an external camera and receive images from the camera”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal in view of Chen further in view of Chen2’s document image input workflow by using Nepo’s mobile device camera capture to supply the document image to Agarwal’s segmentation process. This modification improves Agarwal for supporting remote capture before OCR preprocessing.
Regarding claims 3 and 17, Agarwal in view Chen further in view of Chen2 doesn’t explicitly teach the claim limitations.
However, Nepo teaches wherein the image was captured using a camera of the computing device (mobile device), and wherein the physical artifact comprises a physical bank check (personal check) [Para. 54].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal in view of Chen further in view of Chen2’s document image input workflow by using Nepo’s mobile device camera capture to supply the document image to Agarwal’s segmentation process. This modification improves Agarwal for supporting remote capture before OCR preprocessing.
Regarding claims 4 and 18, Agarwal teaches wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to analyze the image data to identify contours present in the image comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: identify each continuous set of pixels (group of connected foreground pixels) in the image data as a respective contour (CC) [Para. 26].
Claims 8, 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Agarwal et al. (Pub. No. US 2019/0163971) in view of Chen et al. (Pub. No. US 2017/0351913) further in view of Chen et al. (Pub. No. US 2009/0028443 hereinafter “Chen2”) and further in view of Lou (Pub. No. 2003/0198386).
Regarding claim 8, Agarwal teaches wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to evaluate the second group of contours (subset of normal size CCs) and thereby identify the respective locations of each of the array of lines (candidate lines) in the image comprise program instructions that are executable by the at least one processor such that the computing platform [Para. 28 and 38].
However, Agarwal in view of Chen further in view of Chen2 the rest of claim limitations.
Lou teaches scan the image from a first edge to a second edge; and while scanning the image: identify a first contour [Para. 35, 48, and 57]; draw a first bounding box (character box) enclosing the first contour and thereby locate a first line (current line definition) comprising the first contour [Para. 35, 48, and 57]; identify an additional contour [Para. 35, 48, and 57]; draw a projected box enclosing the additional contour, wherein the projected box is extended vertically around the additional contour by a threshold amount (1.1-1.2 times) [Para. 35, 48, and 57]; and extend the projected box laterally in a left-ward direct (left) to identify any preceding contours having a respective line (text line) to which the additional contour may be added [Para. 35, 48, and 57].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal in view of Chen further in view of Chen2’s second group contour line detection process by incorporating Luo’s bounding area merging process using a character box, current line definition, expanded neighbor box by 1.1-1.2 times and left spatial ordering to identify nearby candidates to a text line. This modification improves Agarwal by improving line formation when character components are spatially adjacent but irregularly positioned.
Regarding claim 12, Agarwal teaches wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to for each line in the array of lines, group the line’s respective set of contours into individual characters (word/character segmentation) comprise program instructions that are executable by the at least one processor such that the computing platform is configured to, for each line in the array of lines [Para. 61 and 60].
Agarwal in view Chen further in view of Chen2 doesn’t explicitly teach the rest of claim limitations.
However, Luo teaches begin scanning the line (scan the character box line-by-line) [Para. 48]; identify a beginning of a first contour in the respective set of contours [Para. 48]; based on identifying the beginning of the first contour, begin adding the first contour to a buffer (container) [Para. 48]; and identify an end of the first contour [Para, 48].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal in view of Chen further in view of Chen2’s second group contour line detection process by incorporating Luo’s bounding area merging process using a character box, current line definition, expanded neighbor box by 1.1-1.2 times and left spatial ordering to identify nearby candidates to a text line. This modification improves Agarwal by improving line formation when character components are spatially adjacent but irregularly positioned.
Regarding claim 13, Agarwal in view Chen further in view of Chen2 doesn’t explicitly teach the rest of claim limitations.
However, Luo teaches comprising program instructions that are executable by the at least one processor such that the computing platform is configured to: make a first determination that the end of the first contour (character box) is reached and that a minimum character area (threshold) has not been reached [; based on the first determination, make a second determination that the first contour is to be grouped as an individual character [Para. 35, 45 and 48]; and based on the second determination: draw a bounding box enclosing the first contour; clear the buffer (discarded without storing them); and continue scanning the line [Para. 35, 45 and 48].
It would have been obvious to one of ordinary skill in the art before the effective filing date of modify Agarwal in view of Chen further in view of Chen2’s second group contour line detection process by incorporating Luo’s bounding area merging process using a character box, current line definition, expanded neighbor box by 1.1-1.2 times and left spatial ordering to identify nearby candidates to a text line. This modification improves Agarwal by improving line formation when character components are spatially adjacent but irregularly positioned.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-7 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-0101 (IN USA OR CANADA) or 571-272-1000.
/SOLOMON G BEZUAYEHU/ Primary Examiner, Art Unit 2666
Prosecution Insights