CTNF 18/779,808 CTNF 98066 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Information Disclosure Statement The information disclosure statement (IDS) submitted on 07/24/2024 was filed and is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15-aia AIA Claim(s) 1, 2, 4-6, 11, 12, 14, 19 is/are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Yang (“Improve newspaper digitalization efficacy with generic document segmentation tool using Amazon Textract”, 2021) . Regarding claims 1, 11, 19, Yang teaches A spatial alignment computer system (Yang, pg 6, ¶3-6, reproduced below: PNG media_image1.png 456 1144 media_image1.png Greyscale . “align the boundaries” shows “spatial alignment”) for string alignment (Yang, see nearest image above, “align the boundaries” shows string alignment as the bounding boxes contain strings) within a document processed using an optical character recognition (OCR) tool (Yang, pg 1, ¶2, reproduced below: PNG media_image2.png 174 1150 media_image2.png Greyscale . “newspapers articles” is being interpreted as “document” that is processed. Amazon Textract is being interpreted as OCR tool), the spatial alignment computer system comprising: at least one memory (Yang, see pg 1, ¶2 image above, which shows the newspaper digitization method requires using memory since machine learning and segmentation are involved); and at least one processor in communication with the at least one memory (Yang, see pg 1, ¶2 image above, which shows the newspaper digitization method requires using at least one processor since machine learning and segmentation are involved), wherein the at least one processor is programmed to: receive a plurality of bounding boxes (Yang, see pg 6 image above, “bounding boxes”) of a document scanned (Yang, see pg 1 image above, “newspaper” is an example scanned document) using an OCR tool (Yang, see pg 1, ¶2 image above, “Amazon Textract” is an OCR tool); identify a centroid of each bounding box of the plurality of bounding boxes (Yang, see pg 6 image above, “centroid of each bounding box”); calculate coordinates for each centroid of each bounding box of the plurality of bounding boxes using a weighted Euclidean distance approach (Yang, see pg 6 image above, “Euclidean distances of the centroid of each bounding box”. Bounding boxes are being interpreted as having coordinates, otherwise a computer or user would be unable to find the bounding box); sort the weighted Euclidean distance of the centroid of each bounding box (Yang, see pg 6 image above, “Euclidean distances of the centroid of each bounding box”. ) in ascending order to obtain a sorting index (Yang, see pg 6 image above, “we sorted the bounding boxes by the coordinates, according to a human read order from top to bottom, and left to right”. Sorting is being interpreted to involve a sorting index. Human read order is being interpreted to involve an ascending order); and based upon the sorting index (Yang, see pg 6 image above, “we sorted the bounding boxes”, which is being interpreted to involve an index), align one or more output strings associated with each bounding box of the plurality of bounding boxes (Yang, see pg 6 image above, “we sorted the bounding boxes by the coordinates, according to a human read order from top to bottom, and left to right”. The purpose of the OCR tool is to extract text, or strings. These are sorted according to human read order). Regarding claim 2, Yang teaches The spatial alignment computer system of claim 1, wherein the coordinates for each centroid of each bounding box of the plurality of bounding boxes (Yang, pg 6, ¶3-6, reproduced below: PNG media_image1.png 456 1144 media_image1.png Greyscale . “centroid of each bounding box” and “coordinates”) are determined from a reference point defined at a top-left corner of a document page (Yang, pg 5, sample images, which show the document page has a top-left corner. Using the top-left corner as a reference point is a common reference) in which the one or more aligned output strings are displayed (Yang, see pg 6 image above, “we sorted the bounding boxes by the coordinates, according to a human read order from top to bottom, and left to right”. The purpose of the OCR tool is to extract text, or strings). Regarding claim 4, Yang teaches The spatial alignment computer system of claim 1, wherein a vertical spacing between two rows of a document (Yang, pg 9, the sample newspaper image shows vertical spacing between rows of the document) displaying the one or more aligned output strings (Yang, pg 9, sample newspaper image, which shows the text displayed) is determined based at least in part on a paper size of the document (Yang, pg 9, sample newspaper image, which shows the text alignment is based on at least the paper size of the document, otherwise text would go outside the margins). Regarding claim 5, Yang teaches The spatial alignment computer system of claim 4, wherein the paper size of the document includes at least one of: (i) a letter size; (ii) a legal size; (iii) a tabloid size (Yang, pg 9, sample newspaper image, which is a tabloid size. Examiner notes the phrasing of this claim is being interpreted as an “OR” statement, so only one item will be addressed); (iv) a ledger size; (v) a junior legal size; (vi) a half letter size; (vii) a government letter; or (viii) a government legal size. Regarding claim 6, Yang teaches The spatial alignment computer system of claim 4, wherein the vertical spacing between two rows of the document displaying the one or more aligned output strings is further determined based at least in part on a font type or a font size (Yang, pg 9, sample newspaper image, which shows small and large font size influences the vertical spacing between two rows of the document for the text). Claim 12 is rejected using the same rationale and motivation as applied to claim 2 discussed above. Claim 14 is rejected using the same rationale and motivation as applied to claim 4 discussed above . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 3, 7, 13, 15, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang, in view of Aboulkacim (“Form data enriching using a post OCR clustering process”, 2022) . Regarding claim 3, Yang teaches The spatial alignment computer system of claim 1, wherein a vertical coordinate component of the calculated coordinates for each centroid (Aboulkacim, pg 31, bulletpoint before Section 7.2: “As the Euclidean distance was calculated by using midpoints of each text box”, which is being interpreted to involve a centroid) of each bounding box of the plurality of bounding boxes is more heavily weighted than a horizontal coordinate component (Aboulkacim, pg 7, Section 2.5.2, reproduced below: PNG media_image3.png 422 794 media_image3.png Greyscale . Which shows the vertical coordinate, interpreted as involving the y variable, is more heavily weighted than the horizontal coordinate, interpreted as involving the x variable.). Yang and Aboulkacim are considered to be analogous art because they are directed to OCR and layout analysis. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for OCR and layout analysis (as taught by Yang) to include a more heavily weighted vertical coordinate (as taught by Aboulkacim) because the combination provides an improvement to data enrichment for OCR tools (Aboulkacim, Abstract). Regarding claim 7, Yang teaches The spatial alignment computer system of claim 1, wherein a weight ω_2 of a vertical coordinate component (Aboulkacim, see nearest image below, the 2 weight in the equation is being interpreted as the second weight) of the calculated coordinates of the centroid of each bounding box of the plurality of bounding boxes is at least PNG media_image4.png 124 134 media_image4.png Greyscale times a weight ω_1 of a horizontal coordinate component (Aboulkacim, see nearest image below, 0.5 would be the smaller weight for the horizontal coordinate. The 2 is being interpreted as at least larger than 0.5) of the calculated coordinates of the centroid of each bounding box of the plurality of bounding boxes (Aboulkacim, pg 7, Section 2.5.2, reproduced below: PNG media_image3.png 422 794 media_image3.png Greyscale . Which shows the vertical coordinate, interpreted as involving the y variable, is more heavily weighted than the horizontal coordinate, interpreted as involving the x variable. The midpoint calculated [pg 31, bulletpoint before section 7.2] is being interpreted as the centroid for each text box, or bounding box), wherein h corresponds with a length of a document page (Aboulkacim, pg 21, Section 5, ¶1: “The system has been tested by inputting different images of handwritten filled in forms with the same image of a blank form.” “image of a blank form” is being interpreted to have a height, or a length, or a document page), r corresponds with an aspect ratio of PNG media_image5.png 94 54 media_image5.png Greyscale (Aboulkacim, pg 21, Section 5, ¶1: “The system has been tested by inputting different images of handwritten filled in forms with the same image of a blank form.” “image of a blank form” is being interpreted to have an aspect ratio of 1/sqrt(2), the common aspect ratio for paper sizes), and Δ h corresponds with a height of a single line or row (Aboulkacim, pg 20, Figure 18, the bounding boxes are interpreted to have a height and corresponds to a height of a single line or row). Yang and Aboulkacim are considered to be analogous art because they are directed to OCR and layout analysis. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for OCR and layout analysis (as taught by Yang) to include a more heavily weighted vertical coordinate (as taught by Aboulkacim) because the combination provides an improvement to data enrichment for OCR tools (Aboulkacim, Abstract). Claim 13 is rejected using the same rationale and motivation as applied to claim 3 discussed above. Claims 15 and 20 is rejected using the same rationale and motivation as applied to claim 7 discussed above . 07-21-aia AIA Claim (s) 8, 9, 10, 16, 17, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang, in view of Liang (“Comparative study of table layout analysis”, 2018) . Regarding claim 8, Yang teaches The spatial alignment computer system of claim 1, wherein the at least one processor is further programmed to: based upon the sorting index for a first version of the OCR tool (Yang, see pg 6 image above, “we sorted the bounding boxes”, which is being interpreted to involve an index), align a first set of one or more output strings associated with each bounding box of a plurality of bounding boxes generated by the first version of the OCR tool (Yang, see pg 6 image above, “we sorted the bounding boxes by the coordinates, according to a human read order from top to bottom, and left to right”. The purpose of the OCR tool is to extract text, or strings. These are sorted according to human read order); However, Yang does not appear to explicitly teach second version of the OCR tool. Pertaining to the same field of endeavor, Liang teaches based upon a sorting index for a second version of the OCR tool, align a second set of one or more output strings associated with each bounding box of a plurality of bounding boxes (Liang, see Section 1.2 image below: “segmenting physical contents” is being interpreted as involving bounding boxes) generated by the second version of the OCR tool (Liang, see Section 3.3.4.3 image below, the OCR tool comparison is show using performance metrics in Table 6, which is being interpreted as having first and second version of the OCR tool); compare the spatial alignment of the first set of the one or more output strings to the second set of the one or more output strings (Liang, pg 10, last 3 paragraphs before Section 1.2, reproduced below: PNG media_image6.png 532 1204 media_image6.png Greyscale . Layout analysis is being interpreted to involve “spatial alignment”. “Connected components” are being interpreted to involve output strings. See Section 3.3.4.3 image below, the metric compares the results of the OCR tools); and output a performance metric indicating how the first version of the OCR tool compares to the second version of the OCR tool (Liang, pg 33, Section 3.3.4.3, reproduced below: PNG media_image7.png 630 884 media_image7.png Greyscale . Table 6, each column shows a different OCR tool, which is interpreted as having a first and second version of the OCR tool. F1_micro_avg, is an example performance metric that is used to compare the OCR tools). Yang and Liang are considered to be analogous art because they are directed to OCR and layout analysis. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for OCR and layout analysis (as taught by Yang) to compare OCR tools (as taught by Liang) because the combination provides an improvement to OCR and layout analysis (Liang, Abstract). Further, one with ordinary skill in the art would know that it is common practice to compare tools in the scientific and research disciplines. Regarding claim 9, Yang teaches The spatial alignment computer system of claim 8, wherein the performance metric indicates whether the spatial alignments of the different versions of the OCR tools are compatible or non-compatible (Liang, see Section 3.3.4.3 image above, “Since these two alternatives perform well consistently in all these experiments, we adopted these two alternatives as two branches of solution III and applied them to produce evaluation result at the end”. Which shows the two OCR tools are compatible, while the third one was not). Regarding claim 10, Yang teaches The spatial alignment computer system of claim 1, wherein the at least one processor is further programmed to: However, Yang does not appear to explicitly teach second version of the OCR tool. Pertaining to the same field of endeavor, Liang teaches compare the alignment of the one or more output strings from a first version of the OCR tool to an alignment of one or more output strings from a second version of an OCR tool (Liang, pg 10, last 3 paragraphs before Section 1.2, reproduced below: PNG media_image6.png 532 1204 media_image6.png Greyscale . Layout analysis is being interpreted to involve “spatial alignment”. “Connected components” are being interpreted to involve output strings. See Section 3.3.4.3 image above, the metric compares the results of the OCR tools); and determine from the comparison whether the first version and the second version of the OCR tools are compatible (Liang, see Section 3.3.4.3 image above, “Since these two alternatives perform well consistently in all these experiments, we adopted these two alternatives as two branches of solution III and applied them to produce evaluation result at the end”. Which shows the two OCR tools are compatible, while the third one was not). Yang and Liang are considered to be analogous art because they are directed to OCR and layout analysis. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for OCR and layout analysis (as taught by Yang) to compare OCR tools (as taught by Liang) because the combination provides an improvement to OCR and layout analysis (Liang, Abstract). Further, one with ordinary skill in the art would know that it is common practice to compare tools in the scientific and research disciplines. Claim 16 is rejected using the same rationale and motivation as applied to claim 8 discussed above. Claim 17 is rejected using the same rationale and motivation as applied to claim 9 discussed above. Claim 18 is rejected using the same rationale and motivation as applied to claim 10 discussed above . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure : Vincent et al (US 7697758 B2, 2006) discloses OCR post-processing and centroids of bounding boxes along with distances between bounding boxes. Ramalingam et al (US 2023/0196806 A1, 2022) discloses text regions of interest with centroids and Euclidean distance. Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY B DUONG whose telephone number is (571)272-1358. The examiner can normally be reached Monday - Thursday 10a-9p (ET). Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /J.B.D./Examiner, Art Unit 2667 /MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667 Application/Control Number: 18/779,808 Page 2 Art Unit: 2667 Application/Control Number: 18/779,808 Page 3 Art Unit: 2667 Application/Control Number: 18/779,808 Page 4 Art Unit: 2667 Application/Control Number: 18/779,808 Page 5 Art Unit: 2667 Application/Control Number: 18/779,808 Page 6 Art Unit: 2667 Application/Control Number: 18/779,808 Page 7 Art Unit: 2667 Application/Control Number: 18/779,808 Page 8 Art Unit: 2667 Application/Control Number: 18/779,808 Page 9 Art Unit: 2667
Prosecution Insights