DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
The amendment filed on 07/28/2026 has been entered.
Claims 1, 11, 19 were amended.
Claims 2 and 12 were cancelled.
Claims 1, 3-11, 13-20 were pending.
Response to Arguments
Applicant’s arguments have been considered but are moot in view of the new ground(s) of rejection in view of Patel (US 2021/0201018 A1, 2020) in regards to the amended limitations and re-consideration of the Yang reference (in regards to weighted Euclidean distance using the Multiplicative Identity, see informational reference: https://byjus.com/maths/additive-identity-vs-multiplicative-identity/).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-6, 11, 14, 19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yang (“Improve newspaper digitalization efficacy with generic document segmentation tool using Amazon Textract”, 2021), in view of Patel (US 2021/0201018 A1, 2020).
Regarding claims 1, 11, 19, Yang teaches A spatial alignment computer system (Yang, pg 6, ¶3-6, reproduced below:
PNG
media_image1.png
456
1144
media_image1.png
Greyscale
. “align the boundaries” shows “spatial alignment”) for string alignment (Yang, see nearest image above, “align the boundaries” shows string alignment as the bounding boxes contain strings) within a document processed using an optical character recognition (OCR) tool (Yang, pg 1, ¶2, reproduced below:
PNG
media_image2.png
174
1150
media_image2.png
Greyscale
. “newspapers articles” is being interpreted as “document” that is processed. Amazon Textract is being interpreted as OCR tool), the spatial alignment computer system comprising:
at least one memory (Yang, see pg 1, ¶2 image above, which shows the newspaper digitization method requires using memory since machine learning and segmentation are involved); and
at least one processor in communication with the at least one memory (Yang, see pg 1, ¶2 image above, which shows the newspaper digitization method requires using at least one processor since machine learning and segmentation are involved), wherein the at least one processor is programmed to:
receive a plurality of bounding boxes (Yang, see pg 6 image above, “bounding boxes”) of a document scanned (Yang, see pg 1 image above, “newspaper” is an example scanned document) using an OCR tool (Yang, see pg 1, ¶2 image above, “Amazon Textract” is an OCR tool);
identify a centroid of each bounding box of the plurality of bounding boxes (Yang, see pg 6 image above, “centroid of each bounding box”);
sort the weighted Euclidean distance of the centroid of each bounding box (Yang, see pg 6 image above, “Euclidean distances of the centroid of each bounding box”. The weights being 1, as PHOSITA would know from the identity rule for multiplication) in ascending order to obtain a sorting index (Yang, see pg 6 image above, “we sorted the bounding boxes by the coordinates, according to a human read order from top to bottom, and left to right”. Sorting is being interpreted to involve a sorting index. Human read order is being interpreted to involve an ascending order); and
based upon the sorting index (Yang, see pg 6 image above, “we sorted the bounding boxes”, which is being interpreted to involve an index), align the one or more output strings associated with each bounding box of the plurality of bounding boxes (Yang, see pg 6 image above, “we sorted the bounding boxes by the coordinates, according to a human read order from top to bottom, and left to right”. The purpose of the OCR tool is to extract text, or strings. These are sorted according to human read order).
a weighted Euclidean (Yang, see pg 6 image above, “Euclidean distances of the centroid of each bounding box”. Which is being interpreted as having weights of 1, as PHOSITA would know from the identity rule for multiplication in the field of mathematics.)
However, Yang does not appear to explicitly teach “calculate coordinates for each centroid of each bounding box of the plurality of bounding boxes from a reference point defined at a top-left corner of a document page in which one or more output strings are displayed”.
Pertaining to the same field of endeavor, Patel teaches
calculate coordinates for each centroid of each bounding box (Patel, [0086]: “Centroid point of the bounding box containing label”) of the plurality of bounding boxes (Patel, [0064]: “At step 3, after going through all bounding boxes and obtaining confidence score for all bounding boxes”. “All bounding boxes” is being interpreted as involving “plurality of bounding boxes”) from a reference point defined at a top-left corner of a document page (Patel, [0086]: “Centroid angle in degrees with reference to left, top location of the image containing value”. “image” is being interpreted as involving “document page” as Patel is in the field of endeavor of text detection, which involves document page) in which one or more output strings are displayed (Patel, [0086]: “Type of text contained-numeric, alphanumeric, date string format, address string format, Angle in degrees of line joining left, top point of the document and Centroid point of the bounding box containing label”);
calculate, for each centroid, … distance of the centroid from the reference point (Patel, Fig. 4, reproduced below:
PNG
media_image3.png
756
1320
media_image3.png
Greyscale
. The big dots at the center of each bounding box are being interpreted as “centroid”. The “Label Box” is being interpreted as the top-left reference point. “Proximity distance” is being interpreted as “distance of the centroid from the reference point”);
Yang and Patel are considered to be analogous art because they are directed to document layout analysis and text detection. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for document layout analysis with text detection (as taught by Yang) to calculate coordinates for each centroid of each bounding box of the plurality of bounding boxes from a reference point defined at a top-left corner of a document page in which one or more output strings are displayed (as taught by Patel) because the combination provides an improvement to dynamic text location extraction (Patel, [0003-0004]).
Regarding claim 4, Yang teaches The spatial alignment computer system of claim 1, wherein a vertical spacing between two rows of a document (Yang, pg 9, the sample newspaper image shows vertical spacing between rows of the document) displaying the one or more aligned output strings (Yang, pg 9, sample newspaper image, which shows the text displayed) is determined based at least in part on a paper size of the document (Yang, pg 9, sample newspaper image, which shows the text alignment is based on at least the paper size of the document, otherwise text would go outside the margins).
Regarding claim 5, Yang teaches The spatial alignment computer system of claim 4, wherein the paper size of the document includes at least one of: (i) a letter size; (ii) a legal size; (iii) a tabloid size (Yang, pg 9, sample newspaper image, which is a tabloid size. Examiner notes the phrasing of this claim is being interpreted as an “OR” statement, so only one item will be addressed); (iv) a ledger size; (v) a junior legal size; (vi) a half letter size; (vii) a government letter; or (viii) a government legal size.
Regarding claim 6, Yang teaches The spatial alignment computer system of claim 4, wherein the vertical spacing between two rows of the document displaying the one or more aligned output strings is further determined based at least in part on a font type or a font size (Yang, pg 9, sample newspaper image, which shows small and large font size influences the vertical spacing between two rows of the document for the text).
Claim 14 is rejected using the same rationale and motivation as applied to claim 4 discussed above.
Claim(s) 3, 7, 13, 15, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang and Patel, in view of Aboulkacim (“Form data enriching using a post OCR clustering process”, 2022).
Regarding claim 3, Yang teaches The spatial alignment computer system of claim 1, wherein a vertical coordinate component of the calculated coordinates for each centroid (Aboulkacim, pg 31, bulletpoint before Section 7.2: “As the Euclidean distance was calculated by using midpoints of each text box”, which is being interpreted to involve a centroid) of each bounding box of the plurality of bounding boxes is more heavily weighted than a horizontal coordinate component (Aboulkacim, pg 7, Section 2.5.2, reproduced below:
PNG
media_image4.png
422
794
media_image4.png
Greyscale
. Which shows the vertical coordinate, interpreted as involving the y variable, is more heavily weighted than the horizontal coordinate, interpreted as involving the x variable.).
Yang, Patel, and Aboulkacim are considered to be analogous art because they are directed to OCR and layout analysis. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for OCR and layout analysis with top-left reference point (as taught by Yang and Patel) to include a more heavily weighted vertical coordinate (as taught by Aboulkacim) because the combination provides an improvement to data enrichment for OCR tools (Aboulkacim, Abstract).
Regarding claim 7, Yang teaches The spatial alignment computer system of claim 1, wherein a weight ω_2 of a vertical coordinate component (Aboulkacim, see nearest image below, the 2 weight in the equation is being interpreted as the second weight) of the calculated coordinates of the centroid of each bounding box of the plurality of bounding boxes is at least
PNG
media_image5.png
124
134
media_image5.png
Greyscale
times a weight ω_1 of a horizontal coordinate component (Aboulkacim, see nearest image below, 0.5 would be the smaller weight for the horizontal coordinate. The 2 is being interpreted as at least larger than 0.5) of the calculated coordinates of the centroid of each bounding box of the plurality of bounding boxes (Aboulkacim, pg 7, Section 2.5.2, reproduced below:
PNG
media_image4.png
422
794
media_image4.png
Greyscale
. Which shows the vertical coordinate, interpreted as involving the y variable, is more heavily weighted than the horizontal coordinate, interpreted as involving the x variable. The midpoint calculated [pg 31, bulletpoint before section 7.2] is being interpreted as the centroid for each text box, or bounding box),
wherein h corresponds with a length of a document page (Aboulkacim, pg 21, Section 5, ¶1: “The system has been tested by inputting different images of handwritten filled in forms with the same image of a blank form.” “image of a blank form” is being interpreted to have a height, or a length, or a document page), r corresponds with an aspect ratio of
PNG
media_image6.png
94
54
media_image6.png
Greyscale
(Aboulkacim, pg 21, Section 5, ¶1: “The system has been tested by inputting different images of handwritten filled in forms with the same image of a blank form.” “image of a blank form” is being interpreted to have an aspect ratio of 1/sqrt(2), the common aspect ratio for paper sizes), and Δh corresponds with a height of a single line or row (Aboulkacim, pg 20, Figure 18, the bounding boxes are interpreted to have a height and corresponds to a height of a single line or row).
Yang, Patel, and Aboulkacim are considered to be analogous art because they are directed to OCR and layout analysis. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for OCR and layout analysis with top-left reference point (as taught by Yang and Patel) to include a more heavily weighted vertical coordinate (as taught by Aboulkacim) because the combination provides an improvement to data enrichment for OCR tools (Aboulkacim, Abstract).
Claim 13 is rejected using the same rationale and motivation as applied to claim 3 discussed above.
Claims 15 and 20 is rejected using the same rationale and motivation as applied to claim 7 discussed above.
Claim(s) 8, 9, 10, 16, 17, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang and Patel, in view of Liang (“Comparative study of table layout analysis”, 2018).
Regarding claim 8, Yang teaches The spatial alignment computer system of claim 1, wherein the at least one processor is further programmed to:
based upon the sorting index for a first version of the OCR tool (Yang, see pg 6 image above, “we sorted the bounding boxes”, which is being interpreted to involve an index), align a first set of one or more output strings associated with each bounding box of a plurality of bounding boxes generated by the first version of the OCR tool (Yang, see pg 6 image above, “we sorted the bounding boxes by the coordinates, according to a human read order from top to bottom, and left to right”. The purpose of the OCR tool is to extract text, or strings. These are sorted according to human read order);
However, Yang and Patel does not appear to explicitly teach second version of the OCR tool.
Pertaining to the same field of endeavor, Liang teaches
based upon a sorting index for a second version of the OCR tool, align a second set of one or more output strings associated with each bounding box of a plurality of bounding boxes (Liang, see Section 1.2 image below: “segmenting physical contents” is being interpreted as involving bounding boxes) generated by the second version of the OCR tool (Liang, see Section 3.3.4.3 image below, the OCR tool comparison is show using performance metrics in Table 6, which is being interpreted as having first and second version of the OCR tool);
compare the spatial alignment of the first set of the one or more output strings to the second set of the one or more output strings (Liang, pg 10, last 3 paragraphs before Section 1.2, reproduced below:
PNG
media_image7.png
532
1204
media_image7.png
Greyscale
. Layout analysis is being interpreted to involve “spatial alignment”. “Connected components” are being interpreted to involve output strings. See Section 3.3.4.3 image below, the metric compares the results of the OCR tools); and
output a performance metric indicating how the first version of the OCR tool compares to the second version of the OCR tool (Liang, pg 33, Section 3.3.4.3, reproduced below:
PNG
media_image8.png
630
884
media_image8.png
Greyscale
. Table 6, each column shows a different OCR tool, which is interpreted as having a first and second version of the OCR tool. F1_micro_avg, is an example performance metric that is used to compare the OCR tools).
Yang, Patel, and Liang are considered to be analogous art because they are directed to OCR and layout analysis. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for OCR and layout analysis with top-left reference point (as taught by Yang and Patel) to compare OCR tools (as taught by Liang) because the combination provides an improvement to OCR and layout analysis (Liang, Abstract). Further, one with ordinary skill in the art would know that it is common practice to compare tools in the scientific and research disciplines.
Regarding claim 9, Yang teaches The spatial alignment computer system of claim 8, wherein the performance metric indicates whether the spatial alignments of the different versions of the OCR tools are compatible or non-compatible (Liang, see Section 3.3.4.3 image above, “Since these two alternatives perform well consistently in all these experiments, we adopted these two alternatives as two branches of solution III and applied them to produce evaluation result at the end”. Which shows the two OCR tools are compatible, while the third one was not).
Regarding claim 10, Yang teaches The spatial alignment computer system of claim 1, wherein the at least one processor is further programmed to:
However, Yang and Patel does not appear to explicitly teach second version of the OCR tool.
Pertaining to the same field of endeavor, Liang teaches
compare the alignment of the one or more output strings from a first version of the OCR tool to an alignment of one or more output strings from a second version of an OCR tool (Liang, pg 10, last 3 paragraphs before Section 1.2, reproduced below:
PNG
media_image7.png
532
1204
media_image7.png
Greyscale
. Layout analysis is being interpreted to involve “spatial alignment”. “Connected components” are being interpreted to involve output strings. See Section 3.3.4.3 image above, the metric compares the results of the OCR tools); and
determine from the comparison whether the first version and the second version of the OCR tools are compatible (Liang, see Section 3.3.4.3 image above, “Since these two alternatives perform well consistently in all these experiments, we adopted these two alternatives as two branches of solution III and applied them to produce evaluation result at the end”. Which shows the two OCR tools are compatible, while the third one was not).
Yang, Patel, and Liang are considered to be analogous art because they are directed to OCR and layout analysis. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for OCR and layout analysis with top-left reference point (as taught by Yang and Patel) to compare OCR tools (as taught by Liang) because the combination provides an improvement to OCR and layout analysis (Liang, Abstract). Further, one with ordinary skill in the art would know that it is common practice to compare tools in the scientific and research disciplines.
Claim 16 is rejected using the same rationale and motivation as applied to claim 8 discussed above.
Claim 17 is rejected using the same rationale and motivation as applied to claim 9 discussed above.
Claim 18 is rejected using the same rationale and motivation as applied to claim 10 discussed above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Xu et al (“Graph-based layout analysis for PDF documents”, 2013) discloses document layout analysis with weighted Euclidean distance (interpreted from weight of edges that uses Euclidean distance).
Vincent et al (US 7697758 B2, 2006) discloses OCR post-processing and centroids of bounding boxes along with distances between bounding boxes.
Ramalingam et al (US 2023/0196806 A1, 2022) discloses text regions of interest with centroids and Euclidean distance.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY B DUONG whose telephone number is (571)272-1358. The examiner can normally be reached Monday - Thursday 10a-9p (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.B.D./Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights