DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
1. The information disclosure statement (IDS) submitted on 1/22/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Response to Amendment
2. This action is in response to the amendment filed on January 22nd, 2026. Claims 1-2, 4-6, 8-9, 11-13, 15, and 17-18 have been amended. Claim 16 has been cancelled. Claim 21 has been added. Claims 1-15 and 17-21 are pending. Applicant’s amendments to the claims have overcome each and every objection previously set forth in the non-final office action mailed October 23rd, 2025.
Response to Arguments
3. Applicant’s arguments with respect to independent claims 1, and similarly claims 8 and 15, with respect to the rejection under 35 U.S.C. 102 regarding that the prior art does not teach the limitation(s): “a grid generated using the plurality of parallel curves and the plurality of vertical lines” has been considered but is moot because of the new ground of rejection. In addition, applicant’s argument with respect to claim 2, and similarly claim 9, that the prior art does not teach the limitation(s): “identifying a set of representative points within a probable document boundary region of the warped image, wherein the set of representative points are points on the grid” has also been considered but is moot because of the new ground of rejection. Claims 1, 8, and 15 are now disclosed by Cheng, Fero, and Wei. Claims 2 and 9 are now disclosed by Cheng, Fero, Wei, and Meng.
4. Regarding arguments to claims 3-7, 10-14 and 17-20, they are dependent on independent claims 1, 8 and 15 respectively. Applicant does not argue anything other than independent claim 1, and similarly claims 8 and 15, and dependent claims 2 and 9. The limitations in those claims, in conjunction with their combination, has previously been established and explained.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1, 8, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (CN-116453131-B, hereinafter "Cheng") in view of Fero et al. (US20100014782A1, hereinafter "Fero"), and further in view of Wei (CN-105450900-A).
7. As per claim 1, Cheng discloses: A method comprising:
receiving a warped image depicting a page of a document having an incomplete document boundary, the page including a plurality of curved text lines; (Cheng, page 6, [0006], “In a first aspect, an embodiment of the present application proposes a document image correction method, which is applied to an electronic device, and the method includes: obtaining a distorted document image; ... and the text features including at least one of a document edge, a text line, and an illustration edge line; ...” and page 6, [0007], “In this way, the embodiment of the present application can supplement the edges of a distorted document image with incomplete edges, ...” and page 21, [0152], “For the task of correcting distorted document images, the goal is to make the same line of curved text appear horizontal or vertical after correction. ... The embodiments of this application can use a lightweight cascaded dilated convolutional network as a feature extraction module to extract semantic features such as document edges, text lines and illustration edges with a larger field of view, thereby obtaining information about the entire curved text line.”)
[[applying an offset to the plurality of curved text lines to obtain a plurality of parallel curves;]]
[[generating a plurality of vertical lines across a width of the page;]]
identifying a complete document boundary based on [[a grid generated using the plurality of parallel curves and the plurality of vertical lines;]] (Cheng, page 6, [0007], “In this way, the embodiment of the present application can supplement the edges of a distorted document image with incomplete edges, and sample the first document image after edge supplementation to achieve document correction. Therefore, the embodiment of the present application can improve the loss of text information at the edges of the corrected document image, thereby improving the quality of the corrected document image.” and page 6, [0006], “In a first aspect, an embodiment of the present application proposes a document image correction method, which is applied to an electronic device, and the method includes: obtaining a distorted document image; ... the text features including at least one of a document edge, a text line, and an illustration edge line; ...”)
determining a dewarped image corresponding to the warped image using the complete document boundary; and (Cheng, page 6, [0007], “In this way, the embodiment of the present application can supplement the edges of a distorted document image with incomplete edges, and sample the first document image after edge supplementation to achieve document correction. Therefore, the embodiment of the present application can improve the loss of text information at the edges of the corrected document image, thereby improving the quality of the corrected document image.”)
providing, for display on a client device, the dewarped image. (Cheng, page 17, [0116], “The fifth interface 305 displays the corrected document image, which will be referred to as the second document image below.”)
8. Cheng doesn't explicitly disclose but Fero discloses: applying an offset to the plurality of curved text lines to obtain a plurality of parallel curves; (See Fero [0022] below.)
[[identifying a complete document boundary based on a grid generated using the]] plurality of parallel curves [[and the plurality of vertical lines;]] (Fero, [0022], “Once the text direction has been firmly established, a three-dimensional de-skewing transformation matrix can be created that transforms the calculated horizontal vanishing point into infinity in the horizontal direction and the calculated vertical vanishing point into infinity in the vertical direction, without moving the center of the image. This transformation can transform the camera image so that the text lines on the transformed image are horizontal and parallel and the edges of the columns are vertical and parallel, but the curved text lines are not straightened.” and [0028], “After the de-skewing transformational matrix has been created, step 307, a page unwarping inverse transformation can be calculated, step 308. For a specific destination pixel (x, y) of the expected unwarped image, a source pixel (x′, y′) of the original image is determined.”; Examiner’s note: As disclosed by Fero, a transformation of text can be made so that text lines in the transformed image can be horizontal and parallel while still maintaining the curved nature of the lines (“the curved text lines are not straightened.”). Thus, this will create parallel curves. Also, the transformation process disclosed by Fero is one that produces an offset for the text, specifically translating a (x’, y’) source pixel to a (x, y) destination pixel.)
9. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Cheng to include the disclosure of applying an offset to the plurality of curved text lines to obtain a plurality of parallel curves, of Fero. The motivation for this modification could have been to provide curves that have consistent distances between them. This would help with a document boundary detection or rectification process to properly detect the warped nature of the document.
10. Cheng in view of Fero doesn't explicitly disclose but Wei discloses: generating a plurality of vertical lines across a width of the page; (See Wei, page 11, [0118] and page 4, [0001] below.)
[[identifying a complete document boundary based on]] a grid generated using the [[plurality of parallel curves]] and the plurality of vertical lines; (Wei, page 11, [0118], “In a typical implementation, the grid lines in the grid can consist of horizontal grid lines and vertical grid lines, where the horizontal grid lines will be aligned with the text lines and substantially the same as the baseline extracted for the text lines, and the vertical grid lines are grid lines that are substantially perpendicular to the horizontal grid lines, so that the grid grid will be quadrilateral.” and page 4, [0001], “More specifically, the present invention relates to a method and apparatus for correcting distortion of a document image by at least extending the grid of the text regions contained in the document image to obtain the grid of the entire document image.”)
11. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Cheng in view of Fero to include the disclosure of generating a plurality of vertical lines across a width of the page and using a grid generated using the plurality of parallel curves and the plurality of vertical lines to help identify a document boundary, of Wei. The motivation for this modification could have been to help provide a consistent grid across a document for boundary detection or for a rectification process. The grid presents the warped nature of the document so that a method or algorithm can determine how to properly correct for the distortion.
12. Claim 8 is similar in scope to claim 1 except for additional limitations that Cheng in view of Fero, and further in view of Wei discloses: A system comprising:
a memory component; and
a processing device coupled to the memory component, the processing device to perform operations comprising: (Cheng, page 9, [0022], “In a second aspect, an embodiment of the present application proposes an electronic device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call the computer program to execute the above-mentioned document image correction method.”)
13. As per claim 15, Cheng in view of Fero, and further in view of Wei discloses: A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising: (Cheng, page 34, [0306], “An embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores computer programs or instructions. When the computer program or instructions are executed by a processor, the above method is implemented.”)
receiving, via a graphical user interface (GUI), a user selection to dewarp a warped image of a document having incomplete boundary, wherein the document comprises a plurality of curved text lines; (Cheng, page 6, [0006], “In a first aspect, an embodiment of the present application proposes a document image correction method, which is applied to an electronic device, and the method includes: obtaining a distorted document image; ... and the text features including at least one of a document edge, a text line, and an illustration edge line; ...” and Cheng, page 6, [0007], “In this way, the embodiment of the present application can supplement the edges of a distorted document image with incomplete edges, ...” and Cheng, page 21, [0152], “For the task of correcting distorted document images, the goal is to make the same line of curved text appear horizontal or vertical after correction. ... The embodiments of this application can use a lightweight cascaded dilated convolutional network as a feature extraction module to extract semantic features such as document edges, text lines and illustration edges with a larger field of view, thereby obtaining information about the entire curved text line.”)
applying an offset to the plurality of curved text lines to obtain a plurality of parallel curves; (Fero, [0022], “Once the text direction has been firmly established, a three-dimensional de-skewing transformation matrix can be created that transforms the calculated horizontal vanishing point into infinity in the horizontal direction and the calculated vertical vanishing point into infinity in the vertical direction, without moving the center of the image. This transformation can transform the camera image so that the text lines on the transformed image are horizontal and parallel and the edges of the columns are vertical and parallel, but the curved text lines are not straightened.” and Fero, [0028], “After the de-skewing transformational matrix has been created, step 307, a page unwarping inverse transformation can be calculated, step 308. For a specific destination pixel (x, y) of the expected unwarped image, a source pixel (x′, y′) of the original image is determined.”; Examiner’s note: As disclosed by Fero, a transformation of text can be made so that text lines in the transformed image can be horizontal and parallel while still maintaining the curved nature of the lines (“the curved text lines are not straightened.”). Thus, this will create parallel curves. Also, the transformation process disclosed by Fero is one that produces an offset for the text, specifically translating a (x’, y’) source pixel to a (x, y) destination pixel.)
generating a plurality of vertical lines across a width of the document; (Wei, page 11, [0118], “In a typical implementation, the grid lines in the grid can consist of horizontal grid lines and vertical grid lines, where the horizontal grid lines will be aligned with the text lines and substantially the same as the baseline extracted for the text lines, and the vertical grid lines are grid lines that are substantially perpendicular to the horizontal grid lines, so that the grid grid will be quadrilateral.” and Wei, page 4, [0001], “More specifically, the present invention relates to a method and apparatus for correcting distortion of a document image by at least extending the grid of the text regions contained in the document image to obtain the grid of the entire document image.”)
determining a complete document boundary of the warped image of the document based on a grid generated using the plurality of parallel curves and the plurality of vertical lines; (Cheng, page 6, [0007], “In this way, the embodiment of the present application can supplement the edges of a distorted document image with incomplete edges, and sample the first document image after edge supplementation to achieve document correction. Therefore, the embodiment of the present application can improve the loss of text information at the edges of the corrected document image, thereby improving the quality of the corrected document image.” and Cheng, page 6, [0006], “In a first aspect, an embodiment of the present application proposes a document image correction method, which is applied to an electronic device, and the method includes: obtaining a distorted document image; ... the text features including at least one of a document edge, a text line, and an illustration edge line; ...” and Wei, page 11, [0118], “In a typical implementation, the grid lines in the grid can consist of horizontal grid lines and vertical grid lines, where the horizontal grid lines will be aligned with the text lines and substantially the same as the baseline extracted for the text lines, and the vertical grid lines are grid lines that are substantially perpendicular to the horizontal grid lines, so that the grid grid will be quadrilateral.” and Wei, page 4, [0001], “More specifically, the present invention relates to a method and apparatus for correcting distortion of a document image by at least extending the grid of the text regions contained in the document image to obtain the grid of the entire document image.” and Fero, [0022], “Once the text direction has been firmly established, a three-dimensional de-skewing transformation matrix can be created that transforms the calculated horizontal vanishing point into infinity in the horizontal direction and the calculated vertical vanishing point into infinity in the vertical direction, without moving the center of the image. This transformation can transform the camera image so that the text lines on the transformed image are horizontal and parallel and the edges of the columns are vertical and parallel, but the curved text lines are not straightened.” and Fero, [0028], “After the de-skewing transformational matrix has been created, step 307, a page unwarping inverse transformation can be calculated, step 308. For a specific destination pixel (x, y) of the expected unwarped image, a source pixel (x′, y′) of the original image is determined.”; Examiner’s note: As disclosed by Fero, a transformation of text can be made so that text lines in the transformed image can be horizontal and parallel while still maintaining the curved nature of the lines (“the curved text lines are not straightened.”). Thus, this will create parallel curves. Also, the transformation process disclosed by Fero is one that produces an offset for the text, specifically translating a (x’, y’) source pixel to a (x, y) destination pixel.)
dewarping the warped image of the document based on the complete document boundary; and (Cheng, page 6, [0007], “In this way, the embodiment of the present application can supplement the edges of a distorted document image with incomplete edges, and sample the first document image after edge supplementation to achieve document correction. Therefore, the embodiment of the present application can improve the loss of text information at the edges of the corrected document image, thereby improving the quality of the corrected document image.”)
providing, for display on the GUI, the dewarped image. (Cheng, page 17, [0116], “The fifth interface 305 displays the corrected document image, which will be referred to as the second document image below.”)
14. Claims 2 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (CN-116453131-B, hereinafter "Cheng") in view of Fero et al. (US20100014782A1, hereinafter "Fero"), further in view of Wei (CN-105450900-A), and further in view of Meng et al. (CN-102208025-A, hereinafter "Meng").
15. As per claim 2, Cheng in view of Fero, and further in view of Wei discloses: The method of claim 1, wherein identifying a complete document boundary based on the incomplete document boundary and the plurality of curved text lines, further comprises: (See Cheng, page 6, [0007] and page 6, [0006] below.)
identifying a set of representative points within a probable document boundary region of the warped image, wherein the set of representative points are points on the grid, [[and wherein the plurality of vertical lines are a plurality of equidistant vertical lines;]] and (See Cheng, page 6, [0007] and page 6, [0006] below. Wei, Fig. 10, 15F; page 6, [0028]-[0029], “Preferably, the image distortion information determination step may include an extension intersection point determination step for determining the extension intersection point of the grid line and the corresponding boundary; a grid line extension step for extending the grid line to the extension intersection point; a node positioning step for locating nodes in the extension lines from the grid line to the extension intersection point; and a node connection step for connecting the corresponding nodes in each extension line to form a grid for the corresponding non-text area. Here, the grid of the entire document image can be formed based on the grid of the text area and the grid of the non-text area. Preferably, the step of determining the extension intersection point may include determining the reference extension intersection point of the reference grid line and the corresponding boundary; for each of the remaining grid lines other than the reference grid line, starting from one of the reference grid lines, the extension intersection point of the grid line and the boundary is calculated sequentially based on the distance between the endpoint of the grid line and the corresponding endpoint of the previous horizontal grid line.”)
determining the complete document boundary using the set of representative points. (Cheng, page 6, [0007], “In this way, the embodiment of the present application can supplement the edges of a distorted document image with incomplete edges, and sample the first document image after edge supplementation to achieve document correction. Therefore, the embodiment of the present application can improve the loss of text information at the edges of the corrected document image, thereby improving the quality of the corrected document image.” and page 6, [0006], “In a first aspect, an embodiment of the present application proposes ... processing the distorted document image using a document correction model to obtain the coordinate positions of multiple first control points corresponding to the distorted document image, the first control points including the points where text features in the distorted document image are located, and the text features including at least one of a document edge, a text line, and an illustration edge line; ...”
16. Cheng in view of Fero, and further in view of Wei doesn't explicitly disclose but Meng discloses: [[identifying a set of representative points within a probable document boundary region of the warped image, wherein the set of representative points are points on the grid,]] and wherein the plurality of vertical lines are a plurality of equidistant vertical lines; (Meng, Abstract, “The invention provides a method for correcting geometric distortion of a text image. The method comprises the following steps of: extracting a text row, in a horizontal direction, of the text image, fitting the text row by using a B-spline curve to obtain a text row curve; estimating a vanishing point of a cylindrical surface straight bus, a text row tangent convergence line and a focal length parameter of a camera by using the tangent convergence symmetry of the text row curve; constructing latitude lines of an isometric grid for the fit text row curve by adopting a crossratio interpolation value; estimating a directrix of the cylindrical surface by using similar perspective projection transformation of the text row curve and constructing longitude lines of the isometric grid; estimating a length-to-width ratio of the isometric grid by using the text row curves at the top end and the bottom end of the image; and mapping meshes of each isometric grid into a square grid through a deformation technology based on the grid to finish the correction of the geometric distortion of the text image. By the method, the problem that perspective distortion, nonlinear deformation distortion and compound distortion of the perspective distortion and the nonlinear deformation distortion cannot be corrected at the same time in the prior art can be solved; therefore, total correction of the geometric distortion of the text image shot by the camera can be realized.” And page 5, [0016], “Step S4: Using the perspective projection transformation of the text line curve, estimate the directrix of the cylinder, and thus construct the meridians of the equidistant grid;”; Examiner’s note: The term “isometric” in plain English describes things that have constant dimensions or lengths. In this case, the isometric grid, disclosed by Meng, would qualify as a grid that has equidistant vertical lines. Also, the term “meridian” in plain English, are “vertical lines,” often used for describing lines of longitude on the Earth’s surface that run from the North to South Pole.)
17. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Cheng in view of Fero, and further in view of Wei to include the disclosure of a set of representative points are points on the grid and the plurality of vertical lines are a plurality of equidistant vertical lines, of Meng. The motivation for this modification could have been to provide finer granularity in determining the warped nature of the document. By have points on the grid, those points could be slightly adjusted to help produce better document image rectification.
18. Claim 9, which is similar in scope to claims 2 and 8, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 2.
19. Claims 3 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (CN-116453131-B, hereinafter "Cheng") in view of Fero et al. (US20100014782A1, hereinafter "Fero"), further in view of Wei (CN-105450900-A), further in view of Meng et al. (CN-102208025-A, hereinafter "Meng"), and further in view of Hunt et al. (US-2010/0073735-A1, hereinafter "Hunt").
20. As per claim 3, Cheng in view of Fero, further in view of Wei, and further in view of Meng discloses: The method of claim 2, wherein determining the complete document boundary using the set of representative points, further comprises: (See rejection for claim 2. Cheng, page 6, [0006], “In a first aspect, an embodiment of the present application proposes ... processing the distorted document image using a document correction model to obtain the coordinate positions of multiple first control points corresponding to the distorted document image, the first control points including the points where text features in the distorted document image are located, and the text features including at least one of a document edge, a text line, and an illustration edge line; ...”)
[[fitting at least one parametric curve to the set of representative points.]]
21. Cheng in view of Fero, further in view of Wei, and further in view of Meng doesn't explicitly disclose but Hunt discloses: fitting at least one parametric curve to the set of representative points. (Hunt, Claim 1, “1. A method for processing a photographed image containing text lines comprising text characters having vertical strokes comprising: ... (b) detecting typographical features indicative of the orientation of text; (c) fitting one or more curves to a text line; (d) building a grid of quadrilaterals using vectors that are parallel to the direction of the text lines and vectors parallel to the direction of the vertical stroke lines; ...” and [0090], “In the splines fitting step 122, splines are fitted to the top and bottom of text lines.” and [0146], “In the present embodiment, optimization-based dewarping step 134 is performed in two steps. First, a number of subsets of pixels in the input image are considered and where these pixels should be mapped into the output image are determined. These pixels are called control points.”)
22. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 2 of Cheng in view of Fero, further in view of Wei, and further in view of Meng to include the disclosure of fitting at least one parametric curve to the set of representative points of a document of Hunt. The motivation for this modification could have been to ensure that the warp or distortion of a document (including at the document’s boundaries) is precisely accounted for by fitting a curve to the representative points. It also can assist in the creation of a grid or mesh for a document.
23. Claim 10, which is similar in scope to claims 3 and 8, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 3.
24. Claims 4-7, 11-14, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (CN-116453131-B, hereinafter "Cheng") in view of Fero et al. (US20100014782A1, hereinafter "Fero"), further in view of Wei (CN-105450900-A), further in view of Meng et al. (CN-102208025-A, hereinafter "Meng"), further in view of Hunt et al. (US-2010/0073735-A1, hereinafter "Hunt"), and further in view of Campbell (US-2017/0076169-A1).
25. As per claim 4, Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt discloses: The method of claim 2, wherein identifying, a set of representative points within a probable document boundary region of the warped image further comprises:
determining a probable document boundary region associated with the … (Cheng, page 6, [0007], “In this way, the embodiment of the present application can supplement the edges of a distorted document image with incomplete edges, and sample the first document image after edge supplementation to achieve document correction. Therefore, the embodiment of the present application can improve the loss of text information at the edges of the corrected document image, thereby improving the quality of the corrected document image.” and page 6, [0006], “In a first aspect, an embodiment of the present application proposes ... processing the distorted document image using a document correction model to obtain the coordinate positions of multiple first control points corresponding to the distorted document image, the first control points including the points where text features in the distorted document image are located, and the text features including at least one of a document edge, a text line, and an illustration edge line; ...”) … plurality of parallel curves and an equidistant vertical line of the plurality of equidistant vertical lines;
[[determining a plurality of intersections between the]] plurality of parallel curves and the plurality of equidistant vertical lines within the probable document boundary region; and (Hunt, Claim 1, “1. A method for processing a photographed image containing text lines comprising text characters having vertical strokes comprising: … (c) fitting one or more curves to a text line; (d) building a grid of quadrilaterals using vectors that are parallel to the direction of the text lines and vectors parallel to the direction of the vertical stroke lines; …” and [0075], “To identify regions corresponding to paragraphs, it may be assumed that in a paragraph, a text line is parallel to text lines immediately above or below, and these lines have roughly the same shape and size. Additionally, it may be assumed that the vertical distance between text lines is constant.”)
26. Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt doesn't explicitly disclose but Campbell discloses: determining a plurality of intersections between the [[plurality of parallel curves and the plurality of equidistant vertical lines within the probable document boundary region; and]] (Campbell, FIGS. 6, 17A, [0072], “FIG. 16 illustrates an exemplary workflow process to identify or generate the page curves that span between the page boundary lines. The next step is to identify the top and bottom page curves that span between the boundary lines for each page.” and [0078], “The lines L1, L2, LC1 (FIG. 17A) define a region where the application trims and labels the contours within this region as candidates for the top page curve. ... The contours that intersect the region are trimmed to include only the portions of the contour that lie within the region. Because the segments representing the contour can vary in length, the trimming algorithm also detects when a segment intersects the region boundaries and computes the boundary intersection points to define a contour that includes the boundaries. This is repeated for the bottom curve region.” and [0079], “Likewise, if the curve only intersects L1 but not L2 then it is given leftlabel (1704) and if the curve only intersects L2 but not L1, then it is given rightlabel (1702). These labels define subsets in the possible page contours to assist in search and generation of the best path.”)
determining a representative point based on the plurality of intersections. (Campbell, [0078], “Because the segments representing the contour can vary in length, the trimming algorithm also detects when a segment intersects the region boundaries and computes the boundary intersection points to define a contour that includes the boundaries. This is repeated for the bottom curve region.” and [0079], “Likewise, if the curve only intersects L1 but not L2 then it is given leftlabel (1704) and if the curve only intersects L2 but not L1, then it is given rightlabel (1702). These labels define subsets in the possible page contours to assist in search and generation of the best path.”)
27. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 2 of Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt to include the disclosure of determining a plurality of intersection points between the plurality of curves and lines of a document of Campbell. The motivation for this modification could have been to have a set of representative points that can be “flattened” to remove warp or distortion. In addition, this can also help assist with finding features of a document, such as its boundaries.
28. As per claim 5, Cheng in view of Fero, further in view of Wei, further in view of Meng, further in view of Hunt, and further in view of Campbell discloses: The method of claim 1, further comprising:
dividing the document into a first half and a second half, wherein a first grid is generated according to a first set of the plurality of text lines associated with the first half and a second grid is generated according to a second set of the plurality of text lines associated with the second half. (Campbell, FIGS. 6, 22A-B, [0112]-[0113] “FIGS. 22A and 22B are a flowchart illustrating a method for the correction of a warped page image. ... The method starts at Step 2200. ... Step 2206 projects the filtered edge map and text-likely regions into a polar coordinate system to determine page lines and warped image page curves. Step 2208 creates an adaptive 2D ruled mesh piecewise planar approximation of a warped page surface.” and [0115], “In another aspect, Step 2207a4 filters curves identified as not being between page boundary lines, and divides remaining curves into a top set defined by curves that are within the region defined by boundary lines L1, L2, and LC1, and a bottom set defined by curves that are within the region defined by the boundary lines L1, L2, and LC2, where L1 is a left boundary, L2 is a right boundary, and LC2 is a bottom boundary. Step 2207a5 selects the top page curve as a top set curve with highest completeness and lowest curvature between the boundary lines L1 and L2. Step 2207a6 selects the bottom page curve as bottom set curve with the highest completeness and lowest curvature between the boundary lines L1 and L2. Step 2207a7 processes an image of adjacent pages, where each page is divided into upper and lower curve sets using boundary lines and gutter, and upper and lower curve lines are favored that join at the gutter.”)
29. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 2 of Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt to include the disclosure of dividing the document into two halves with grids associated with each half of Campbell. The motivation for this modification could have been to help improve accuracy of dewarping a document image. In addition, this could also make it easier for certain document features, such as boundaries, to be identified.
30. As per claim 6, Cheng in view of Fero, further in view of Wei, further in view of Meng, further in view of Hunt, and further in view of Campbell discloses: The method of claim 5, further comprising:
determining, using the first grid, an upper probable boundary region;
identifying a first set of representative points within the upper probable boundary region corresponding to an upper side boundary;
determining, using the second grid, a lower probable boundary region; and
identifying a second set of representative points within the lower probable boundary region corresponding to a lower side boundary. (Campbell, FIGS. 6, 22A-B, [0115]-[0116], “In another aspect, Step 2207a4 filters curves identified as not being between page boundary lines, and divides remaining curves into a top set defined by curves that are within the region defined by boundary lines L1, L2, and LC1, and a bottom set defined by curves that are within the region defined by the boundary lines L1, L2, and LC2, where L1 is a left boundary, L2 is a right boundary, and LC2 is a bottom boundary. Step 2207a5 selects the top page curve as a top set curve with highest completeness and lowest curvature between the boundary lines L1 and L2. Step 2207a6 selects the bottom page curve as bottom set curve with the highest completeness and lowest curvature between the boundary lines L1 and L2. Step 2207a7 processes an image of adjacent pages, where each page is divided into upper and lower curve sets using boundary lines and gutter, and upper and lower curve lines are favored that join at the gutter. In one aspect, subsequent to Step 2207a7, Step 2207b1 performs a multi-scale edge localization refinement for each page curve using the following substeps. Step 2207b2 transforms top and bottom page curves into point sets having a first resolution. Step 2207b3 scales the point sets to a second resolution, lower than the first resolution. Step 2207b4 creates a gradient image of the second resolution point sets. Step 2207b5 identifies zero crossings points in the gradient image as a page edge. Step 2207b6 scales the zero crossing points to the first resolution.”)
31. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 5 of Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt to include the disclosure of determining probable upper and lower boundary regions through use of a first and second grid of Campbell. The motivation for this modification could have been to make it easier for certain document features, such as boundaries, to be identified.
32. As per claim 7, Cheng in view of Fero, further in view of Wei, further in view of Meng, further in view of Hunt, and further in view of Campbell discloses: The method of claim 6, further comprising:
determining the upper side boundary using the first set of representative points; and
determining the lower side boundary using the second set of representative points. (Campbell, FIGS. 6, 22A-B, [0115]-[0116], “Step 2207a5 selects the top page curve as a top set curve with highest completeness and lowest curvature between the boundary lines L1 and L2. Step 2207a6 selects the bottom page curve as bottom set curve with the highest completeness and lowest curvature between the boundary lines L1 and L2. Step 2207a7 processes an image of adjacent pages, where each page is divided into upper and lower curve sets using boundary lines and gutter, and upper and lower curve lines are favored that join at the gutter. In one aspect, subsequent to Step 2207a7, Step 2207b1 performs a multi-scale edge localization refinement for each page curve using the following substeps. Step 2207b2 transforms top and bottom page curves into point sets having a first resolution. Step 2207b3 scales the point sets to a second resolution, lower than the first resolution. Step 2207b4 creates a gradient image of the second resolution point sets. Step 2207b5 identifies zero crossings points in the gradient image as a page edge. Step 2207b6 scales the zero crossing points to the first resolution.”)
33. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 6 of Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt to include the disclosure of determining the upper and lower document boundaries of Campbell. The motivation for this modification could have been to make it easier for certain document features, such as boundaries, to be identified.
34. Claim 11, which is similar in scope to claims 4 and 8, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 4.
35. Claim 12, which is similar in scope to claims 5 and 8, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 5.
36. Claim 13, which is similar in scope to claims 6 and 8, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 6.
37. Claim 14, which is similar in scope to claims 7 and 8, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 7.
38. Claim 17 is similar in scope to claim 4, and 15 except for a different limitation that Cheng in view of Fero, further in view of Wei, further in view of Meng, further in view of Hunt, and further in view of Campbell discloses: The non-transitory computer-readable medium of claim 15, wherein the plurality of vertical lines are a plurality of equidistant vertical lines, storing instructions that further cause the processing device to perform operations comprising: (Meng, Abstract, “The invention provides a method for correcting geometric distortion of a text image. The method comprises the following steps of: extracting a text row, in a horizontal direction, of the text image, fitting the text row by using a B-spline curve to obtain a text row curve; estimating a vanishing point of a cylindrical surface straight bus, a text row tangent convergence line and a focal length parameter of a camera by using the tangent convergence symmetry of the text row curve; constructing latitude lines of an isometric grid for the fit text row curve by adopting a crossratio interpolation value; estimating a directrix of the cylindrical surface by using similar perspective projection transformation of the text row curve and constructing longitude lines of the isometric grid; estimating a length-to-width ratio of the isometric grid by using the text row curves at the top end and the bottom end of the image; and mapping meshes of each isometric grid into a square grid through a deformation technology based on the grid to finish the correction of the geometric distortion of the text image. By the method, the problem that perspective distortion, nonlinear deformation distortion and compound distortion of the perspective distortion and the nonlinear deformation distortion cannot be corrected at the same time in the prior art can be solved; therefore, total correction of the geometric distortion of the text image shot by the camera can be realized.” And page 5, [0016], “Step S4: Using the perspective projection transformation of the text line curve, estimate the directrix of the cylinder, and thus construct the meridians of the equidistant grid;”; Examiner’s note: The term “isometric” in plain English describes things that have constant dimensions or lengths. In this case, the isometric grid, disclosed by Meng, would qualify as a grid that has equidistant vertical lines. Also, the term “meridian” in plain English, are “vertical lines,” often used for describing lines of longitude on the Earth’s surface that run from the North to South Pole.)
determining a representative point associated with the probable document boundary region. (Campbell, [0078], “Because the segments representing the contour can vary in length, the trimming algorithm also detects when a segment intersects the region boundaries and computes the boundary intersection points to define a contour that includes the boundaries. This is repeated for the bottom curve region.” and [0079], “Likewise, if the curve only intersects L1 but not L2 then it is given leftlabel (1704) and if the curve only intersects L2 but not L1, then it is given rightlabel (1702). These labels define subsets in the possible page contours to assist in search and generation of the best path.”)
The motivation for this modification is the same as claim 4.
39. As per claim 18, Cheng in view of Fero, further in view of Wei, further in view of Meng, further in view of Hunt, and further in view of Campbell discloses: The non-transitory computer-readable medium of claim 15, wherein the plurality of vertical lines are a plurality of equidistant vertical lines, storing instructions that further cause the processing device to perform operations comprising: (Meng, Abstract, “The invention provides a method for correcting geometric distortion of a text image. The method comprises the following steps of: extracting a text row, in a horizontal direction, of the text image, fitting the text row by using a B-spline curve to obtain a text row curve; estimating a vanishing point of a cylindrical surface straight bus, a text row tangent convergence line and a focal length parameter of a camera by using the tangent convergence symmetry of the text row curve; constructing latitude lines of an isometric grid for the fit text row curve by adopting a crossratio interpolation value; estimating a directrix of the cylindrical surface by using similar perspective projection transformation of the text row curve and constructing longitude lines of the isometric grid; estimating a length-to-width ratio of the isometric grid by using the text row curves at the top end and the bottom end of the image; and mapping meshes of each isometric grid into a square grid through a deformation technology based on the grid to finish the correction of the geometric distortion of the text image. By the method, the problem that perspective distortion, nonlinear deformation distortion and compound distortion of the perspective distortion and the nonlinear deformation distortion cannot be corrected at the same time in the prior art can be solved; therefore, total correction of the geometric distortion of the text image shot by the camera can be realized.” And page 5, [0016], “Step S4: Using the perspective projection transformation of the text line curve, estimate the directrix of the cylinder, and thus construct the meridians of the equidistant grid;”; Examiner’s note: The term “isometric” in plain English describes things that have constant dimensions or lengths. In this case, the isometric grid, disclosed by Meng, would qualify as a grid that has equidistant vertical lines. Also, the term “meridian” in plain English, are “vertical lines,” often used for describing lines of longitude on the Earth’s surface that run from the North to South Pole.)
determining the complete document boundary by fitting a curve using a plurality of representative points, the plurality of representative points are associated with the plurality of equidistant vertical lines. (Campbell, FIGS. 6, 17A, [0072], “FIG. 16 illustrates an exemplary workflow process to identify or generate the page curves that span between the page boundary lines. The next step is to identify the top and bottom page curves that span between the boundary lines for each page. The process groups the curves into top and bottom candidates based on relationships to the content boundaries, then caches the boundary intersection state for each curve. The grouped curves then are parsed to identify the best path for top and bottom curve. Finally a point set is generated for each curve.” and [0076], “FIGS. 17A and 17B illustrate how the page curves may be labeled into sets. In FIG. 17A the boundary lines L1, L2 are illustrated both with the long thin vertical lines and the thicker shorter horizontal lines.” and [0079], “Likewise, if the curve only intersects L1 but not L2 then it is given leftlabel (1704) and if the curve only intersects L2 but not L1, then it is given rightlabel (1702). These labels define subsets in the possible page contours to assist in search and generation of the best path.” and Hunt, [0075], “To identify regions corresponding to paragraphs, it may be assumed that in a paragraph, a text line is parallel to text lines immediately above or below, and these lines have roughly the same shape and size. Additionally, it may be assumed that the vertical distance between text lines is constant.”)
40. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the non-transitory computer-readable medium of claim 15 of Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt to include the disclosure of determining a plurality of curves and lines to identify a document's boundaries of Campbell. The motivation for this modification could have been to clearly establish the distortion or warp of a document image including at the document’s boundaries.
41. As per claim 19, Cheng in view of Fero, further in view of Wei, further in view of Meng, further in view of Hunt, and further in view of Campbell discloses: The non-transitory computer-readable medium of claim 17, determining the representative point associated with the probable document boundary region causes the processing device to perform operations comprising:
determining a set of intermediate points, wherein the set of intermediate points is based on intersections of the plurality of parallel curves and the plurality of equidistant vertical lines. (Campbell, [0093], “The curves marked as 700 and 702 respectively denote the top and bottom page curves. The end points of these curves intersect the page boundary lines L1, L2, which intersect at the page's vanishing point. The vertical ruled lines intersect the curves and also intersect the vanishing point illustrated by the dashed lines 704 through 722.” and [0097], “The vertical subdivision (704 through 722 in FIG. 7) are created using the approximation and the vanishing point to create lines that intersect both the vanishing point and the points on the polygonal approximation picked in prior steps. In one aspect, the line is then intersected to the opposing curve using a line Bezier curve intersection function rather than intercepting the polygonal approximation.” and Hunt, [0075], “To identify regions corresponding to paragraphs, it may be assumed that in a paragraph, a text line is parallel to text lines immediately above or below, and these lines have roughly the same shape and size. Additionally, it may be assumed that the vertical distance between text lines is constant.”)
42. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the non-transitory computer-readable medium of claim 17 of Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt to include the disclosure of determining a set of intermediate points based on the intersections between the plurality of curves and lines of a document of Campbell. The motivation for this modification could have been to have a more precise set of points to more accurately represent the warp or distortion of a document.
43. As per claim 20, Cheng in view of Fero, further in view of Wei, further in view of Meng, further in view of Hunt, and further in view of Campbell discloses: The non-transitory computer-readable medium of claim 19, determining the representative point associated with the probable document boundary region causes the processing device to perform operations comprising:
averaging the set of intermediate points with respect to a number of the plurality of parallel curves. (Campbell, [0076], “FIGS. 17A and 17B illustrate how the page curves may be labeled into sets. In FIG. 17A the boundary lines L1, L2 are illustrated both with the long thin vertical lines and the thicker shorter horizontal lines. The extent of the shorter lines illustrates the marking of the content boundary by text region detection. The extent is defined by points p0, p1 on L1 and the points p2, p3 on L2. Then, the line LC1 is defined by the points ptLC1_0=0.9*p0+0.1*p1 and ptLC1_2=0.9*p2+0.1*p3 as a weighted average of the content boundary points, where p0, p2 are closest to top and p1, p3 are closest to bottom. Likewise, the line LC2 is defined by the points ptLC2_0=0.9*p1+0.1*p0 and ptLC2_2=0.9*p3+0.1*p2 as a weighted average of the content boundary points. The lines LC1 and LC2 are illustrated in FIG. 17A. Alternatively stated, the filtered conters are trimmed and divided into sets for curves likely to be top or bottom page curves. The top region is defined by boundary lines L1, L2 and content boundary LC1. While the bottom region is defined by boundary lines L1, L2 and content boundary LC2.” and [0097]-[0098], “In one aspect line Bezier curve intersection method uses the same piecewise Bezier curves polygonization used earlier to initially generate a fine polygonized approximation of the curve before simplification, and then the intersection is selected as the point closest to the intersection line. In another aspect, the line is intersected with the polygonal approximation. When the page's boundary lines L1 and L2 are nearly parallel, the calculations are modified to create the intersection lines using the point on the curve and the average direction of L1 and L2. This is done to improve numerical stability when vanishing point is ill-defined or becomes a vanishing direction.”)
44. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the non-transitory computer-readable medium of claim 19 of Cheng in view of Fero, further in view of Wei, further in view of Meng, and further in view of Hunt to include the disclosure of averaging the set of intermediate points based on the intersections between the plurality of curves and lines of a document of Campbell. The motivation for this modification could have been to use the average as a means of smoothing out the representative curve and more accurately representing the curve of a document.
45. Claims 21 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (CN-116453131-B, hereinafter "Cheng") in view of Fero et al. (US20100014782A1, hereinafter "Fero"), further in view of Wei (CN-105450900-A), and further in view of An et al. (CN-113610089-A, hereinafter "An").
46. As per claim 21, Cheng in view of Fero, further in view of Wei discloses: The method of claim 1, [[wherein the offset is a margin size.]] (See rejection for claim 1.)
47. Cheng in view of Fero, further in view of Wei doesn't explicitly disclose but An discloses: [[The method of claim 1,]] wherein the offset is a margin size. (An, p. 5-6, [0008]-[0015], “Optionally, before determining the current target text line to be corrected, the method further includes: identifying text lines in the page image to be corrected; Optionally, the step of identifying lines of text in the page image to be corrected includes any one or a combination of the following: Detect text targets in the page image to be corrected and obtain the position information of the text targets; based on the position information of the text targets, determine the text lines located on the same line; or, The text lines in the page image to be corrected are identified based on the image grayscale. If the page image to be corrected has a horizontally or vertically extending grayscale region with a width that meets a preset value, it is determined to be a text line; or The text lines are detected using an artificial intelligence model designed to detect text lines in an image. Optionally, the step of detecting text targets in the page image to be corrected and obtaining the position information of the text targets includes any one or more of the following methods in combination: The text target is detected by performing OCR recognition on the page image to be corrected, and the position information of the recognized text target is obtained; The text targets in the image are detected by an artificial intelligence model, and the location information of the detected text targets is obtained.” And p. 6-7, [0032]-[0037], “Optionally, before determining the target text line to be corrected, a step of pre-correcting the page is also included; Optionally, the step of pre-correcting the page includes: determining the top corner of the paper page from the page image to be corrected; comparing the coordinates of the top corner with the corresponding reference horizontal line and/or reference vertical line, and calculating the offset that needs to be corrected for each top corner; and pre-correcting the page image to be corrected according to the calculated offset. Optionally, a trained artificial intelligence model can be used to automatically identify the top corners in the page image to be corrected, and the position information of the identified top corners can be stored in a database; Optionally, the page pre-correction includes: removing irrelevant information from the page image to be corrected, and stretching the page region in the page image to be corrected to a normalized shape corresponding to the page. Optionally, the step of pre-correcting the page image to be corrected based on the calculated relative offset further includes: The relative offset of the pre-correction auxiliary point located between any two of the vertices is calculated by interpolation or curve fitting.”; Examiner’s note: An discloses two additional procedures that can occur prior to “determining the target text line to be corrected.” The first is “identifying text lines in the page image to be corrected” (a plurality of text lines) disclosed in [0008]-[0015] and the second is “a step of pre-correcting the page” disclosed in [0032]-[0037]. The first procedure, “Detect(s) text targets in the page image to be corrected and obtain(s) the position information of the text targets; based on the position information of the text targets, determine the text lines located on the same line” ([0010]). The second procedure discloses: “determining the top corner of the paper page from the page image to be corrected; comparing the coordinates of the top corner with the corresponding reference horizontal line and/or reference vertical line, and calculating the offset that needs to be corrected for each top corner; and pre-correcting the page image to be corrected according to the calculated offset” ([0033]). Using the corners to determine an offset from the reference line accounts for the margin of the page in relation to the entire document. With positional information of a line’s text targets known from the first procedure, the “relative offset of the pre-correction auxiliary point located between any two of the vertices is calculated by interpolation or curve fitting.” ([0037]). This indicates an additional offset utilizing curve fitting based on the positions (vertices) of a line’s text targets as part of the pre-correcting the page (which would indicate the text curves).)
48. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Cheng in view of Fero, further in view of Wei to include the disclosure of the offset being a margin size, of An. The motivation for this modification could have been to help provide a consistent margin so that the grid is consistent across a document for boundary detection or for a rectification process. This helps to standardize the warped nature of the document so that a method or algorithm can determine how to properly correct for the distortion.
Conclusion
49. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
50. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW CLOTHIER whose telephone number is (571)272-4667. The examiner can normally be reached Mon-Fri 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW CLOTHIER/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights