DETAILED ACTION
Applicant's amendment of January 22, 2026 overcomes the following:
Claim objections
Applicant has amended claims 1, 3, 6-10, 12-18, and 24. Claims 21-22 have been cancelled. Claims 25-26 are new. Claims 1-3, 6-18, and 23-26 are pending.
Response to Arguments
Applicant’s arguments filed on January 22, 2026 with respect to pending claims have been considered but are moot in view of the new ground(s) of rejection. The amended claims resulted in changes to the scope and contents; therefore, the grounds of rejection are modified accordingly. It is noted that previously applied prior arts remain in effect.
Regarding the interview conducted on January 13, 2026, Applicant asserts that the “Examiner agreed that the proposed amendments, substantially included in this response, appear to overcome the applied references pending further search” (Remarks, Pg. 10).
Examiner respectfully disagrees.
During the interview conducted on January 13, 2026, the examiner informed representative that “any remarks and proposed claim amendments would require further consideration and search of prior art than the allotted time for the interview in order to properly assess allowability of instant application. Further informed representative that any remarks and/or claim amendments will be fully considered and submitted to further search of prior art after a formal response/proper reply to the pending OA is received” and no agreement was reached during the interview, as indicated in the Examiner Interview Summary Record of January 16, 2026.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8-18, 23-24, and 26 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 8 now recites the limitation “calculate a first offset value and a second offset value” in line 12 of the claim. However, it is not clear if any of the claimed “first offset value” and “second offset value”, respectively, include embodiments corresponding to the claimed “offset values” previously recited in line 9 of the claim, or if any of the claimed “first offset value” and “second offset value”, respectively, do not include embodiments corresponding to the claimed “offset values” previously recited in line 9 of the claim, for example.
Therefore, based on above, the metes and bounds of the claim are not clearly set forth and the examiner cannot clearly determine which elements are encompassed by the claim language, which renders the claim indefinite.
Claims 13-14 are rejected by virtue of being dependent upon rejected base claim 8.
Claim 15 now recites the limitation “calculate a first offset value and a second offset value” in line 17 of the claim. However, it is not clear if any of the claimed “first offset value” and “second offset value”, respectively, include embodiments corresponding to the claimed “offset values” previously recited in line 11 of the claim, or if any of the claimed “first offset value” and “second offset value”, respectively, do not include embodiments corresponding to the claimed “offset values” previously recited in line 11 of the claim, for example.
Therefore, based on above, the metes and bounds of the claim are not clearly set forth and the examiner cannot clearly determine which elements are encompassed by the claim language, which renders the claim indefinite.
Claims 16-18, 24, and 26 are rejected by virtue of being dependent upon rejected base claim 15.
Claim 23 recites the limitation “wherein the changed orientation of the document changes a file size” in lines 1-2 of the claim.
However, the claimed “file size” recited in claim 23 is not defined by the claims and it is not clear if the claimed “file” recited in claim 23 encompass embodiments corresponding to a digital “file” of the claimed “image” recited in line 1 of claim 1, or if the claimed “file” recited in claim 23 encompass embodiments corresponding to another “file” that is different form a digital “file” of the claimed “image” recited in line 1 of claim 1, for example.
Therefore, based on above, the metes and bounds of the claim are not clearly set forth and the examiner cannot clearly determine which elements are encompassed by the claim language, which renders the claim indefinite.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-7, and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Cali et al. (US PG Publication No. 2019/0220660 A1), hereafter referred to as Cali, in view of MINAMINO et al. (US PG Publication No. 2009/0268264 A1), hereafter referred to as MINAMINO, Applicant cited prior art originally cited by the examiner during examination of parent application, in further view of CHEN et al. (US PG Publication No. 2013/0163896 A1), hereafter referred to as CHEN.
Regarding claim 1, Cali teaches a method (Par. [0002]: a method to extract data from documents… a computer-implemented method for classifying a type of a document in an image and extracting data from the classified document), comprising:
changing, by a user device, an orientation of a document in an image relative to one or more reference axes (Par. [0028]: extract image data from the document image may be performed by a process that comprises: finding a transform to define a bounding box of a template of the determined document type in the document image. The transform may comprise a rotation or scaling of the determined document image relative to the document image axes. This ensures that the method can accept image data that has not been acquired under ideal conditions. For example, it may use image data acquired using a camera that is skewed due the angle at which the camera was operated; Par. [0059-61]: data extraction process for a document will now be described… As an example, image data may be acquired by using the camera 253 of the mobile electronic device 201 to take an image of an official document 500… The acquired image may include multiple documents or a document in a background setting… Before or after cropping, the image data of the image may be rotated, preferably to align the document with the image axis using known methods; Par. [0104]: the official document, or a portion of an official document, is located within the image data and the rotation of the official document with the image axis is calculated; Par. [0213]: wherein the transform comprises a rotation or scaling of the determined document image relative to the document image axes; changing, by a user device, an orientation of a document relative to one or more reference axes (e.g. computer-implemented method to extract data from documents in images includes acquiring an image comprising image data relating to at least a part (i.e. a dimension, a portion, a section, a segment, a side, etc.) of a document, for example, and manipulating (i.e. changing, transforming, etc.) the image data of the document to obtain a document image by using a user device, such as mobile electronic device 201 (i.e. by a user device), for example, in which the document, or a portion of the document, is located within the image data, for example, and the image data of the document within the image is rotated (i.e. change an orientation) to align (i.e. de-skew) the determined document image relative to the image axes (i.e. changing, by a user device, an orientation of a document relative to one or more reference axes), as indicated above), for example); and
performing, by the user device, an action related to the image based on at least one dimension of the document aligning with the one or more reference axes (Par. [0028]: extract image data from the document image may be performed by a process that comprises: finding a transform to define a bounding box of a template of the determined document type in the document image. The transform may comprise a rotation or scaling of the determined document image relative to the document image axes. This ensures that the method can accept image data that has not been acquired under ideal conditions. For example, it may use image data acquired using a camera that is skewed due the angle at which the camera was operated; Par. [0061]: Before or after cropping, the image data of the image may be rotated, preferably to align the document with the image axis using known methods; Par. [0104]: the official document, or a portion of an official document, is located within the image data and the rotation of the official document with the image axis is calculated. The relevant image data is then identified or extracted; Par. [0213]: wherein the transform comprises a rotation or scaling of the determined document image relative to the document image axes; and
performing, by the user device, an action related to the image based on at least one dimension of the document aligning with the one or more reference axes (e.g. computer-implemented method to extract data from documents in images includes acquiring an image comprising image data relating to at least a part (i.e. a dimension, a portion, a section, a segment, a side, etc.) of a document (i.e. based on at least one dimension of the document), for example, and manipulating the image data of the document to obtain a document image by using a user device, such as mobile electronic device 201 (i.e. by the user device), for example, in which the document, or a portion of the document, is located within the image data, for example, and the image data of the document within the image is rotated to align (i.e. de-skew) the determined document image relative to the image axes, for example, and relevant image data is then identified or extracted (i.e. performing, by the user device, an action related to the image) after rotating the image data of the document within the image (i.e. performing, by the user device, an action related to the image based on at least one dimension of the document aligning with the one or more reference axes), as indicated above), for example), but fails to teach the following as further recited in claim 1.
However, MINAMINO teaches wherein image pixels included in the document are part of the changed orientation of the document;
determining two dimensions of the document based on the image pixels (Par. [0088-96]: feature point rotation calculating unit 81 preferably inputs the value regarding the original document inclination acquired… and then calculates positions of rotated points obtained by rotating and moving the plurality of feature points… by the inclination angle (i.e., in a direction for correcting the original document inclination) centering around a prescribed center point… Based on the positions of the feature points after the rotation (hereinafter, referred to as the rotated feature points) acquired through the feature point rotation calculating unit 81, the rectangular area calculating unit 82 preferably calculates a position and a size of a non-inclined rectangular area having an outline that is disposed in the vicinity of the rotated feature points; Par. [0182-200]: a process of rotating the original document target area 12 around the center point 13 by the inclination angle θ is performed as illustrated in FIG. 19… a rectangular area (extraction area 14) inclined by the same angle as the inclination angle of the original document can be acquired as illustrated in FIG. 19… the coordinates of three vertexes 14a, 14b, and 14c among the four vertexes of the rectangular extraction area 14 are transferred as parameters to the extraction rotation process unit 90… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated… the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired… Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image… Assuming that the first row and first column of the rotated image are the target pixels, FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image… FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image… In the example of FIG. 21, a rectangular image slightly rotated in the clockwise direction from the correct direction is acquired as an original document pixel area… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated (S703). In this process, firstly, the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired… Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image represented… Next, the pixel value "Q(m, n)" of the target pixel of the rotated image is acquired through two-dimensional linear interpolation. As illustrated in FIG. 24, the two-dimensional linear interpolation uses four pixels: the corresponding target pixel (i, j) of the original image; the pixel (i-1, j) arranged next to the corresponding target pixel in the x-direction; the pixel (i, j+1) arranged next to the corresponding target pixel in the y-direction; wherein image pixels included in the document are part of the changed orientation of the document;
determining two dimensions of the document based on the image pixels (e.g. image processing apparatus (i.e. user device) automatically detects a prescribed area, including a portion (i.e. a dimension, a portion, a section, a segment, a side, etc.) of an original document (i.e. the document), based on image data acquired, for example, by detecting feature points (i.e. pixels, coordinates, edges, etc.) related to an outline (i.e. boundary, area, shape, dimensions, size, etc.) of the original document, and calculating positions of rotated points (i.e. coordinates, dimensions, etc.) obtained by rotating and moving the detected feature points, for example, including calculated pixel values of target pixels of the rotated image (i.e. wherein image pixels included in the document are part of the changed orientation of the document), for example, and calculates a position and a size (i.e. dimensions) of a non-inclined (i.e. rotated, aligned, de-skewed, etc.) rectangular area having an outline that is disposed in the vicinity of rotated feature points of the image data acquired, for example, by performing two-dimensional interpolation process to acquire each pixel value of the target pixels of the rotated image, for example, including x-coordinates and y-coordinates of each corresponding target pixel of the target pixels of the rotated image (i.e. determining two dimensions of the document based on the image pixels), as indicated above), for example);
calculating, by the user device, a first offset value and a second offset value,
the first offset value being associated with a first dimension of the two dimensions of the document, and
the second offset value being associated with a second dimension of the two dimensions of the document (Par. [0088-96]: feature point rotation calculating unit 81 preferably inputs the value regarding the original document inclination acquired… and then calculates positions of rotated points obtained by rotating and moving the plurality of feature points… by the inclination angle (i.e., in a direction for correcting the original document inclination) centering around a prescribed center point… Based on the positions of the feature points after the rotation (hereinafter, referred to as the rotated feature points) acquired through the feature point rotation calculating unit 81, the rectangular area calculating unit 82 preferably calculates a position and a size of a non-inclined rectangular area having an outline that is disposed in the vicinity of the rotated feature points…. By performing a prescribed calculation based on a position of a target pixel (m, n) of a rotated image, the original image corresponding position calculating unit 92 preferably acquires a position of a corresponding target pixel (i, j), which corresponds to the target pixel (m, n) in the original image. By performing the prescribed calculation based on the position of the target pixel, the original image corresponding position calculating unit 92 preferably acquires an x-direction weighting factor "kwx" and a y-direction weighting factor "kwy" that are used in an interpolation process performed through the two-dimensional interpolation unit 93… Based on the corresponding target pixel (i, j) and three pixels each having at least one of the x-coordinate and the y-coordinate that are different from that of the corresponding target pixel, the two-dimensional interpolation unit 93 performs the two-dimensional interpolation process to acquire a pixel value "Q (m, n)" of the target pixel of the rotated image; Par. [0182-200]: a process of rotating the original document target area 12 around the center point 13 by the inclination angle θ is performed as illustrated in FIG. 19… a rectangular area (extraction area 14) inclined by the same angle as the inclination angle of the original document can be acquired as illustrated in FIG. 19… the coordinates of three vertexes 14a, 14b, and 14c among the four vertexes of the rectangular extraction area 14 are transferred as parameters to the extraction rotation process unit 90… In the example of FIG. 21, a rectangular image slightly rotated in the clockwise direction from the correct direction is acquired as an original document pixel area… in FIG. 21, the extraction area 14 having substantially the same inclination as the original document inclination is determined… the extraction rotation process unit 90 preferably calculates and acquires the difference between the x-coordinates and the difference between the y-coordinates… The acquired difference between the y-coordinates will be referred to as "dy", and the difference between the x-coordinates will be referred to as "dx"… extraction rotation process unit 90 first acquires two inclination integer parameters from the difference of the x-coordinates and the difference of the y-coordinates of the two vertexes 14a and 14b of the extraction area 14… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated (S703). In this process, firstly, the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired by adding the offset value "moff" to the x-coordinate "m" of the target pixel of the rotated image (i=m+moff). Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired by adding the offset value "noff" to the y-coordinate "n" of the target pixel of the rotated image (j=n+noff)… Each time the target pixel of the rotated image moves by one pixel in the y-direction, the first integer parameter "a" is added to the x-direction weighting factor "kwx" (S712 of FIG. 22). Each time the target pixel of the rotated image moves by one pixel in the x-direction, the first integer parameter "a" is added to the y-direction weighting factor "kwy" (S705); calculating, by the user device, a first offset value and a second offset value, the first offset value being associated with a first dimension of the two dimensions of the document, and the second offset value being associated with a second dimension of the two dimensions of the document (e.g. image processing apparatus (i.e. user device) automatically detects a prescribed area, including a portion (i.e. a dimension, a portion, a section, a segment, a side, etc.) of an original document (i.e. the document), based on image data acquired, for example, by detecting feature points (i.e. pixels, coordinates, edges, etc.) related to an outline (i.e. boundary, area, shape, dimensions, size, etc.) of the original document, and calculating positions of rotated points (i.e. coordinates, dimensions, etc.) obtained by rotating and moving the detected feature points, including calculated pixel values of target pixels of the rotated image, for example, by performing two-dimensional interpolation process to acquire each pixel value of the target pixels of the rotated image, including x-coordinates and y-coordinates of each corresponding target pixel of the target pixels of the rotated image (i.e. two dimensions of the document based on the image pixels), for example, by calculating and acquiring a difference (i.e. offset, change, variance, deviation, shift, displacement, etc.) between the x-coordinates (i.e. calculating, by the user device, a first offset value being associated with a first dimension of the two dimensions of the document) and a difference between the y-coordinates of the rotated image (i.e. calculating, by the user device, a second offset value being associated with a second dimension of the two dimensions of the document), for example, as shown by “dx” and “dy” in Fig. 21 below:
PNG
media_image1.png
556
444
media_image1.png
Greyscale
, for example); and
performing, by the user device, an action related to the first offset and the second offset value (Par. [0192-199]: the image rotating process executed through the extraction rotation process unit 90 will be described in detail with reference to the flowchart of FIG. 22… When the flow of FIG. 22 is started, the extraction rotation process unit 90 first acquires two inclination integer parameters from the difference of the x-coordinates and the difference of the y-coordinates of the two vertexes 14a and 14b of the extraction area 14, and input the acquired parameters as the first integer parameter "a" and the second integer parameter "b" (S701)… Then, an initialization process of variables is performed (S702). In the initialization process, the x-coordinate "m" and the y-coordinate "n" of the target pixel of the rotated image is reset to zero. Further, as for an x-direction offset value "moff" and a y-direction offset value "noff", the x-coordinate (s) and the y-coordinate (t) of the vertex 14a positioned at the upper left of the extraction area 14 of FIG. 21 are set as initial values. The x-direction offset value "moff" and the y-direction offset value "noff" are used to calculate the position of the corresponding target pixel (the original image pixel that corresponds to the target pixel)… Then, the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated (S703). In this process, firstly, the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired by adding the offset value "moff" to the x-coordinate "m" of the target pixel of the rotated image (i=m+moff). Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired by adding the offset value "noff" to the y-coordinate "n" of the target pixel of the rotated image (j=n+noff)… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image… FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image with the grids surrounded by double-lines. As illustrated in the upper drawing of FIG. 23, each time the target pixel of the rotated image moves by five pixels (i.e., by "b/a" pixels) in the x-direction, the corresponding target pixel of the original image is displaced by one pixel in the y-direction. Each time the target pixel of the rotated image moves by five pixels in the y-direction, the corresponding target pixel of the original image is displaced by one pixel in the x-direction… Next, the pixel value "Q(m, n)" of the target pixel of the rotated image is acquired through two-dimensional linear interpolation; and performing, by the user device, an action related to the first offset and the second offset value (e.g. image processing apparatus (i.e. user device) automatically detects a prescribed area, including a portion (i.e. a dimension, a portion, a section, a segment, a side, etc.) of an original document (i.e. the document), based on image data acquired, for example, calculates positions of rotated points (i.e. coordinates, dimensions, etc.) obtained by rotating and moving detected feature points, including calculated pixel values of target pixels of the rotated image, for example, and calculates the pixel values of target pixels of the rotated image, by performing two-dimensional interpolation process to acquire each pixel value of the target pixels of the rotated image, including x-coordinates and y-coordinates of each corresponding target pixel of the target pixels of the rotated image, for example, by calculating and acquiring a difference (i.e. offset, change, variance, deviation, shift, displacement, etc.) between the x-coordinates (i.e. the first offset) and the y-coordinates of the rotated image (i.e. the second offset), for example, and adds (i.e. performing, by the user device, an action) calculated offset values to the x-coordinates and the y-coordinates of the target pixel of the rotated image (i.e. and performing, by the user device, an action related to the first offset and the second offset value), as indicated above), for example).
Cali and MINAMINO are considered to be analogous art because they pertain to image processing applications. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the computer-implemented method to extract data from documents in an image of a document by manipulating the image data to obtain a document image by rotating the image data of the document within the image to align the determined document image relative to the image axes (as disclosed by Cali) with wherein image pixels included in the document are part of the changed orientation of the document; determining two dimensions of the document based on the image pixels; calculating, by the user device, a first offset value and a second offset value, the first offset value being associated with a first dimension of the two dimensions of the document, and the second offset value being associated with a second dimension of the two dimensions of the document (as taught by MINAMINO, Abstract, Par. [0088-96, 182-200]) to calculate an extraction area of the image data by rotating the original document target area, extracting the image data stored in the image memory in accordance with the extraction area, and electronically correcting the original document inclination by rotating the extracted data (MINAMINO, Abstract, Par. [0003-10, 25, 88-96, 199-200]).
The combination of Cali and MINAMINO, as a whole, teaches the method, as indicated above, but fails to teach the following as further recited in claim 1.
However, CHEN teaches adding one or more padding pixels to the image based on the changed orientation of the document (Par. [0003-11]: present invention relates to image registration, including estimating translation, rotation, and scaling parameters between a reference image and a distorted image… image registration method and system for processing a first image Y into a registered image Y* which is aligned with a second image X… determining the scale factor "a" by computing a ratio of a sum of pixel values of the reference image X to a sum of pixel values of the distorted image Y… obtaining the registered image Y* by translating the rotated distorted image Y3 to be aligned horizontally and vertically with the reference image X … The step (a) further comprises padding the smaller of the reference image X and the distorted image Y with zero-valued pixels to equalise horizontal and vertical dimensions of the reference image X and the distorted image Y; Par. [0020-30]: processing a distorted image Y into a registered image Y* by aligning it with a reference image X… determining a scale factor "a" between the reference image X and the distorted image Y; (b) resizing the reference image X with an inverse of the scale factor "a", thereby generating a resized reference image X2; (c) determining a rotation angle… between the resized reference image X2 and the distorted image Y; (d) rotating the distorted image Y by the rotation angle… to generate a rotated distorted image Y3; and (e) resizing the rotated distorted image Y3 with the scale factor "a", thereby obtaining the registered image Y*… determining the scale factor "a" by computing a ratio of a sum of pixel values of the reference image X to a sum of pixel values of the distorted image Y… padding the smaller of the reference image X and the distorted image Y with zero-valued pixels to equalise horizontal and vertical dimensions of the reference image X and the distorted image Y; Par. [0111-154]: systems and methods for pre-processing the distorted image by which the distorted image is aligned with the reference image… The process of aligning a distorted image with a reference image may also variously be referred to as "registering" or "registration"… Affine Transform Parameter Estimation sub-system 210 includes a Pixel Padding module 230… In the Pixel Padding module 230, the two images X and Y are passed through unchanged unless they are not the same size in terms of their horizontal and vertical dimensions "M" and "N", measured in pixels. If they are not of equal size, they are modified and their horizontal and vertical dimensions made equal in size by padding one or both of the images with zero-valued pixels appropriately… FIG. 3 is a flow chart of an affine transform based realignment method 300 according to the first embodiment of the invention, including steps:… In step 310 "Zero Pad Image(s)" which is performed in the Pixel Padding module 230 of FIG. 2a, the dimensions m and n of the Reference Image X are compared with the corresponding dimensions m' and n' of the Distorted Image Y… If the Reference Image X and the Distorted Image Y do not have the same number of pixels, the dimension of the equalized images X and Y are set as M and N, where M is the larger of m and m', and N is the larger of n and n'. Either image may then be padded by adding zero pixels around the original image in the shorter dimensions, to create the Equalized Reference Image X' or the Equalized Distorted Image Y' replacing the Reference Image X and the Distorted Image Y as the case may be; adding one or more padding pixels to the image based on the changed orientation of the document (e.g. methods of image quality assessment of a distorted image based on a comparison of the distorted image with a reference image, including images of objects and/or documents, for example, includes obtaining a registered image by translating the rotated distorted image aligned horizontally and vertically with the reference image (i.e. changed orientation of the document), for example, and padding the smaller of the reference image and the distorted image with zero-valued pixels to equalize horizontal and vertical dimensions of the reference image and the distorted image (i.e. adding one or more padding pixels to the image based on the changed orientation of the document), as indicated above), for example),
determining two dimensions of the document based on the image pixels and the one or more padding pixels (Par. [0003-11]: present invention relates to image registration, including estimating translation, rotation, and scaling parameters between a reference image and a distorted image… image registration method and system for processing a first image Y into a registered image Y* which is aligned with a second image X… determining the scale factor "a" by computing a ratio of a sum of pixel values of the reference image X to a sum of pixel values of the distorted image Y… obtaining the registered image Y* by translating the rotated distorted image Y3 to be aligned horizontally and vertically with the reference image X … The step (a) further comprises padding the smaller of the reference image X and the distorted image Y with zero-valued pixels to equalise horizontal and vertical dimensions of the reference image X and the distorted image Y; Par. [0020-30]: processing a distorted image Y into a registered image Y* by aligning it with a reference image X… determining a scale factor "a" between the reference image X and the distorted image Y; (b) resizing the reference image X with an inverse of the scale factor "a", thereby generating a resized reference image X2; (c) determining a rotation angle… between the resized reference image X2 and the distorted image Y; (d) rotating the distorted image Y by the rotation angle… to generate a rotated distorted image Y3; and (e) resizing the rotated distorted image Y3 with the scale factor "a", thereby obtaining the registered image Y*… determining the scale factor "a" by computing a ratio of a sum of pixel values of the reference image X to a sum of pixel values of the distorted image Y… padding the smaller of the reference image X and the distorted image Y with zero-valued pixels to equalise horizontal and vertical dimensions of the reference image X and the distorted image Y; Par. [0111-154]: systems and methods for pre-processing the distorted image by which the distorted image is aligned with the reference image… The process of aligning a distorted image with a reference image may also variously be referred to as "registering" or "registration"… Affine Transform Parameter Estimation sub-system 210 includes a Pixel Padding module 230… In the Pixel Padding module 230, the two images X and Y are passed through unchanged unless they are not the same size in terms of their horizontal and vertical dimensions "M" and "N", measured in pixels. If they are not of equal size, they are modified and their horizontal and vertical dimensions made equal in size by padding one or both of the images with zero-valued pixels appropriately (for each dimension, we pad the image with the smallest resolution in that dimension). The dimension-equalized images will be referred to as Equalized Reference Image X' and Equalized Distorted Image Y', which are both of size MxN pixels… FIG. 3 is a flow chart of an affine transform based realignment method 300 according to the first embodiment of the invention… In step 310 "Zero Pad Image(s)" which is performed in the Pixel Padding module 230 of FIG. 2a, the dimensions m and n of the Reference Image X are compared with the corresponding dimensions m' and n' of the Distorted Image Y… If the Reference Image X and the Distorted Image Y do not have the same number of pixels, the dimension of the equalized images X and Y are set as M and N, where M is the larger of m and m', and N is the larger of n and n'. Either image may then be padded by adding zero pixels around the original image in the shorter dimensions, to create the Equalized Reference Image X' or the Equalized Distorted Image Y' replacing the Reference Image X and the Distorted Image Y as the case may be; determining two dimensions of the document based on the image pixels and the one or more padding pixels (e.g. methods of image quality assessment of a distorted image based on a comparison of the distorted image with a reference image, including images of objects and/or documents, for example, includes obtaining a registered image by translating the rotated distorted image aligned horizontally and vertically with the reference image, for example, and determining a scale factor by computing a ratio of a sum of pixel values of the reference image to a sum of pixel values of the distorted image (i.e. the image pixels), for example, and padding the smaller of the reference image and the distorted image with zero-valued pixels to equalize horizontal and vertical dimensions of the reference image and the distorted image, for example, and by determining if the reference image and the distorted image are not the same size in terms of their horizontal and vertical dimensions "M" and "N" (i.e. determining two dimensions of the document), or measured in pixels (i.e. based on the image pixels), for example, then their modified horizontal and vertical dimensions are made equal (i.e. equalized) in size (i.e. dimensions) by padding one or both of the images with zero-valued pixels (i.e. determining two dimensions of the document based on the image pixels and the one or more padding pixels), as indicated above), for example).
Cali, MINAMINO and Chen are considered to be analogous art because they pertain to image processing applications. Therefore, the combined teachings of Cali, MINAMINO and Chen, as a whole, would have rendered obvious the invention recited in claim 1 with a reasonable expectation of success in order to modify the computer-implemented method to extract data from documents in an image of an official document by manipulating the image data to obtain a document image by rotating the image data of the document within the image to align the determined document image relative to the image axes (as disclosed by Cali) with adding one or more padding pixels to the image based on the changed orientation of the document, and determining two dimensions of the document based on the image pixels and the one or more padding pixels (as taught by CHEN, Abstract, Par. [0003-11, 20-30, 111-154]) to improve the objective measurement of visual image quality, and provide image registration that is robust to noise (CHEN, Abstract, Par. [0007, 48, 325]).
Regarding claim 2, claim 1 is incorporated and the combination of Cali, MINAMINO and Chen, as a whole, teaches the method (Cali, Par. [0002]), wherein performing the action comprises:
creating a rectangular outline of the document (MINAMINO, Par. [0080-95]: automatic image acquiring unit 95 preferably extracts a rectangular area of a proper size including an original document area from the image data, and thus acquires an original document image having no inclination by rotating the extracted area. The automatic image acquiring unit 95 preferably includes an inclination detecting unit 70, an image extraction determining unit 80, and an extraction rotation process unit 90… inclination detecting unit 70 preferably detects an inclination of the original document scanned through the CCD 28. When the image data is input line by line from the data correction unit 65 of the scanner unit 21, the inclination detecting unit 70 analyzes the input image data and detects an inclination (i.e., an angle to be rotated to correct the inclination) of the original document… feature point detecting unit 72 can store the positions of the edge pixels of a prescribed number of lines acquired through the edge pixel acquiring unit 71. Based on features of the positions of the edge pixels of the plurality of lines, feature points related to an outline of the original document are detected, and positions of the feature points can be acquired. In the present preferred embodiment, the "feature point" refers to a point that is positioned at a graphic characteristic portion of the outline of the original document, such as the top of a corner portion of the original document… Based on the positions of the feature points after the rotation (hereinafter, referred to as the rotated feature points) acquired through the feature point rotation calculating unit 81, the rectangular area calculating unit 82 preferably calculates a position and a size of a non-inclined rectangular area having an outline that is disposed in the vicinity of the rotated feature points… extraction area calculating unit 85 calculates an extraction area of the image data by rotating, around the center point, the original document target area determined through the target area determining unit 84… Based on the process results of the inclination detecting unit 70 and the image extraction determining unit 80, the extraction rotation process unit 90 preferably extracts the image data stored in the image memory 44 in accordance with the extraction area, and electronically corrects the original document inclination by rotating the extracted data; Par. [0174-187]: inclination detecting process of the above-described present preferred embodiment can properly detect the inclination regardless of the content of the original document by analyzing the original document pixels. Moreover, the inclination of original documents of various shapes or in various states can be accurately detected even when the original document is dog-eared, torn, for example, or has a round-cornered rectangular shape, a non-rectangular shape, or other unconventional shape… a process of determining an extraction area of a prescribed size including the original document area from the original image data (i.e., a process executed through the image extraction determining unit 80) will be described. FIG. 17 is a flowchart representing the extraction area determining process executed through the image extraction determining unit 80… a process of rotating, around a point predetermined as the center, the coordinates of each feature point acquired in the above-described process, by the inclination angle (i.e., in a direction for correcting the inclination) acquired in the above-described inclination detecting process is performed (S601). Such a rotational transfer can preferably be carried out by performing a well-known affine transformation on the x-coordinate and the y-coordinate of each feature point, for example. FIG. 18 represents a process of rotating a plurality of feature points 10p acquired from the data of FIG. 16 around a center point 13 by the inclination angle θ, and then acquiring rotated feature points 10q… Next, a rectangular area 11 that includes all the rotated feature points 10q is determined (S602). The rectangular area 11 has a non-inclined, rectangular outline including an outline; wherein performing the action comprises: creating a rectangular outline of the document (e.g. image processing apparatus (i.e. user device) automatically detects a prescribed area, including a portion (i.e. a dimension, a portion, a section, a segment, a side, etc.) of an original document (i.e. the document), based on image data acquired, for example, by detecting feature points (i.e. pixels, coordinates, edges, etc.) related to a rectangular outline (i.e. boundary, area, shape, dimensions, size, etc.) of the original document, for example, to calculate a position and a size of a non-inclined rectangular area having an outline that is disposed in the vicinity of the rotated feature points (i.e. wherein performing the action comprises creating a rectangular outline of the document), as indicated above), for example).
The same motivation to combine above-mentioned teachings applies, as previously indicated in claim 1.
Regarding claim 3, claim 1 is incorporated and the combination of Cali, MINAMINO and Chen, as a whole, teaches the method (Cali, Par. [0002]), wherein determining the at least two dimensions of the document based on the changed orientation of the document comprises:
determining one or more of a height of the image or a width of the image (MINAMINO, (Par. [0088-96]: feature point rotation calculating unit 81 preferably inputs the value regarding the original document inclination acquired… and then calculates positions of rotated points obtained by rotating and moving the plurality of feature points… by the inclination angle (i.e., in a direction for correcting the original document inclination) centering around a prescribed center point… Based on the positions of the feature points after the rotation (hereinafter, referred to as the rotated feature points) acquired through the feature point rotation calculating unit 81, the rectangular area calculating unit 82 preferably calculates a position and a size of a non-inclined rectangular area having an outline that is disposed in the vicinity of the rotated feature points; Par. [0182-200]: a process of rotating the original document target area 12 around the center point 13 by the inclination angle θ is performed as illustrated in FIG. 19… a rectangular area (extraction area 14) inclined by the same angle as the inclination angle of the original document can be acquired as illustrated in FIG. 19… the coordinates of three vertexes 14a, 14b, and 14c among the four vertexes of the rectangular extraction area 14 are transferred as parameters to the extraction rotation process unit 90… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated… the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired… Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image… Assuming that the first row and first column of the rotated image are the target pixels, FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image… FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image… In the example of FIG. 21, a rectangular image slightly rotated in the clockwise direction from the correct direction is acquired as an original document pixel area… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated (S703). In this process, firstly, the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired… Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image represented… Next, the pixel value "Q(m, n)" of the target pixel of the rotated image is acquired through two-dimensional linear interpolation. As illustrated in FIG. 24, the two-dimensional linear interpolation uses four pixels: the corresponding target pixel (i, j) of the original image; the pixel (i-1, j) arranged next to the corresponding target pixel in the x-direction; the pixel (i, j+1) arranged next to the corresponding target pixel in the y-direction; wherein determining the at least two dimensions of the document based on the changed orientation of the document comprises:
determining one or more of a height of the image or a width of the image (e.g. image processing apparatus (i.e. user device) automatically detects a prescribed area (i.e. one or more of a height or a width), including a portion (i.e. a dimension, a portion, a section, a segment, a side, etc.) of an original document (i.e. the document), based on image data acquired (i.e. determining one or more of a height of the image or a width of the image), for example, by detecting feature points (i.e. pixels, coordinates, edges, etc.) related to an outline (i.e. boundary, area, shape, dimensions, size, etc.) of the original document, for example, and calculating positions of rotated points (i.e. coordinates, dimensions, etc.) obtained by rotating and moving the detected feature points, for example, including calculated pixel values of target pixels of the rotated image, including x-coordinates and y-coordinates of each corresponding target pixel of the target pixels of the rotated image, for example, by calculating a position and a size (i.e. dimensions) of a non-inclined (i.e. rotated, aligned, de-skewed, etc.) rectangular area having an outline that is disposed in the vicinity of rotated feature points of the image data acquired (i.e. wherein determining the at least two dimensions of the document based on the changed orientation of the document comprises: determining one or more of a height of the image or a width of the image), as indicated above), for example).
The same motivation to combine above-mentioned teachings applies, as previously indicated in claim 1.
Regarding claim 6, claim 1 is incorporated and the combination of Cali, MINAMINO and Chen, as a whole, teaches the method (Cali, Par. [0002]), further comprising:
calculating at least one dimension of the document after rotating the image (MINAMINO, Par. [0088-96]: feature point rotation calculating unit 81 preferably inputs the value regarding the original document inclination acquired… and then calculates positions of rotated points obtained by rotating and moving the plurality of feature points… by the inclination angle (i.e., in a direction for correcting the original document inclination) centering around a prescribed center point… Based on the positions of the feature points after the rotation (hereinafter, referred to as the rotated feature points) acquired through the feature point rotation calculating unit 81, the rectangular area calculating unit 82 preferably calculates a position and a size of a non-inclined rectangular area having an outline that is disposed in the vicinity of the rotated feature points; Par. [0182-200]: a process of rotating the original document target area 12 around the center point 13 by the inclination angle θ is performed as illustrated in FIG. 19… a rectangular area (extraction area 14) inclined by the same angle as the inclination angle of the original document can be acquired as illustrated in FIG. 19… the coordinates of three vertexes 14a, 14b, and 14c among the four vertexes of the rectangular extraction area 14 are transferred as parameters to the extraction rotation process unit 90… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated… the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired… Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image… Assuming that the first row and first column of the rotated image are the target pixels, FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image… FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image… In the example of FIG. 21, a rectangular image slightly rotated in the clockwise direction from the correct direction is acquired as an original document pixel area… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated (S703). In this process, firstly, the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired… Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image represented… Next, the pixel value "Q(m, n)" of the target pixel of the rotated image is acquired through two-dimensional linear interpolation. As illustrated in FIG. 24, the two-dimensional linear interpolation uses four pixels: the corresponding target pixel (i, j) of the original image; the pixel (i-1, j) arranged next to the corresponding target pixel in the x-direction; the pixel (i, j+1) arranged next to the corresponding target pixel in the y-direction; further comprising:
calculating at least one dimension of the document after rotating the image (e.g. image processing apparatus (i.e. user device) automatically detects a prescribed area, including a portion (i.e. a dimension, a portion, a section, a segment, a side, etc.) of an original document (i.e. the document), based on image data acquired, for example, by detecting feature points (i.e. pixels, coordinates, edges, etc.) related to an outline (i.e. boundary, area, shape, dimensions, size, etc.) of the original document, and calculating positions of rotated points (i.e. coordinates, dimensions, etc.) obtained by rotating and moving the detected feature points, for example, including calculated pixel values of target pixels of the rotated image, for example, and calculates a position and a size (i.e. dimensions) of a non-inclined (i.e. rotated, aligned, de-skewed, etc.) rectangular area having an outline that is disposed in the vicinity of rotated feature points of the image data acquired, for example, by performing two-dimensional interpolation process to acquire each pixel value of the target pixels of the rotated image, for example, including x-coordinates and y-coordinates (i.e. two dimensions) of each corresponding target pixel of the target pixels of the rotated image (i.e. calculating at least one dimension of the document after rotating the image), as indicated above), for example).
The same motivation to combine above-mentioned teachings applies, as previously indicated in claim 1.
Regarding claim 7, claim 1 is incorporated and the combination of Cali, MINAMINO and Chen, as a whole, teaches the method (Cali, Par. [0002]), wherein changing the orientation of the document relative to the one or more reference axes comprises:
rotating the image based on an angle of the orientation so that at least one dimension of the two dimensions of the document aligns with the one or more reference axes (MINAMINO, Par. [0088-96]: feature point rotation calculating unit 81 preferably inputs the value regarding the original document inclination acquired… and then calculates positions of rotated points obtained by rotating and moving the plurality of feature points… by the inclination angle (i.e., in a direction for correcting the original document inclination) centering around a prescribed center point… Based on the positions of the feature points after the rotation (hereinafter, referred to as the rotated feature points) acquired through the feature point rotation calculating unit 81, the rectangular area calculating unit 82 preferably calculates a position and a size of a non-inclined rectangular area having an outline that is disposed in the vicinity of the rotated feature points; Par. [0182-200]: a process of rotating the original document target area 12 around the center point 13 by the inclination angle θ is performed as illustrated in FIG. 19… a rectangular area (extraction area 14) inclined by the same angle as the inclination angle of the original document can be acquired as illustrated in FIG. 19… the coordinates of three vertexes 14a, 14b, and 14c among the four vertexes of the rectangular extraction area 14 are transferred as parameters to the extraction rotation process unit 90… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated… the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired… Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image… Assuming that the first row and first column of the rotated image are the target pixels, FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image… FIG. 23 illustrates the target pixels and the corresponding target pixels of the original image… In the example of FIG. 21, a rectangular image slightly rotated in the clockwise direction from the correct direction is acquired as an original document pixel area… the pixel value "Q(m, n)" of the target pixel (m, n) of the rotated image is calculated (S703). In this process, firstly, the position (i, j) of the corresponding target pixel of the original image is calculated. The x-coordinate "i" of the corresponding target pixel can be acquired… Similarly, the y-coordinate "j" of the corresponding target pixel can be acquired… FIG. 23 illustrates a correspondence between the target pixel of the rotated image and the corresponding target pixel of the original image represented… Next, the pixel value "Q(m, n)" of the target pixel of the rotated image is acquired through two-dimensional linear interpolation. As illustrated in FIG. 24, the two-dimensional linear interpolation uses four pixels: the corresponding target pixel (i, j) of the original image; the pixel (i-1, j) arranged next to the corresponding target pixel in the x-direction; the pixel (i, j+1) arranged next to the corresponding target pixel in the y-direction; wherein changing the orientation of the document relative to the one or more reference axes comprises:
rotating the image based on an angle of the orientation so that at least one dimension of the two dimensions of the document aligns with the one or more reference axes (e.g. image processing apparatus (i.e. user device) automatically detects a prescribed area, including a portion (i.e. a dimension, a portion, a section, a segment, a side, etc.) of an original document (i.e. the document), based on image data acquired, for example, by detecting feature points (i.e. pixels, coordinates, edges, etc.) related to an outline (i.e. boundary, area, shape, dimensions, size, etc.) of the original document, and calculating positions of rotated points (i.e. coordinates, dimensions, etc.) obtained by rotating and moving the detected feature points, for example, including calculated pixel values of target pixels of the rotated image, for example, and calculates a position and a size (i.e. dimensions) of a non-inclined (i.e. rotated, aligned, de-skewed, etc.) rectangular area having an outline that is disposed in the vicinity of rotated feature points of the image data acquired, for example, by performing two-dimensional interpolation process to acquire each pixel value of the target pixels of the rotated image, for example, including x-coordinates and y-coordinates (i.e. two dimensions) of each corresponding target pixel of the target pixels of the rotated image (i.e. wherein changing the orientation of the document relative to the one or more reference axes comprises: rotating the image based on an angle of the orientation so that at least one dimension of the two dimensions of the document aligns with the one or more reference axes), as indicated above), for example).
The same motivation to combine above-mentioned teachings applies, as previously indicated in claim 1.
Regarding claim 25, claim 1 is incorporated and the combination of Cali, MINAMINO and Chen, as a whole, teaches the method (Cali, Par. [0002]), further comprising:
receiving information indicating that data extracted from the image is accurate (Cali, Par. [0009-11]: computer-implemented method for extracting information from an image of a document comprising: acquiring an image comprising image data relating to at least a part of the document; manipulating the image data to obtain a document image by machine recognition… Manipulating the image data to obtain a document image… comprises applying a transform to the image data, or cropping the image data to a different size. This ensures that the method can accept image data that has not been acquired under ideal conditions. For example, it may use image data acquired using a camera that is skewed due the angle at which the camera was operated… The recognized data may comprise recognized textual data and recognized positional data… the resultant extracted data can then be independently confirmed by comparison with OCR results from other parts of the document. Therefore, it can provide results before other classification methods are complete. Additionally, some MRC segments can provide a large quantity of data and may have data integrity checking features, such as checksums, to increase confidence in the extracted results; Par. [0141]: additional methods may be performed to increase confidence in the extracted data reported by the identification authentication application; further comprising:
receiving information indicating that data extracted from the image is accurate (e.g. computer-implemented method to extract data from documents in an image of an official document (i.e. a document in an image) includes acquiring an image, comprising image data relating to at least a part (i.e. portion, section, segment, etc.) of the document, for example, includes resultant extracted data is independently confirmed by comparison with OCR results from other parts of the document to increase confidence (i.e. accuracy) in the extracted results (i.e. further comprising: receiving information indicating that data extracted from the image is accurate), as indicated above), for example).
Conclusion
Applicant’s amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GUILLERMO RIVERA-MARTINEZ whose telephone number is 571-272-4979. The examiner can normally be reached on Monday-Friday (8am - 5pm Eastern Time). If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GUILLERMO M RIVERA-MARTINEZ/ Primary Examiner, Art Unit 2677
Prosecution Insights