DETAILED ACTION
Allowance Subject Matter
Claims 6-7 and 9 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is an examiner’s statement of reasons for allowance:
Regarding claims 6-7, the prior art of record, alone or in combination, fails to fairly teach or suggest these limitations, including the concept of image processing method, comprising: performing key point detection on an image to-be-processed, to determine a plurality of key points in the image to-be-processed, wherein the plurality of key points are corner points in the image to-be-processed; determining at least one key point combination among the plurality of key points, wherein each of the at least one key point combination includes four key points which are used to define a quadrilateral object; in response to a key point combination selection instruction input by a user, determining a target key point combination in the at least one key point combination; and performing perspective distortion correction on a quadrilateral area defined by a quadrilateral object corresponding to the target key point combination on the image to-be- processed, to obtain a processed image, wherein determining the at least one key point combination among the plurality of key points includes: removing suspicious key points from the plurality of key points to obtain a plurality of reserved key points, wherein the reserved key points are key points of the plurality of key points other than the suspicious key points; and determining the at least one key point combination among the plurality of reserved key points, wherein removing the suspicious key points from the plurality of key points to obtain the plurality of reserved key points includes: using one key point of key points which are close to each other among the plurality of key points as a reserved key point, and removing other key points of the key points which are close to each other among the plurality of key points as the suspicious key points. wherein using one key point of the key points which are close to each other among the plurality of key points as a reserved key point, and removing other key points of the key points which are close to each other among the plurality of key points as the suspicious key points, include: using one key point with a largest variance of the key points which are close to each other among the plurality of key points as the reserved key point, and removing other key points of the key points which are close to each other among the plurality of key points as the suspicious key points.
Regarding claims 9, the prior art of record, alone or in combination, fails to fairly teach or suggest these limitations, including the concept of an image processing method, comprising: performing key point detection on an image to-be-processed, to determine a plurality of key points in the image to-be-processed, wherein the plurality of key points are corner points in the image to-be-processed; determining at least one key point combination among the plurality of key points, wherein each of the at least one key point combination includes four key points which are used to define a quadrilateral object; in response to a key point combination selection instruction input by a user, determining a target key point combination in the at least one key point combination; and performing perspective distortion correction on a quadrilateral area defined by a quadrilateral object corresponding to the target key point combination on the image to-be- processed, to obtain a processed image, method according to claim 1, wherein determining the at least one key point combination among the plurality of key points includes: removing suspicious key points from the plurality of key points to obtain a plurality of reserved key points, wherein the reserved key points are key points of the plurality of key points other than the suspicious key points; and determining the at least one key point combination among the plurality of reserved key points, method according to claim 4, wherein: determining the at least one key point combination among the plurality of reserved key points includes: determining N optimal key point combinations among the plurality of reserved key points, wherein the optimal key point combinations are key point combinations with the highest similarity to a rectangle and N>1; and determining the target key point combination among the at least one key point combination in response to the key point combination selection instruction input by the user, includes: determining the target key point combination among the N optimal key point combinations in response to the key point combination selection instruction input by the user, method according to claim 8, wherein determining the N optimal key point combinations among the plurality of reserved key points includes: calculating a similarity value simi between each key point combination of the plurality of reserved key points and the rectangle separately according to simi =
PNG
media_image1.png
110
330
media_image1.png
Greyscale
, wherein ϴi ε [0°,180°) is an angle of an i-th corner in a quadrilateral object corresponding to the key point combination and 1<i<4; and using N key point combinations with the largest similarity value simi to the rectangle as the optimal key point combinations
For example, Pribble teaches a system for detecting corners in an image of a document in order to select the borders of the document and perform perspective correction, see Abstract and col. 5, second paragraph from bottom. Col. 6, ¶ 5 and col. 12 ¶ 8 teach a selection determining the key points/corner points that make up the quadrilateral borders of the document. See process at Pribble Col. 6 last 3 paragraphs which teaches that for each corner of the multiple corner candidate points (points close to each other) choosing one candidate corner point as the corner point if it passes a validation test. Macciola teaches a technique for document corner detection for perspective correction. ¶ 0198 teaches performing key point combination selection instruction input by a user via user tapping or dragging corners. None of the closest prior art teaches or suggests all of the limitations of the above objected-to claims. The claim language goes beyond the similarities of these devices and Applicant’s invention and a combination could not reasonably be made without impermissible hindsight. The differences here are viewed as allowable over the prior art.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 8, 10 and 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pribble (US Pat. No. 10,331,966) in view of Macciola (US PGPub 2013/0182002)
Regarding claim 1, Pribble discloses an image processing method, comprising:
performing key point detection on an image to-be-processed, to determine a plurality of key points in the image to-be-processed, wherein the plurality of key points are corner points in the image to-be-processed; (Pribble teaches a system for detecting corners in an image of a document in order to select the borders of the document and perform perspective correction, see Abstract and col. 5, second paragraph from bottom.)
determining at least one key point combination among the plurality of key points, wherein each of the at least one key point combination includes four key points which are used to define a quadrilateral object; (Paragraph spanning cols. 5 and 6., col. 6, ¶ 4 and col. 12 ¶ 8, teach determining the key point/corner point candidate combinations that make up the quadrilateral borders of the document.)
determining a target key point combination in the at least one key point combination; and (Col. 6, ¶ 5 and col. 12 ¶ 8 teach a selection determining the key points/corner points that make up the quadrilateral borders of the document.)
performing perspective distortion correction on a quadrilateral area defined by a quadrilateral object corresponding to the target key point combination on the image to-be- processed, to obtain a processed image. (Col. 7, last paragraph)
In the field of document corner detection Macciola teaches that in response to a key point combination selection instruction input by a user, determining a target key point combination in the at least one key point combination; and (Macciola teaches a technique for document corner detection for perspective correction. ¶ 0198 teaches performing key point combination selection instruction input by a user via user tapping or dragging corners on a device.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Pribble’s document corner detection with Macciola’s document corner detection. Both references automated printed document corner detection for document perspective correction. Macciola expressly teaches a user selection interaction for inputting the corner/key point combination. The combination constitutes the repeatable and predictable result of simply applying Macciola’s technique here in the way in which it was intended to be used. This cannot be considered a non-obvious improvement in view of the relevant prior art here. Using known engineering design, no “fundamental” operating principle of the teachings are changed; they continue to perform the same functions as originally taught prior to being combined.
Regarding claim 2, the above combination discloses the method according to claim 1, wherein performing the key point detection on the image to-be-processed to determine the plurality of key points in the image to-be-processed includes: performing edge detection on the image to-be-processed to obtain an edge detection image corresponding to the image to-be-processed; and performing corner point detection on the edge detection image to obtain the plurality of key points in the image to-be-processed. (See Pribble col. 4, bottom three paragraphs.)
Regarding claim 3, the above combination discloses the method according to claim 2, wherein performing the edge detection on the image to-be-processed to obtain the edge detection image corresponding to the image to-be-processed includes: performing image enhancement processing on the image to-be-processed to obtain an enhanced image corresponding to the image to-be-processed; and performing edge detection on the enhanced image to obtain the edge detected image corresponding to the image to-be-processed. (See Prrible’s image enhancement processing prior to edge detection at Fig. 1B and col. 3, second paragraph from bottom.)
Regarding claim 4, the above combination discloses the method according to claim 1, wherein determining the at least one key point combination among the plurality of key points includes: removing suspicious key points from the plurality of key points to obtain a plurality of reserved key points, wherein the reserved key points are key points of the plurality of key points other than the suspicious key points; and determining the at least one key point combination among the plurality of reserved key points. (Pribble Col. 6, ¶ 5 and col. 5, last 2 paragraphs.)
Regarding claim 5, the above combination discloses the method according to claim 4, wherein removing the suspicious key points from the plurality of key points to obtain the plurality of reserved key points includes: using one key point of key points which are close to each other among the plurality of key points as a reserved key point, and removing other key points of the key points which are close to each other among the plurality of key points as the suspicious key points. (See process at Pribble Col. 6 last 3 paragraphs which teaches that for each corner of the multiple corner candidate points (points close to each other) choosing one candidate corner point as the corner point if it passes a validation test.)
Regarding claim 8, the above combination discloses the method according to claim 4, wherein: determining the at least one key point combination among the plurality of reserved key points includes: determining N optimal key point combinations among the plurality of reserved key points, wherein the optimal key point combinations are key point combinations with the highest similarity to a rectangle and N>1; and determining the target key point combination among the at least one key point combination in response to the key point combination selection instruction input by the user, includes: determining the target key point combination among the N optimal key point combinations in response to the key point combination selection instruction input by the user. (See process at Pribble Col. 6 last 3 paragraphs which teaches that for each corner of the multiple corner candidate points (points close to each other) choosing one candidate corner point as the corner point if it is most similar to a rectangle corner. Also see combination in rejection of claim 1.)
Regarding claim 10, the above combination discloses the method according to claim 8, further comprising: displaying perspective graphics of quadrilateral objects corresponding to the optimal key point combinations in a display interface; wherein determining the target key point combination among the optimal key point combinations in response to the key point combination selection instruction input by the user, includes: in response to the key point combination selection instruction triggered by the user in the perspective graphics of the quadrilateral objects, determining the target key point combination among the N optimal key point combinations. (Macciola teaches a technique for document corner detection for perspective correction. ¶ 0198 teaches performing key point combination selection instruction input by a user via user tapping or dragging corners. See graphics at Figs. 5a and 5b.)
Regarding claim 16, the above combination discloses the method according to claim 1, further comprising: determining a default key point combination among the plurality of key points, wherein the default key point combination includes four default key points and the four default key points are used to define a quadrilateral object; and performing perspective distortion correction on the quadrilateral area defined by the quadrilateral object corresponding to the default key point combination on the image to-be- processed, to obtain the processed image. (See process at Pribble Col. 6 last 3 paragraphs which teaches that for each corner of the multiple corner candidate points (points close to each other) choosing one candidate corner point as the corner point if it passes a validation test, including a default corner point closest to the rectangle corner which is selected first.)
Regarding claim 17, the above combination discloses the method according to claim 16, wherein determining the default key point combination among the plurality of key points includes: calculating a distance between each key point in the plurality of key points and each edge vertex of four edge vertices of the image to-be-processed respectively; and use one key point closest to each edge vertex as one default key point corresponding to the edge vertex, to obtain the default key point combination comprising the four default key points. (See process at Pribble Col. 6 last 3 paragraphs which teaches that for each corner of the multiple corner candidate points (points close to each other) choosing one candidate corner point as the corner point if it passes a validation test of being closest to rectangle corner/vertex.)
Regarding claim 18, the above combination discloses the method according to claim 17, wherein determining the default key point combination among the plurality of key points further includes: when one edge vertex in the four edge vertices does not have a corresponding default key point, configuring a recommended key point for the edge vertex; and using the recommended key point as the default key point of the edge vertex. (See process at Pribble Col. 6 last 3 paragraphs which teaches that for each corner of the multiple corner candidate points (points close to each other) choosing one candidate corner point as the corner point if it passes a validation test, including using a recommended key point if there is no default key point which passes the test.)
Regarding claim 19, the above combination discloses the image processing method, comprising: displaying perspective graphics of quadrilateral objects corresponding to N optimal key point combinations in a display interface, wherein the optimal key point combinations are key point combinations with a highest similarity to a rectangle and N>1; and in response to a key point combination selection instruction triggered by a user in the perspective graphics of the quadrilateral objects, determining a target key point combination among the N optimal key point combinations. (See rejection of claim 1 and process at Pribble Col. 6 last 3 paragraphs which teaches that for each corner of the multiple corner candidate points (points close to each other) choosing one candidate corner point as the corner point if it is most similar to a rectangle corner. Also see combination in rejection of claim 1.)
Claims 20 is the computer readable medium claim corresponding to the method claim 1. Pribble, col. 11, ¶ 1 teaches a computer readable medium. Remaining limitations are rejected similarly. See detailed analysis above.
Claim(s) 11-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pribble (US Pat. No. 10,331,966) in view of Macciola (US PGPub 2013/0182002) and Segalovitz (US PGPub 2017/0083762).
Regarding claim 11, the above combination discloses the method according to claim 10, wherein displaying the perspective graphics of the quadrilateral objects corresponding to the optimal key point combinations in a display interface, but not the remaining limitations.
In the field of document corner detection Segalovitz discloses displaying the image to-be-processed and thumbnails of the perspective graphics of the quadrilateral objects corresponding to the optimal key point combinations in the display interface, wherein the image to-be-processed and the thumbnails do not overlap in the display interface. (Segalovitz is a system for print document extraction and corner detection. Figs. 8b-c and Figs. 18a-e show document corner detection and thumbnails of the graphics of the quadrilateral objects corresponding to the optimal key point combinations in the display interface. Extracted thumbnails for example at Fig. 18c do not overlap in the display interface. See also ¶ 0268)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the above combination’s document corner detection with Segalovitz’s document corner detection and graphical display. Macciola teaches a graphical user interface and selection interaction for inputting the corners but does not teach a thumbnail display. The combination constitutes the repeatable and predictable result of simply applying Segalovitz’s technique of using a thumbnail display for a collection of candidate images, a well-known and widely-used common graphical user element. This cannot be considered a non-obvious improvement in view of the relevant prior art here. Using known engineering design, no “fundamental” operating principle of the teachings are changed; they continue to perform the same functions as originally taught prior to being combined.
Regarding claim 12, the above combination discloses the method according to claim 11, further comprising: in response to a thumbnail switching instruction triggered by the user, switching some or all of the thumbnails displayed in the display interface to thumbnails of perspective graphics of quadrilateral objects corresponding to other optimal key point combinations. (Segalovitz Figs. 18b, c, e, g and i teach thumbnail switching in a scroll swipe function which allows the mobile device user to navigate the screen to different thumbnails.)
Regarding claim 13, the above combination discloses the method according to claim 10, wherein displaying the perspective graphics of the quadrilateral objects corresponding to the optimal key point combinations in the display interface includes: displaying the image to-be-processed and trigger controls corresponding to the optimal key point combinations in the display interface, and displaying a target perspective graphic on the image to-be-processed, wherein: the target perspective graphic is a perspective graphic of a quadrilateral object of one optimal key point combination corresponding to one trigger control in a selected state; and the image to-be-processed and the trigger controls do not overlap in the display interface. (As above, Segalovitz Figs. 18b, c, e, g and i teach thumbnail switching in a scroll swipe function which allows the mobile device user to navigate the screen to different thumbnails. See also ¶ 0268 in which the thumbnail images correspond to trigger controls/buttons which may be selected for approving the quadrilateral object boundary detection/cropping)
Regarding claim 14, the above combination discloses the method according to claim 13, further comprising: in response to a perspective graphic switching instruction triggered by the user, switching the trigger control in the selected state, to switch the perspective graphic displayed on the image to-be-processed. (Segalovitz Figs. 18b, c, e, g and i teaches graphic switching user interface elements.)
Regarding claim 15, the above combination discloses the method according to claim 13, further comprising: in response to a trigger control switching instruction triggered by the user, switching some or all of the trigger controls displayed in the display interface to other trigger controls corresponding to other optimal key point combinations. (As above, Segalovitz Figs. 18b, c, e, g and i teach thumbnail switching in a scroll swipe function which allows the mobile device user to navigate the screen to different thumbnails. See also ¶ 0268 in which the thumbnail images correspond to trigger controls/buttons which may be selected for approving the quadrilateral object boundary detection/cropping and are switched as the user navigates the GUI.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Raphael Schwartz whose telephone number is (571)270-3822. The examiner can normally be reached Monday to Friday 9am-5pm CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571) 272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAPHAEL SCHWARTZ/ Examiner, Art Unit 2671