DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments regarding the rejection of claims 1-10 under 35 U.S.C. 112(b) as being indefinite have been fully considered and are persuasive in view of the amendments to the claims. Accordingly, the rejection has been withdrawn.
Applicant’s arguments regarding the rejections of the claims under 35 U.S.C. 103 have been fully considered, but are not persuasive. Some of Applicant’s arguments are directed to limitations that were in the original claims, but are no longer in the amended claims. Because these arguments are moot in view of the amendments to the claims, the examiner will not address them. All of the other arguments regarding the rejections under 35 U.S.C. 103 are addressed below.
Regarding the rejection of claim 2, Applicant argues that the examiner’s reliance on Wang as teaching not utilizing points based on their proximity to the blade contours is improper because Wang allegedly teaches not utilizing entire frames rather than not utilizing individual match points based on their proximity. Wang was relied on in the rejection for its teaching that it is undesirable to use points that are within a predetermined distance of the blade contours because they can comprise “redundant and irrelevant image data”. (Wang, para. [0081]). Wang discloses using a masking process that masks out points near the edges of the blades so that they are not used because “some of the image data in the image frames may depict subject matter that is not relevant to the inspection task, such as background environment behind the target object….” (Wang, para. [0017]; see also para. [0019] of Wang: “[f]urthermore, the embodiments of the inspection system and method may filter or mask regions of the key frames that are determined to be irrelevant for the inspection task. For example, the regions of the key frames that depict the background environment….”). Therefore, Wang does teach not utilizing points for the final data pairs that are within a predetermined distance of the blade contours.
Applicant also argues that Wang is nonanalogous art. A prior art reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). It should be note that the "same field of endeavor" and the "reasonably pertinent" tests are two separate tests for establishing analogous art; it is not necessary for a reference to fulfill both tests in order to qualify as analogous art. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004).
The instant invention and Wang are in the same field of endeavor because they are both directed to using image analysis to inspect blades of a turbine assembly, as indicated by the para. [0009] of the present specification and in para. [0005] of Wang. Furthermore, they both are directed to a solution that removes background areas around the blades from the image data via preprocessing prior to processing the image data to perform the inspection analysis, as indicated by para. [0012] of the present specification and paras. [0017] and [0019] of Wang. Therefore, the present invention and the invention of Wang are in the same field of endeavor.
Information Disclosure Statement
The information disclosure statements (IDSs) submitted on January 22, 2026 has been considered by the examiner and placed in the file.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation (BRI) using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The BRI of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
In the following, some of the terms in the claims have been given BRIs in light of the specification. These BRIs are used for purposes of searching for prior art and examining the claims, but cannot be incorporated into the claims. Should Applicant believe that different interpretations are appropriate, Applicant should point to the portions of the specification that clearly support a different interpretation.
The BRI of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
In the nonfinal Office Action, some of the claim elements were interpreted as invoking 35 U.S.C. 112(f). The claims have been amended to recite sufficient structure to perform the recited functions. Therefore, the claim elements are no longer interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3 are rejected under 35 U.S.C. 103 as being unpatentable over an article entitled “An image Stitching Method for Blades of Wind Turbine Based on Background Removal Preprocessing”, by Li, published in 2020 in 5th International Conference on Communication, Image and Signal Processing (CCISP) (2020, Page(s): 174-178) (hereinafter referred to as “Li”) in view of view of U.S. Publ. Appl. No. 2019/0304077 A1 to Wang et al. (hereinafter referred to as “Wang”) and further in view of an article entitled “An Image Registration Algorithm Based on Slope and Distance Ratio Constraints”, by Huang et al., published in 2019 in IEEE International Conference on Signal, Information and Data Processing (ICSIDP) (Page(s): 1-5) (hereinafter referred to as “Huang”).
Regarding claim 1, Li discloses an image stitching apparatus for inspecting a wind turbine (Abstract: “t[]he damage detection of the wind turbine (WT) blade is an important aspect of its health situation monitoring. In order to catch small damages, it is necessary to stitch the multi segment images of the blade to obtain a high resolution image”), comprising:
at least one processor configured to execute instructions stored on a storage medium to (presumably Li uses some type of processor executing software stored on some type of storage medium to perform the operations described in Li):
generate a preprocessed image pair by removing background areas around blades of a wind turbine from photographed images of the wind turbine captured by a drone (Abstract: “[i]In image stitching, a blurring of the image is carried out to reduce the influence of noise…To overcome these problems, a preprocessing method based on background removal is proposed”; Section II of Li discloses preprocessing to remove background images of the wind turbine before image stitching is performed; on page 174, under section I. Introduction, Li cites an article entitled "Autonomous Visual Inspection Of Large-Scale Infrastructures Using Aerial Robots", published in arXiv preprint arXiv:1901.05510 (2019) as teaching the photographs being preprocessed can be captured by a drone);
determine first initial match points in the preprocessed image pair by using a pre-trained deep learning module (section IIIB discusses performing initial point determination after preprocessing has been performed to match feature points; Li does not explicitly disclose using a pre-trained deep learning module for this purpose);
determine non-utilized match points out of the determined first initial match points by calculating value of position change for each of the first initial match points between the images of the preprocessed image pair, and identifying the non-utilized match points based on a statistical analysis of the calculated value of position change (the BRI for the term “non-utilized match points” is that it means initial match points that will not be used in the final set of match point pairs that will be used during image stitching; section IIIC of Li discusses performing a Random Sample Consensus (RANSAC) algorithm to determine "the final matching feature points" as valid match points by excluding match points determined to be outliers; Li does not explicitly disclose identifying the non-utilized match points based on a statistical analysis of the calculated value of position change); and
determine valid match points by excluding the determined non-utilized match points from the determined first initial match points (section IIIC, the match points that are excluded in Li are the outliers that cannot be fitted to the model); and
perform image stitching between a plurality of images (section IIIC: “[a]fter the transformation matrix is obtained, the size of the stitched image and the overlap of the two images can be solved algebraically. In the non overlapping part, the gray value of the new image can be obtained based on the transformation matrix and interpolation method; in the overlapping area of two images, the weighted smoothing algorithm is used to realize the fusion transition between the two images”; Fig. 2 shows the stitching results).
As indicated above, Li does not explicitly disclose that the initial match point determination uses a pre-trained deep learning module to determine the first initial match points. Wang, in the same field of endeavor, discloses using a pre-trained deep learning module to determine feature match points in images of rotor blades (paras. [0071]-[0073], one or more artificial neural networks 102, 304, 404, Figs. 1, 3, 5 and 8).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to use one or more of the artificial neural networks of Wang in the system of Li to determine the first initial match points. One of ordinary skill in the art would have been motivated to make the modification to benefit from the high accuracy and robustness that can be achieved using neural networks to perform feature extraction and matching. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods (programming one or more processors with suitable software to implement a neural network) to yield predictable results.
As indicated above, Li does not explicitly disclose the limitation of determining non-utilized match points by calculating the value of position change for each of the first initial match points between the images of the preprocessed image pair, and identifying the non-utilized match points based on a statistical analysis of the calculated value of position change.
Huang, in the same field of endeavor, discloses performing an “improved SIFT algorithm based on slope and distance ratio constraint (SDRC-SIFT)” that performs such a statistical analysis of the position change values. Section IIIC of Huang discloses that the SDRC-SIFT algorithm calculates the value of position change for each of the initial match points between the images of the preprocessed image pair by using equations 5 -7 to calculate distance ratios that constitute change in position values. The algorithm then performs a statistical histogram analysis that excludes matching pairs corresponding to points that are not in the highest bin of the histogram.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to further modify the system and methods of Li as modified by Wang based on the teachings of Huang to determine and exclude non-utilized match points based on the statistical histogram analysis of the position change values as taught by Huang. One of ordinary skill in the art would have been motivated to make the modification to improve the RANSAC method used in Li to use the additional distance ratio constraint of Huang to “improve the matching efficiency and enhance the stability of the algorithm”, as taught by Huang. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (programming one or more processors with suitable software to implement the modified RANSAC algorithm of Huang).
Regarding claim 2, Li does not explicitly disclose determining first initial match points located within a predetermined distance from edges of the blades in the preprocessed image pair as the non-utilized match points. As indicated above in the Response to Arguments section of this Action, Wang discloses this limitation. In particular, Wang discloses excluding match points that are within a predetermined distance from the edges of the blades by detecting the contour of the blade (Fig. 8, step 704, para. [0075]), identifying the subset of image frames that correspond to a reference blade pose (Fig. 8, steps 706, 708 and 710, paras. [0076]-[0078]) and masking out points that are less than a predetermined distance from the contour (Fig. 8, steps 714 and 716, paras. [0079]-[0080]). The width of the masked portion of the contour corresponds to a predetermine distance from the blade edges and points that are less than this distance from the edges of the blade are non-utilized match points because they are masked out and therefore not utilized.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to further modify the system and methods of Li as modified based on the teachings of Huang and Wang to determine and exclude non-utilized match points within a predetermined distance from the blade contours as taught by Wang. One of ordinary skill in the art would have been motivated to make the modification to improve the efficiency and accuracy of the method used in Li by excluding points near the edges as taught by Huang. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods (programming one or more processors with suitable software to identify contours and mask out regions near the blade edges) to yield predictable results.
Regarding claim 3, Li does not explicitly disclose calculating a slope value for each line connecting corresponding pairs of the first initial match points and determining the non-utilized match points based on the calculated slope values.
Huang discloses calculating the slope values of each line connecting each of the corresponding pairs of initial match points (Fig. 2, equation 3 gives the value of the slope, k), determining whether the slope value is less than or equal to a predetermined value, and if so, the point is treated as a non-utilized point that is excluded (section III.B. of Huang discloses that a histogram is constructed of the slope values and if the value H of the column of the histogram corresponding to point being considered is greater than 0.3 x H, the match point is used, which means that if the value is less than or equal to 0.3 x H, the match point is considered a non-utilized match point and is excluded).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to further modify the system and methods of Li as modified based on the teachings of Huang and Wang to compare the slope values corresponding to lines connecting the corresponding pairs of match points to a predetermined value and to exclude the match points as non-utilized match points if the slope values are less than or equal to a predetermined value as taught by Huang. One of ordinary skill in the art would have been motivated to make the modification to improve the RANSAC method used in Li to use slope values in this manner in addition to using the Euclidian distance values to identify and exclude non-utilized match points as taught by Huang. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (programming one or more processors with suitable software to implement the modified RANSAC algorithm of Huang).
Allowable Subject Matter
Claim 4 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 5-10 are allowed.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 4, none of the prior art calculating non-utilized match points by calculating the value of position change of the plurality of first initial match points in the preprocessed image pair and determining the non-utilized match points based on an average value of the calculated value of position change.
Regarding claim 5, none of the art of record teaches or suggests making the three determinations of the first to third non-utilized match points in the manner described in limitations (1)-(3) of claim 5 in the sequence recited in claim 5.
Regarding claims 6-10, these claims recite allowable subject matter due to their direct or indirect dependencies from claim 5.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL J SANTOS whose telephone number is (571)272-2867. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL J. SANTOS/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights