DETAILED ACTION
This action is in response to the remarks and amendment filed on April 17th, 2026. Claims 1-2 and 5-20 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s replaced figure 14 is sufficient to overcome the drawing objection, and it is therefore withdrawn.
Applicant’s amendment to claim 12 is sufficient to overcome the claim objection, and it is therefore withdrawn.
Applicant's arguments filed April 17th, 2026 have been fully considered but they are not persuasive.
Applicant alleges that “… even combining Ebayyeh, Badanes and/or Zhang would not teach, disclose or suggest an image analysis module which includes an object detection model, a golden sample analysis module, or a comparison module.” Examiner respectfully disagrees. Ebayyeh is previously cited for teaching an object detection model and a golden sample analysis module, as seen on pages 5-6 of the Non-Final office action dated November 17th, 2025. Furthermore, Badanes is cited to teach a comparison module which takes as input two defect detection modules and outputs a single combined defect detection result, as seen on pages 10-11 of the Non-Final office action dated November 17th, 2025. Therefore, when Ebayyeh and Badanes are considered in combination, they teach an image analysis module which includes an object detection model, a golden sample analysis module, or a comparison module. Therefore, the rejection is maintained.
Furthermore, Applicant alleges that “In other words, Ebayyeh in view of Badanes does not teach, disclose or suggest two distinct layers of comparison, where a first layer of comparison is a comparison of the masked inspection image to the golden sample image to generate golden sample output data, and a second layer of comparison is comparing location data of the defects or anomalies detected in the object detection output data and the golden sample output data. Moreover, neither Ebayyeh nor Badanes teach, disclose or suggest that a comparison module compares location data of the defects or anomalies detected in object detection output data and golden sample output data.” Examiner respectfully disagrees. Ebayyeh is cited to teach the first “layer” of comparison, where a golden template is compared against a target image to detect a defect and output defect detection data. Badanes is then cited to teach the second “layer” of comparison with the comparison module that compares the location data output of two distinct defect detection models. Furthermore, the two distinct defect detection models of Badanes both output locations of the defects in the form of a grade map, then those grade maps are compared and combined according to the locations of the defects. Therefore, the rejection is maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 and 5-20 are rejected under 35 U.S.C. 103 as being unpatentable over “A Review and Analysis of Automatic Optical Inspection and Quality Monitoring Methods in Electronics Industry” (herein after referred to by its primary author, Ebayyeh) in view of US20210209418 (herein after referred to by its primary author, Badanes) and US20170148226 (herein after referred to by its primary author, Zhang).
In regards to claim 1, Ebayyeh teaches a system for visual inspection of a target article using adaptive region of interest ("ROI") segmentation, the system comprising: a camera for acquiring an inspection image of the target article; an Al visual inspection computing device for detecting defects or anomalies in the target article, the Al visual inspection computing device comprising: a communication interface for receiving the inspection image acquired by the camera (Ebayyeh Abstract “Automatic optical inspection (AOI) is one of the non-destructive techniques used in quality inspection of various products. This technique is considered robust and can replace human inspectors who are subjected to dull and fatigue in performing inspection tasks. A fully automated optical inspection system consists of hardware and software setups…Hardware setups used in acquiring images are then discussed in terms of the camera and lighting source selection and configuration”); an adaptive ROI segmentation module for processing the inspection image using an ROI segmentation model to generate a masked inspection image in which regions not of interest ("nROls") are masked (Ebayyeh Figure 34; Page 183214 “In both studies it was found that the dark background of the image can affect the BDCT and can be confused with the defect features. Therefore, after specifying the ROI, an image mask is used to delineate the ROI.” Examiner note: In figure 34, regions of interest are highlighted in white, while non-regions of interest are masked and comprise black pixels. Furthermore, the cited paragraph shows that the areas outside of the ROI are masked, these areas are analogous to the nROIs of the current disclosure as they are areas not contained within the ROI.); an image analysis module for receiving the masked inspection image and analyzing the masked inspection image using an image analysis model (Ebayyeh Page 183221 “In AOI applications, template matching algorithm works by first identifying a reference template which usually represent the non-defected case (also known as golden template) that can be used for comparison. The selected template can be compared to the target samples using various kind of correlation functions.”) to generate output data indicating presence of the defects or anomalies detected by the image analysis model (Ebayyeh Page 183222 “The output of the image subtraction operation can be one of the following three cases: positive (potential defect), negative (potential defect), or zero (non-defective).”), wherein analysis of the masked inspection image is limited to non-masked ROIs (Examiner note: The masking portion of this reference is referred to as “Preprocessing” (See Page 183212 Section V A), so it is performed before the comparison analysis.); and an output interface for displaying the output data (Ebayyeh Page 183249 “These algorithms were integrated with a user interface system that allows users to execute the following system operations including: loading the analysed dataset, adding or retracting the decision knowledge, controlling the parameters in WBM patterns clustering system, and monitoring the clustering results.”); wherein the image analysis module includes: an object detection model configured for performing an object detection task on the masked inspection image, and generating object detection output data describing one or more objects identified in the masked inspection image (Ebayyeh Page 183235 “In terms of the nature of the output, classification can be subdivided into binary classification and multi-class classification. In binary classification, the outputs are categorized into two groups (e.g. pass/fail, defect/non-defect). In multi-class classification, the outputs are categorized into more than two groups”; Page 183237 “Wu et al. in [171] used Bayes classifier as binary first-stage classification approach to filter the defected and non-defected PCB samples before sending the non-defected results to an SVM multi-class classifier to specify the type of defect” Examiner note: The first excerpt shows that classifiers are split into 2 groups, binary and multi-class. The second excerpt shows that multi-class classifiers can identify types of defects, which is analogous to data describing one or more objects (defects) identified in the masked inspection image); a golden sample analysis module configured for Ebayyeh Page 183221 “In AOI applications, template matching algorithm works by first identifying a reference template which usually represent the non-defected case (also known as golden template) that can be used for comparison”).
Ebayyeh does not teach a golden sample analysis module configured for generating a golden sample image of the target article; and a comparison module configured for comparing the object detection output data generated by the object detection model with the golden sample output data generated by the golden sample analysis module, wherein comparing includes comparing location data of the defects or anomalies detected in the object detection output data and the golden sample output data.
However, Zhang teaches a golden sample analysis module configured for generating a golden sample image of the target article (Zhang Paragraph [0104] “In one embodiment, the imaging system is configured to acquire the one or more actual images and the one or more simulated images and to detect defects on the specimen by comparing the one or more actual images to the one or more simulated images. For example, the embodiments described herein can be used to generate a “golden” or “standard” reference to improve die-to-database defect detection algorithms for mask and wafer inspection and metrology.”).
Zhang is considered to be analogous to the claimed invention because they are both in the same field of defect detection. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Ebayyeh to include the teachings of Zhang, to provide the advantage of a computationally efficient method of creating a reference image (Zhang Paragraph [0028] “Some embodiments described herein include a deep generative model for realistic rendering of computer aided design (CAD) for applications such as semiconductor inspection and metrology. In addition, the embodiments described herein can provide a computationally efficient methodology for generating a realistic-looking image from associated CAD for tools such as electron beam and optical tools.”)
Furthermore, Badanes teaches a comparison module configured for comparing first object detection output data generated by a first defect detection module with second object detection output data generated by a second defect detection module (Badanes Figure 5 “Supervised model (303)” which outputs “First grade Map” and “Unsupervised model (304)” which output “Second grade Map”), wherein the comparing includes comparing location data of the defects or anomalies detected in the first and second object detection output data (Badanes Figure 5; Paragraph [0086] “FIG. 5 illustrates an example of a runtime defect detection process using optimized thresholds in accordance with certain embodiments of the presently disclosed subject matter. A runtime image 501 is received and processed by the trained supervised model 303 and the trained unsupervised model 304, giving rise to the first grade map and the second grade map respectively. The optimized threshold of the supervised model Th.sub.s, as obtained through the optimization stage as described above, is applied to the first grade map (see 502). Similarly, the optimized threshold of the unsupervised model Th.sub.us is applied to the second grade map (see 503). The thresholded outputs (e.g., defect maps) can be combined (e.g., through the logical operation OR 504) to obtain a defect detection result 505. By way of example, the pixel values in the first grade map that exceed the threshold Th.sub.s and the pixel values in the second grade map that exceed the threshold Th.sub.us are combined to be declared as defects.” Examiner note: This reference teaches combining two defect detection results to yield a final defect detection. In this reference, the supervised model (303) is analogous to the first defect detection module, and the unsupervised model (304) is analogous to the second defect detection module. The first and second grade maps are analogous to the first and second object detection output data. This reference, when considered in light of Ebayyeh in view of Zhang, teaches that the outputs of two defect detection modules, such as the object detection model and golden sample analysis module, could be compared (in the case of figure 5, with the OR operator) and the location of the corresponding defects in the combined grade map would be identified as the defect detection result).
Badanes is considered to be analogous to the claimed invention because they are both in the same field of defect detection. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Ebayyeh in view of Zhang to include the teachings of Badanes, to provide the advantage of improved defect detection results (Badanes Paragraph [0097] “Thus, as illustrated above, the proposed system, comprising two or more supervised and unsupervised models, as well as the combination and optimization thereof, is capable of detecting defects, which may or may not have been seen during training (the unsupervised model can be used as a safety net for detection of unseen anomalies), thereby providing improved defect detection results.”).
In regards to claim 2, Ebayyeh in view of Zhang and Badanes teaches the system of claim 1, wherein the object detection model is trained to detect at least one defect class in the masked inspection image (Ebayyeh Page 183235 “In terms of the nature of the output, classification can be subdivided into binary classification and multi-class classification. In binary classification, the outputs are categorized into two groups (e.g. pass/fail, defect/non-defect). In multi-class classification, the outputs are categorized into more than two groups”; Page 183237 “Wu et al. in [171] used Bayes classifier as binary first-stage classification approach to filter the defected and non-defected PCB samples before sending the non-defected results to an SVM multi-class classifier to specify the type of defect”).
In regards to claim 5, Ebayyeh in view of Zhang and Badanes teaches the system of claim 1, further comprising an annotation module configured to automatically assign metadata to the masked inspection image, the metadata including defect location information, defect size data, defect class information, or a defect confidence level (Ebayyeh Page 183237 “Wu et al. in [171] used Bayes classifier as binary first-stage classification approach to filter the defected and non-defected PCB samples before sending the non-defected results to an SVM multi-class classifier to specify the type of defect”); Page 183216 “The proposed approach determines the defect type through image analysis using various features, such as the geometric characteristics and the shape descriptor with intensity distribution. Various rule-based algorithms were used to classify the defects according to features extracted such as minimum bounding rectangle, actual defect region and region-based descriptor.” Examiner note: These excerpts show that Ebayyeh in view of Zhang and Badanes teaches detecting a defect and classifying at least its type/class and location. While this does not explicitly state that metadata would indicate the defect type/class and location, it can be appreciated that a bounding box on an image could be considered analogous to metadata as both the bounding box and the metadata are embedded data that provides data about the determined defect).
In regards to claim 6, Ebayyeh in view of Zhang and Badanes teaches the system of claim 1, wherein the golden sample module includes a generative model for generating the golden sample image from the inspection image (Zhang Paragraph [0104] “In one embodiment, the imaging system is configured to acquire the one or more actual images and the one or more simulated images and to detect defects on the specimen by comparing the one or more actual images to the one or more simulated images. For example, the embodiments described herein can be used to generate a “golden” or “standard” reference to improve die-to-database defect detection algorithms for mask and wafer inspection and metrology.”).
In regards to claim 7, Ebayyeh in view of Zhang and Badanes teaches the system of claim 2, wherein the output data includes a defect type and a defect location for each defect detected by the object detection model (Ebayyeh Page 183216 “The proposed approach determines the defect type through image analysis using various features, such as the geometric characteristics and the shape descriptor with intensity distribution. Various rule-based algorithms were used to classify the defects according to features extracted such as minimum bounding rectangle, actual defect region and region-based descriptor” Examiner note: This section teaches that defect types and a minimum bounding rectangle can be identified. The minimum bounding rectangle is analogous to the defect location, as they both describe where the defect is located within the image).
In regards to claim 8, Ebayyeh in view of Zhang and Badanes teaches the system of claim 1, wherein the adaptive ROI segmentation model is trained to identify and mask a non-uniform area of the inspection image (Ebayyeh Figure 34; Page 183214 “In both studies it was found that the dark background of the image can affect the BDCT and can be confused with the defect features. Therefore, after specifying the ROI, an image mask is used to delineate the ROI.” Examiner note: These images are masked in such a way that the white region can be any shape).
In regards to claim 9, Ebayyeh in view of Zhang and Badanes teaches the system of claim 8, wherein the non-uniform area comprises any one or more of an improperly illuminated area in the inspection image, a user-defined no-uniform area, a component of the target article that varies across different target articles of the same class, and an irregularly textured area of the target article (Ebayyeh Figure 34 “The brighter regions in b1–b6 are obtained by applying Otsu’s auto-thresholding [37]” Examiner note: This reference teaches identifying the non-uniform area based on the brightness of the region, this is analogous to an improperly illuminated areas, as they both are finding a non-uniform area based on a difference in brightness).
In regards to claim 10, Ebayyeh in view of Zhang and Badanes teaches the system of claim 2, wherein the output data classifies the target article as either defective or non-defective (Ebayyeh Page 183235 “In this stage the inspection algorithm uses the extracted features as an input in order to produce a an output of categorized classes. In terms of the nature of the output, classification can be subdivided into binary classification and multi-class classification. In binary classification, the outputs are categorized into two groups (e.g. pass/fail, defect/nondefect).”).
In regards to claim 11, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 1.
In regards to claim 12, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 5.
In regards to claim 13, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 7.
In regards to claim 14, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 8.
In regards to claim 15, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 9.
In regards to claim 16, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 1.
In regards to claim 17, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 5.
In regards to claim 18, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 7.
In regards to claim 19, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 8.
In regards to claim 20, Ebayyeh in view of Zhang and Badanes renders obvious the claim language as in the consideration of claim 9.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“Unsupervised fabric defect detection based on a deep convolutional generative adversarial network” teaches a method of detecting defects in fabric, where an input image is used by two models (see Hu Figure 2(b) D and E) and the output of the two models is combined to create a final fusion map for defect detection.
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CALEB LOGAN ESQUINO whose telephone number is (703)756-1462. The examiner can normally be reached M-Fr 8:00AM-4:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CALEB L ESQUINO/ Examiner, Art Unit 2677
/ANDREW W BEE/ Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights