DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/17/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3 and 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Lei et al. (WO 2023160986 A1) in view of Albright et al. US PG-Pub(US 20190258901 A1).
Regarding Claim 1, Lei teaches a method for training a second Machine Learning (ML) system to determine semiconductor wafer defects(¶[0079] discloses retraining a machine learning system with misclassified images to determine defects.), the method comprising: receiving a first set of semiconductor wafer images for which defect detection failed in a first ML system(¶0078], “The samples may be included in a first training pool for use in the first phase of a multi-phase classification process in the classification tree 800. When data set 802 is sorted by classifier 804, nuisances which may otherwise have been misclassified as a defect 808 or 810 may instead be classified as a particular nuisance type 806 or 807. For instance, the otherwise misclassified data may be classified as a particular nuisance type 807, while the remaining nuisances are classified as 806. It should be understood that the invention is not limited to two categories of defect review or nuisance review types.”, ¶[0078] discloses generating training data with images that were misclassified by the classifier.); generating a first dataset based on the received first set of images and corresponding prediction results of the first ML system([0077] “FIG. 7A shows an example case of a classification tree 700 with multi-phase training. Multiphase classification tree 700 includes an image data set 702, an ADC classifier 704, a nuisance bin 706, a hole missing defect bin 708, a hole bridge defect bin 710, and a revised training pool 712. Classifier 704 may be, e.g., a C4 type classifier. The values shown in boxes 706, 708 and 710 represent the final binning results after a series of classification phases. As discussed above with respect to FIG. 6, these results are achieved after a plurality of classification phases in which classifier 704 sorts raw data 702 using the most recent iteration of a revised training pool 712.”, ¶[0077] discloses training a classifier with a first data set and sorting images to compile into another training dataset.); identifying, using the first ML system, defects in the second set of images(¶[0051], “In some embodiments, a computer system may be provided that can identify defects in a wafer image and classify the defects into categories according to the defect type.”, discloses classifying the defects into categories based on type.); assigning ground truths to the second set of images based on the identified defects(¶[0077] discloses assigning labels to the different types of defects (i.e. hole missing, hole bridge defects.)); generating a second dataset based on the first dataset, the second set of images, and the ground truths associated therewith and training, based on the generated second dataset, the second ML system to determine semiconductor wafer defects. ([0079] “After the first classification phase, the results are compared to results of a manual review. Based on different pattern features and strength thresholds, some misclassified defects and misclassified nuisances from each type are re-labeled correctly and put into a revised training pool 812 for a second training in phase II. Multiple iterations of this process can be performed until the training result meets expectations, similar to the process discussed above with respect to FIG. 7.”, ¶[0079] discloses after training with the first dataset, images are re-labelled and put into a revised training data pool for a second training process with a classifier)
Lei does not explicitly teach modifying the images in the first dataset using predefined image adjustment parameters to generate a second set of images;
Albright teaches modifying the images in the first dataset using predefined image adjustment parameters to generate a second set of images ([0020] “As shown in FIG. 1, the computer system 130 includes a training data augmentation module 140 and a training module 150. The training data augmentation module 140 uses an initial training dataset 135 to generate an augmented training dataset 145. The initial training dataset 135 includes a set of initial images. An initial image 137 of the set shows that a gorilla is in a zoo with a green-centric scenery and the gorilla is located in the center of the initial image 137. The augmented training dataset 145 includes a set of modified images associated with the set of initial images. For example, the training data augmentation module 140 generates two modified images 147A and 147B based on the initial image 137.”, ¶[0020] discloses performing data augmentation to generate a second set of images and ¶[0035] further discloses the types of image transformations performed such as changing brightness of the image or color of the image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei with Albright in order to modify the images in the first dataset and generate a second set of images. One skilled in the art would have been motivated to modify Lei in this manner in order to training neural networks using augmented training datasets. (Albright, ¶[0002])
Regarding Claim 2, the combination of Lei and Albright teach the method of claim 1, where Albright further teaches wherein the predefined image adjustment parameters comprise an image enhancement parameter or an image degradation parameter. (¶[0038] “The parameter determination module 275 determines one or more transformation parameters based on several factors. Each transformation parameter determines an amount the transformation performed by the transformation module 260. Examples of the factors include information describing the client device 110 that captures the image (e.g., camera settings, camera type, etc.), information describing the target object captured in the image (e.g., brand, color, style, etc.), or information describing the surrounding environment of the target object (e.g., lighting conditions, indoor environment, outdoor environment, etc.), or historic data (e.g., past test images, past training datasets, etc.”, ¶[0038] discloses determining a parameter to enhance the original image to generate a modified image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei with Albright in order to modify an image by a predefined adjustment parameter. One skilled in the art would have been motivated to modify Lei in this manner in order to training neural networks using augmented training datasets. (Albright, ¶[0002])
Regarding Claim 3, the combination of Lei and Albright teach the method as claimed in claim 1, where Albright further teaches wherein the predefined image adjustment parameters comprise a noise adjustment parameter, a resolution adjustment parameter, or a blurriness adjustment parameter(¶[0038] discloses changing resolution or brightness of the image as a transformation parameter), and wherein modifying the images in the first dataset comprises: performing, on each of the images in the first set of images, at least one of addition of noise, removal of noise, resolution upscaling, resolution down-scaling, or an attribute change. (¶[0038], “Examples of the transformation parameters include parameters (e.g., upper and lower limits, resize ratio, etc.) associated with scaling the target object, parameters (e.g., upper and lower limits, an amount of a rotation in degrees, a rotation direction, etc.) associated with rotating the target object, parameters (e.g., upper and lower limits, translation value, translation vector, etc.) associated with translating the target object, parameters (e.g., upper and lower limits, a brightness value relative to an original brightness, etc.) associated with changing a brightness of the target object, parameters (e.g., upper and lower limits, a specific range of colors, a color value relative to an original color, etc.)”, discloses up-scaling, downscaling and adjusting brightness in the image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei with Albright in order to modify the first training data with certain parameters changed. One skilled in the art would have been motivated to modify Lei in this manner in order to training neural networks using augmented training datasets. (Albright, ¶[0002])
Regarding Claim 10, claim 10 is considered an apparatus claim substantially corresponding to claim 1. Please see the discussion of claim 1 above for a discussion of similar limitations. Furthermore, Lei teaches a system for training a second Machine Leaning (ML) system to determine defects in a semiconductor wafer(¶[0079] discloses retraining a machine learning system with misclassified images to determine defects.), the system comprising: one or more processors([0046] In some embodiments, image processing system 290 may include an image acquirer 292, a storage 294, and a controller 296. Image acquirer 292 may comprise one or more processors); a memory coupled with the one or more processors and storing instructions configured to cause the one or more processors(See, ¶[0046])
Regarding claim 11, it is substantially similar to claim 2 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 12, it is substantially similar to claim 3 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Claims 4, 7-8, 13 and 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lei et al. (WO 2023160986 A1) in view of Albright et al. US PG-Pub(US 20190258901 A1) in view of Gera et al. US PG-Pub(US 20210160466 A1).
Regarding Claim 4, while the combination of Lei and Albright teach the method of claim 1, they do not explicitly teach comprising: determining, by applying the second ML system to the second dataset, a set of image corrections.
Gera teaches determining, by applying the second ML system to the second dataset, a set of image corrections. (¶[0041] “In at least some implementations, the parameter model training processing pipeline begins with an input of training data 202. The training data 202 includes, for instance, the image set 110 of FIG. 1. The image set 110 includes a plurality of base images 204 along with a plurality of corrected images 206. A base image 204 is a captured image that has not been edited, enhanced, altered, ‘touched up’, or so forth. A corrected image 206 corresponds to the base image 204, and has been adjusted or enhanced by a colorist to achieve a visually pleasing version of the base image 204.”, ¶[0041] discloses using a machine learning model to correct images from a training dataset.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei and Albright with Gera in order to correct images using the machine learning system. One skilled in the art would have been motivated to modify Lei and Albright in this manner in order to train a parameter adjustment model through machine learning techniques that captures feature patterns and interactions. (Gera, Abstract)
Regarding Claim 7, the combination of Lei, Albright and Gera teach the method of claim 4, where Gera further teaches comprising: generating a set of image modification guidelines based on the set of image corrections(¶[0055] discloses a parameter adjustment model may be created based on a set of correct images.), using the predefined image adjustment parameters with respect to one or more characteristics of the images and assigned ground truths. ([0071] “The neural network learning process utilizes a machine learning model with a loss function that modifies functions or parameters used to train the parameter adjustment model by minimizing a loss between output values and ground truth values. In doing so, the machine learning model generates predicted parameter values for a base item of digital visual content and compares the predicted parameter values to a corresponding corrected item of digital visual content.”, discloses training the model using ground truth images and the corrected images.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei and Albright with Gera in order to correct images using the machine learning system. One skilled in the art would have been motivated to modify Lei and Albright in this manner in order to train a parameter adjustment model through machine learning techniques that captures feature patterns and interactions. (Gera, Abstract)
Regarding Claim 8, the combination of Lei, Albright and Gera teach the method of claim 7, where Lei further teaches comprising: determining, using the second ML system, semiconductor wafer defects in the first set of images as modified based on the generated set of image corrections. (¶[0051], “In some embodiments, a computer system may be provided that can identify defects in a wafer image and classify the defects into categories according to the defect type.”, discloses classifying the defects into categories based on type.);
Regarding claim 13, it is substantially similar to claim 4 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 16, it is substantially similar to claim 7 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 17, it is substantially similar to claim 8 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Claims 5-6 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Lei et al. (WO 2023160986 A1) in view of Albright et al. US PG-Pub(US 20190258901 A1) in view of Bar US PG-Pub(US 20220050061 A1).
Regarding Claim 5, while the combination of Lei and Albright teach the method of claim 1, they do not explicitly teach further comprising: training the second ML system based on a third set of semiconductor wafer images, wherein the third set of images are images for which the first ML system successfully detected semiconductor wafer defects.
Bar teaches training the second ML system based on a third set of semiconductor wafer images(¶[0090], “In some further cases, additionally or alternatively to the inspection features including the first features and/or the second features, the classifier can be retrained using additional third features, such as, e.g., tool features characterizing physical attributes of an inspection tool that captures the one or more inspection images. ”, discloses third features or images are used to retrain the classifier.), wherein the third set of images are images for which the first ML system successfully detected semiconductor wafer defects. ([0087], “Specifically, in step 202 the one or more inspection images received by the recipe optimization system 101 from the inspection tool 110 are in fact one or more inspection image patches corresponding to the list of defect candidates selected by the classifier, as described above. Each inspection image patch is extracted from the inspection image of the specimen (e.g., surrounding the location of each given defect candidate) and thus represents at least a portion of the specimen. The one or more inspection image patches are indicative of the selected defect candidates.”, discloses determining patches of defect candidates in the image using a classifier.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei and Albright with Bar in order to train the system using regions of successfully captured defects. One skilled in the art would have been motivated to modify Lei and Albright in this manner in order to train a layer-specific classifier so as to improve the performance. (Bar, ¶[0139])
Regarding Claim 6, the combination of Lei, Albright and Bar teach the method of claim 5, where Albright teaches modifying a set of images using predefined image adjustment parameters and generating a dataset including the modified set of images and corresponding ground truths; . ([0020] “As shown in FIG. 1, the computer system 130 includes a training data augmentation module 140 and a training module 150. The training data augmentation module 140 uses an initial training dataset 135 to generate an augmented training dataset 145. The initial training dataset 135 includes a set of initial images. An initial image 137 of the set shows that a gorilla is in a zoo with a green-centric scenery and the gorilla is located in the center of the initial image 137. The augmented training dataset 145 includes a set of modified images associated with the set of initial images. For example, the training data augmentation module 140 generates two modified images 147A and 147B based on the initial image 137.”, ¶[0020] discloses performing data augmentation to generate a set of images and ¶[0035] further discloses the types of image transformations performed such as changing brightness of the image or color of the image.) training the second ML system based on the generated dataset ([0021] “The training module 150 uses the generated augmented training dataset 145 to train neural network models.”, the generated images are used to train the machine learning model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei with Albright in order to modify the images to generate a new dataset used to train a machine learning system. One skilled in the art would have been motivated to modify Lei in this manner in order to training neural networks using augmented training datasets. (Albright, ¶[0002])
However, Lei and Albright do not explicitly teach using a third dataset to train the machine learning system.
Bar teaches using a third dataset to train the machine learning system. (¶[0090], “In some further cases, additionally or alternatively to the inspection features including the first features and/or the second features, the classifier can be retrained using additional third features, such as, e.g., tool features characterizing physical attributes of an inspection tool that captures the one or more inspection images. ”, discloses third features or images are used to retrain the classifier.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei and Albright with Bar in order to train the system using a third set of image data. One skilled in the art would have been motivated to modify Lei and Albright in this manner in order to train a layer-specific classifier so as to improve the performance. (Bar, ¶[0139])
Regarding claim 14, it is substantially similar to claim 5 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 15, it is substantially similar to claim 6 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Lei et al. (WO 2023160986 A1) in view of Albright et al. US PG-Pub(US 20190258901 A1) in view of Ishihashi US PG-Pub(US 20250286957 A1).
Regarding Claim 9, while the combination of Lei and Albright teach the method of claim 1, they do not explicitly teach comprising: performing root cause analysis to determine a reason the first ML system failed in defect detection, in the first set of images, based on a defect size threshold and a noise level threshold.
Ichihashi teaches performing root cause analysis to determine a reason the first ML system failed in defect detection, in the first set of images, based on a defect size threshold and a noise level threshold. (0109] “In step S6004, the CPU 223 detects blobs from the difference image created in step S6003, and acquires a size of each blob. In the processing in step S6004, it is possible to make analysis using an image processing method called blob analysis.”
[0110] “In step S6005, the CPU 223 determines whether the largest blob among the blobs obtained in step S6004 is smaller than the level determined in step S1006. If the largest blob is smaller than the level determined in step S1006 (YES in step S6005), the processing proceeds to step S6006. In step S6006, the CPU 223 determines that a result of defect detection is OK. In contrast, if the largest blob is larger than the level determined in step S1006 (NO in step S6005), the CPU 223 determines that a result of defect detection is a fail.”,¶[0109]-¶[0110] disclose determining if the defect detection is a failure by using a size threshold in the blob in the image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Lei and Albright with Ichihashi in order to determine the cause of why the defect detection failed. One skilled in the art would have been motivated to modify Lei and Albright in this manner in order to inspect image quality of printed products. (Ichihashi, ¶[0003])
Regarding Claim 18, it is substantially similar to claim 9 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAN D HOANG whose telephone number is (571)272-4344. The examiner can normally be reached Monday-Friday 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JOHN M VILLECCO can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAN HOANG/Primary Examiner, Art Unit 2661
Prosecution Insights