Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Application has new claims 25-26 and now pending claims 1-26.
Response to Arguments
Applicant’s arguments, see Remarks, filed 11/03/2025, with respect to claims 1-24 have been fully considered and are persuasive. The U.S.C. 101 rejection of claims 1-24 has been withdrawn.
Applicant’s arguments with respect to claim(s) 1, 9, and 11 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Therefore, the new references of Gonzales and YEHUDA, in combination with Bjorn, discloses the amended claims of 1, 9, and 11. This action is made FINAL.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 9, 11-12, 14, 18-21, and 25-26 are rejected under 35 U.S.C. 103 as being unpatentable over Daniel S. Gonzales US-20230095647-A1, hereinafter Gonzales, in view of YAVETS-CHEN YEHUDA WO-2010029549-A1, hereinafter YEHUDA, in further view of Bjorn Brauer US-20180157933-A1, hereinafter Bjorn.
As per claim 9, Gonzales discloses a system for detecting a defect, comprising:at least one processor (see Gonzales ¶30, wherein the system contains a processor); andnon-transitory computer-readable memory storing instructions thereon that, when executed by the at least one processor (see Gonzales ¶37, wherein the non-transitory computer-readable medium is disclosed), cause the at least one processor to:receive a first multi-channel image, the first multi-channel image being constructed of images of the target object, each image channel of the multi-channel image corresponding to one of the images (see Gonzales ¶34-35, wherein an image of a target object is captured within the predefined search space. The imaging assembly includes a photo-realistic imaging assembly camera, which is an RGB based camera. Wherein the imaging assembly camera captures/receives the RGB image which is a three channel image as disclosed in ¶64);generate another image using the first multi-channel image (see Gonzales ¶56 and FIG. 3A, wherein an anomaly heatmap is generated using the first image 302. This image is an RGB image as disclosed in ¶34);construct a second multi-channel image in which one of the image channels is the other image (see Gonzales ¶63-64 and FIG. 3B, wherein a second four-channel image, is generated using a concatenation of the first multi-channel image and the anomaly image);determine, using Artificial Intelligence, based on the second multi-channel image, whether the target object includes the defect (see Gonzales ¶65-67 and FIG. 3C, wherein the machine learning model generates the hybrid mask and then identifies localized anomalies on the target object. See also ¶72, wherein AI models are disclosed); andoutput, by the Artificial Intelligence, an indication as to whether the target object includes the defect (see Gonzales ¶68-70, wherein the anomalies, patterns, and labels are output in the hybrid output mask),wherein each channel of the first multi-channel image and each channel of the second-multi-channel image corresponds to a same region of the target object (see Gonzales ¶65, wherein the localized anomaly regions in the hybrid image, i.e., second image, correspond to the localized anomalies within the original input image).
However, while Gonzales discloses a first multi-channel image, it fails to explicitly disclose where YEHUDA teaches:receive a first multi-channel image, the first multi-channel image being constructed of images, the images having different imaging attributes from one another (see YEHUDA ¶83, wherein an image of a product surface is acquired. See more specifically ¶85-88 and FIG. 6B, wherein the RGB camera that acquires the image contains an illumination system that acquires each colored illumination, i.e., each image channel (red, green, and blue), at different divergence angles, i.e., different imaging attributes).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s system by using YEHUDA’s teaching by modifying the RGB images to have different illumination conditions in order to detect defects that may appear more visibly on different color channels.
However, while Gonzales, in combination with YEHUDA, discloses a target object, it fails to explicitly disclose where Bjorn teaches:images of a manufactured article (see Bjorn ¶88 and FIG. 5, wherein the wafer, i.e., manufactured article, image is received).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s, in combination with YEHUDA, system by using Bjorn’s teaching by including a manufactured article as the target object in order to further specialize the defect identification on manufactured articles, such as wafers.
As per claims 1 and 11, the rationale provided in claim 9 is incorporated herein. In addition, the method of claim 1 and the system of claim 11 correspond to the system of claim 9.
As per claim 2, Gonzales, in combination with YEHUDA and Bjorn, discloses the method of claim 1, wherein a second channel of the second multi-channel image includes a difference image (see Gonzales ¶64-66, wherein the hybrid mask output is disclosed, wherein the hybrid mask is the difference of the ground truth binary mask), and a third channel of the second multi-channel image includes a mask image (see Gonzales ¶66 and ¶70 and FIGS. 3C-3D, wherein the ground truth binary mask 332 is disclosed, wherein the truth masks correspond to the input multi-channel image).
As per claim 3, Gonzales, in combination with YEHUDA and Bjorn, discloses the method of claim 1, wherein the manufactured article includes at least one of a semiconductor chip, a light emitting diode (LED), or a solid-state battery (see Bjorn ¶43 wherein a semiconductor is disclosed).
As per claim 4, Gonzales, in combination with YEHUDA and Bjorn, discloses the method of claim 1, wherein the images of the manufactured article are generated using a metrology device that scans the manufactured article, such as a light camera, an acoustic camera, a spectrometer, or an electron microscope (see Bjorn ¶68, wherein metrology of specimens, such as wafers, i.e., the manufactured article, using electron microscopy is disclosed).
As per claim 5, Gonzales, in combination with YEHUDA and Bjorn, discloses the method of claim 1, wherein the different imaging attributes include different lighting conditions at the manufactured article and/or at a metrology tool when the images are taken (see Bjorn ¶70-73, wherein different light conditions at the wafer, i.e., manufactured article, for the image data is disclosed).
As per claim 6, Gonzales, in combination with YEHUDA and Bjorn, discloses the method of claim 1, wherein the Artificial Intelligence is performed using a neural network (see Bjorn ¶50, wherein a CNN is used for defect detection).
As per claim 7, Gonzales, in combination with YEHUDA and Bjorn, discloses the method of claim 1, wherein the different imaging attributes include different angles of a metrology tool with respect to the manufactured article when the images are taken (see Bjorn ¶70, wherein the different lighting angles is disclosed. See prior ¶67-68, wherein the metrology systems are disclosed).
As per claim 12, Gonzales, in combination with YEHUDA and Bjorn, discloses the system of claim 11, wherein the processor determines whether the manufactured article includes a defect by using a multi-channel neural network, wherein each channel of the multi-channel neural network includes a plurality of layers of neurons (see Bjorn ¶51-52, wherein the CNN is constructed with layers with a plurality of neurons. See further ¶77-78, wherein the multi-channels is disclosed), including an input layer and one or more hidden layers, wherein each neuron in each layer is connected by a connection to each neuron of a subsequent layer within each channel (see Bjorn ¶52, wherein the input layer, the subsequent layers, i.e., hidden layers, and the neurons which are connected between layers are disclosed), wherein each neuron in each layer is connected by the connection to each neuron of the subsequent layer within each of the channels (see Bjorn ¶52, wherein the CNN enforces connectivity pattern between neurons of adjacent layers), wherein the multi-channel neural network further includes an output neuron connected to each neuron of a previous layer for each channel (see Bjorn ¶54, wherein the neuron output for the stack of unique layers is disclosed), wherein each neuron of the multi-channel neural network is assigned a bias value, and wherein each connection of the multi-channel neural network is assigned a weight value (see Bjorn ¶57-58, wherein the neuron weight and bias values are disclosed).
As per claim 14, Gonzales, in combination with YEHUDA and Bjorn, discloses the system of claim 11, wherein the plurality of images includes at least a first image wherein light is directed at the manufactured article from a first direction, and a second image wherein light is directed at the manufactured article from a second direction different from the first direction (see Bjorn ¶72 and FIG. 3, wherein the light directed at the wafer can be configured in multiple angles, or directions for the image data acquisition).
As per claim 18, the rationale provided in claim 2 is incorporated herein. In addition, the system of claim 18 corresponds to the method of claim 2.
As per claim 19, Gonzales, in combination with YEHUDA and Bjorn, discloses a method of detecting a defect in a manufactured article, comprising:receiving a first image of the manufactured article (see Bjorn ¶88, wherein the wafer image is received);comparing the first image to a reference image of the manufactured article to generate form a difference image, wherein the reference image represents a manufactured article with no observable defects (see Bjorn ¶91, wherein the test image, i.e., first image, is differenced with the reference image in order to create the difference image. See also FIG. 2, wherein the reference location is different from a defect location, i.e., no defect in the reference location);comparing the difference image to the reference image to generate a mask image (see Bjorn ¶93, wherein the reference images and the difference images are packaged to create amplified images, i.e., mask images. See also ¶96);constructing a multi-channel image using the first image, the difference image, and the mask image, each image channel of the multi-channel image corresponding to one of the first image, the difference image, and the mask image (see Bjorn ¶90-91, wherein the difference, reference, and test images are received alongside the image capture parameters, i.e., multi-channel image data, and merged together to create the augmented images as disclosed in ¶96); anddetermining, by a processor, based on the multi-channel image, whether the manufactured article includes a defect (see Bjorn ¶47 and ¶50, wherein a CNN is used for defect detection. See also FIG. 5, wherein the input data for the CNN is performed after receiving and construction the image. See further ¶83 and ¶90); andoutputting, by the processor an indication as to whether the manufactured article includes the defect (see Bjorn ¶67-68, wherein a defect review output is disclosed. The defect review tool is used for wafer inspection, such as determining defects as discussed prior in ¶47-50. The defect review system, which is an image data acquisition subsystem, encompasses one or more processors as disclosed in ¶81).
As per claims 20-21, the rationale provided in claim 3 and claim 6 respectively is incorporated herein.
Claim 8, 10, 15, 17, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Gonzales, in combination with YEHUDA and Bjorn, in further view of Osamu Nakayama JP-2004191112-A, hereinafter Nakayama.
As per claim 8, Gonzales, in combination with YEHUDA and Bjorn, fails to explicitly disclose where Nakayama teaches:The method of claim 1, wherein the images are all of an identical region of the manufactured article (see Nakayama ¶50-51, wherein a singular unique region of the object is extracted).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s, in combination with YEHUDA and Bjorn, method by using Nakayama’s teaching by including all identical regions to the manufactured article images in order to maintain consistency amongst all the images for easier detection.
As per claim 10, the rationale provided in claim 8 is incorporated herein. In addition, the system of claim 10 corresponds to the method of claim 8.
As per claim 15, Gonzales, in combination with YEHUDA and Bjorn, fails to explicitly disclose where Nakayama teaches: The system of claim 14, wherein the plurality of images further includes a third image wherein light is directed at the manufactured article from a third direction different from the first direction and different from the second direction (see Nakayama ¶43 and FIGS. 5A-5D, wherein a plurality of inspection images are taken at different illumination angles), and a fourth image wherein light is directed at the manufactured article from a fourth direction different from the first direction and different from the second direction and different from the third direction (see Nakayama ¶43-46 and FIGS. 5A-5D, wherein more alternative angles of the illumination are disclosed for the images).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s, in combination with YEHUDA and Bjorn, system by using Nakayama’s teaching by including different illumination angles to the plurality of images in order to further include multiple lighting conditions so as to detect defects in a variety of conditions.
As per claim 17, Gonzales, in combination with YEHUDA and Bjorn, fails to explicitly disclose where Nakayama teaches: The system of claim 11, wherein the multiple images captured include at least a first image wherein light is directed at the manufactured article from a first illumination source type, and a second image wherein light is directed at the manufactured article from a second illumination source type (see Nakayama ¶16-17, wherein the different optical condition using a plurality of imaging devices is disclosed).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s, in combination with YEHUDA and Bjorn, system by using Nakayama’s teaching by including illumination source types to the multiple images in order to further include multiple lighting conditions so as to detect defects in a variety of conditions.
As per claim 23, the rationale provided in claim 8 is incorporated herein.
Claim 13 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Gonzales, in combination with YEHUDA and Bjorn, in further view of Hari Pathangi US-20200161081-A1, hereinafter Hari.
As per claim 13, Gonzales, in combination with YEHUDA and Bjorn, fails to explicitly disclose where Hari teaches:The system of claim 12, wherein the output neuron is configured to produce a numerical value representative of a probability of a presence of a defect in the manufactured article (see Hari ¶47, wherein the heat map of the defects in an image is represented with a probability index).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s, in combination with YEHUDA and Bjorn, system by using Hari’s teaching by including a numerical value of probability to the output neurons in order to more accurately determine the likelihood of a defect by acquiring a probabilistic value.
As per claim 22, Gonzales, in combination with YEHUDA and Bjorn, fails to explicitly disclose where Hari teaches:The method of claim 21, wherein the Artificial Intelligence is a neural network that is configured to produce a numerical value representative of a probability of a presence of a defect in the manufactured article (see Hari ¶92, wherein the processor is configured to represent heat map of probable defects in an image as a probability index. See further Hari ¶93, wherein the processor operates a neural network).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s, in combination with YEHUDA and Bjorn, method by using Hari’s teaching by including a numerical value of probability to the neural network in order to more accurately determine the likelihood of a defect by acquiring a probabilistic value.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Gonzales, in combination with YEHUDA and Bjorn, in further view of Paren Indravadan Shah US-20200160497-A1, hereinafter Shah.
As per claim 16, Gonzales, in combination with YEHUDA and Bjorn, fails to explicitly disclose where Shah teaches:The system of claim 11, wherein the multiple images captured include at least a first image of the manufactured article under a bright field condition, and a second image of the manufactured article under a dark field condition (see Shah ¶51-52, wherein an illumination and a dark-field illumination of the object are disclosed from a first and second light source).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s, in combination with YEHUDA and Bjorn, system by using Shah’s teaching by including a bright and dark field condition to the multiple images in order to further include multiple lighting conditions so as to detect defects in a variety of conditions.
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Gonzales, in combination with YEHUDA and Bjorn, in further view of Raghu Aniruddh US-10346969-B1, hereinafter Raghu.
As per claim 24, Gonzales, in combination with YEHUDA and Bjorn, fails to explicitly disclose where Raghu teaches:The method of claim 19, further comprising: inputting the multi-channel image to a neural network to generate a first output classification probability (see Raghu col. 19 lines 34-65 and FIG. 3B, wherein the image is input into the neural network and classifies the subject as damaged or undamaged based on probability. See also Raghu (100) and FIG. 10); processing an image of the manufactured article to generate a processed image (see Raghu FIG. 3A); extracting a feature from the processed image to generate an extracted feature (see Raghu col. 17 lines 26-67 and FIG. 3A, wherein extracted features are acquired from the images); classifying the extracted feature, including generating a second output classification probability (see Raghu col. 19 lines 24-33 and FIG. 3B, wherein a training set with the classification features as well as a validation set of the patches and labels, i.e., second output classification set, is disclosed); comparing the first output classification probability and the second output classification probability (see Raghu cols. 19-20 lines 34-67 and lines 1-10 and FIG. 3B, wherein the classification is compared to each corresponding label between the training and validation sets); and based on the comparing, accepting or rejecting a classification of a defect of the manufactured article (see Raghu cols. 26-27 lines 60-67 and 1-14 and FIG. 10, wherein the determine if the subject is damaged based on all the probabilities of the patches. See also Raghu col. 7 lines 16-48, wherein an acceptable tolerance is disclosed).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Gonzales’s, in combination with YEHUDA and Bjorn, method by using Raghu’s teaching by including a first and second output classification probabilities to the neural network in order to further verify the accuracy of training by using the joint probabilities checking against itself to guarantee a precise classification.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Bradley Obas Felix whose telephone number is (703)756-1314. The examiner can normally be reached M-F 8-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 5712728243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRADLEY O FELIX/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights