Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “25” has been used to designate both “DOI shaping module” in FIG. 1 and “ML Training module” in FIG. 2. The specifications refer to “ML Training module” as reference number “30”, so FIG. 2 more than likely should be updated to reflect that reference number so that it matches the specifications.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description:
a. FIG. 3, reference number “310” is not mentioned within the specifications.
b. FIG. 4, reference number “407” is not mentioned within the specifications (see Specifications section below for potential fix)
c. FIG. 4, reference number “411” is not mentioned within the specifications
d. FIG. 5, reference number “507” is not mentioned within the specifications (See Specifications section below for potential fix)
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
Page 14, Paragraph 46, Line 4, the word “incudes” should be “includes”
Page 20, Paragraph 68, Line 6, the word “the” should be removed before “Notably”, so that it then reads as “... max value in the shape.
Page 23, Paragraph 85, Line 2, the wrong reference numbers are used for the step of “Applying augmentation on the final DOI shape” and “DOI augmentation module”. According to FIG. 1 and FIG. 5 of the drawings, they should be “507” and “27” respectively, and not “505” and “29” and should then read as, “Augmentation may be applied on the final DOI shape to further increase diversity (507; e.g., by DOI augmentation module 27).”
Page 24, Paragraph 86, Line 5, the wrong reference numbers are used for the step of “Planting DOI in image” and “DOI planting module”. According to FIG.1 and FIG. 4 of the drawings, they should be “407” and “29” respectively, and not “409” and “31” and should then read as, “...thus creating a respective synthetic fault image (407; e.g., by DOI planting module 29).”
Page 26, Paragraph 94, Lines 7 and 8, the wrong reference number is used for “Examination Tools”. According to FIG. 2 of the drawings, it should be “220” and not “200” and should then read as, “... one or more examination tools 220, thereby facilitating and enhancing the functionalities of the examination tools 220 in examination-related processes.”
Appropriate correction is required.
Claim Objections
Claims 1, 11 and 19 are objected to because of the following informalities:
Claim 1, Page 1, Line 14, there should be an “a” before “secondary DOI shape” so that it reads as, “and a secondary DOI shape, ...” otherwise, “secondary DOI shape” is lacking antecedent basis
Claim 11, Page 3, Line 15, there should be an “a” before “secondary DOI shape” so that it reads as, “and a secondary DOI shape, ...” otherwise, “secondary DOI shape” is lacking antecedent basis
Claim 19, Page 6, Line 6, there should be an “a” before “secondary DOI shape” so that it reads as, “and a secondary DOI shape, ...” otherwise, “secondary DOI shape” is lacking antecedent basis
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 9-15 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (Pub. No.: US 2025/0117925 A1), hereinafter Zhang, in view of Yen (Pub. No.: US 2022/0005170 A1).
Regarding claim 1, Zhang discloses a computer-implemented method of generating synthetic fault images of a semiconductor specimen, wherein a synthetic fault image comprises at least one synthetic defect of interest (DOI) (Paragraph 30 teaches that in general, the embodiments described herein are configured for determining information for a specimen via defect synthesis and/or defect detection. Defect synthesis is generally defined herein as a process to simulate a defect of interest (DOI) in semiconductor applications for understanding the process condition, process window, and enabling yield improvement for semiconductor fabrication. ... In addition, the defect detection and defect synthesis methods and systems described herein are all constructed based on a DefectGPT model with different training and inference methods.), the method comprising:
obtaining examination output images of a semiconductor specimen generated by an examination tool (Paragraph 64 teaches that the training dataset may include only target semiconductor images (i.e., the images specific to the specimen) or the target semiconductor images and optional out-of-domain images (i.e., the images that are unrelated to determining the information for the specimen). In one such example, when the application is scanning electron microscope (SEM) defect detection, the target semiconductor images are SEM images, and the optional out-of-domain images may include other semiconductor images, nature images, or other application domain images. Additionally, paragraph 76 teaches that the inspection optical parameters may include optical conditions of the inspection tool. For example, on some inspection tools, these parameters may include the optical wavelength band, focus condition, aperture, etc.);
generating, from a plurality of examination output images, a respective plurality of synthetic fault images (Paragraph 56 teaches that the inspection subsystem may be configured to generate output, e.g., images, of the specimen with multiple modes. In general, a “mode” is defined by the values of parameters of the inspection subsystem used for generating images of a specimen (or the output used to generate images of the specimen). Therefore, modes may be different in the values for at least one of the parameters of the inspection subsystem (other than position on the specimen at which the output is generated). For example, the modes may be different in any one or more alterable parameters (e.g., illumination polarization(s), angle(s), wavelength(s), etc., detection polarization(s), angle(s), wavelength(s), etc.) of the inspection subsystem. The inspection subsystem may be configured to scan the specimen with the different modes in the same scan or different scans, e.g., depending on the capability of using multiple modes to scan the specimen at the same time.), comprising:
for each examination output image, determining at least one synthetic DOI (Paragraph 73 teaches that although the embodiments are described herein with respect to one prompt DOI query embedding, the embodiments may generate multiple prompt DOI query embeddings (one for each of multiple, different DOIs) and compare any one visual token embedding to the multiple prompt DOI query embeddings to see if any one of the DOIs are present in an image.), comprising:
determining a DOI planting location in the image (Paragraph 70 teaches that the computer subsystem may also generate other inspection results from the detection map such as defect list 508, which may include any information for the locations on the specimen or in the images at which a defect has been detected via the similarity measure.). However, Zhang fails to disclose determining a planting strength range.
Yen discloses determining a planting strength range (Paragraph 29 teaches that it should be noted that referring to FIG. 4A, during the process of comparing the reference image 300 with the reference image, the wafer inspection equipment 120 may further indicate a position of the gray level value difference in the range of the image processing range 310. Moreover, the wafer inspection equipment 120 may also report the position information of the image processing range 310 with the gray level value difference to the electronic apparatus 110. Therefore, in step S220, the processor 111 determines a position and a range of a defect of interest (DOI) image 510 according to the position information reported by the wafer inspection equipment 120, so as to capture the DOI image 510 in the second image 500. A pixel size of the DOI image 510 is less than a pixel size of the second image 500.). Since Zhang teaches the initial method steps for an inspection system to be used for determining defects of interest, along with data and information related to the generation of a synthetic DOI and Yen teaches an inspection system with the capabilities to analyze and compare the range of pixel values (strength range) of a DOI in accordance with its position, it would have been obvious to a person having ordinary skill in the art to combine the functions together so that while determining a DOI’s location, the strength or range of the pixel values of the DOI could be accounted for to help more accurately determine a DOI’s specific location and shape.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Yen, so that by incorporating a DOI’s range of pixel values, a complex DOI’s location and overall shape could be discovered with more accurate precession.
Furthermore, Zhang in view of Yen disclose determining a final DOI shape by fusing a primary DOI shape that is based on optical configuration of the examination tool and secondary DOI shape, selected from a collection of predefined secondary shapes, each comprising a plurality of pixels (Paragraph 77 of Zhang teaches that the fusion that is performed in step 604 may be performed as follows and in similar ways in other networks described herein (albeit with different inputs). DOI location hints 600 may be a spatial matrix with size (Height, Width, 1), and each of the elements has a value of 0 or 1, with 0 representing non-DOI locations and 1 representing DOI locations. Optical parameters in DOI descriptors 602 is a spatial matrix with the same H and W, but possibly with a different number of channels (i.e., size of (H, W, C)). The C value represents the number of optical parameters. Additionally, paragraph 31 of Yen teaches that referring to FIG. 6B, FIG. 6B is a diagram of a gray level distribution of the second image according to the embodiment of FIG. 5A. In step S240, the processor 111 analyzes a plurality of pixels of the second image 500 according to the minimum gray level value (the gray level value=90) 621 to obtain a number of DOI pixels and a number of non-DOI pixels.);
for each pixel in the final DOI shape, determining a respective DOI pixel strength based on a respective current pixel value and a planting strength selected from within the planting strength range (Paragraph 30 of Yen teaches that in step S230, the processor 111 analyzes a plurality of pixels of the DOI image 510 to obtain a minimum gray level value of the pixels. In the embodiment, the processor 111 may, for example, execute the image analysis module 122_2 to analyze the DOI image 510 and obtain a pixel value of each pixel in the DOI image 510. Referring to FIG. 6A, FIG. 6A is a diagram of a gray level distribution of the DOI image according to the embodiment of FIG. 5A. For example, the pixel size of the DOI image 510 is 3×3, so that the processor 111 may obtain nine gray level values of nine pixels of the DOI image 510. As shown in a gray level distribution 620 of FIG. 6A, the processor 111 may obtain a minimum gray level value 621 of the DOI image 510 as 90 (as indicated by dotted line in FIG. 6A).);
and planting the synthetic DOI in the planting location in the image (Paragraph 75 of Zhang teaches that as shown in FIG. 6, in some embodiments, the computer subsystem is configured for generating fused DOI features 604 from DOI location hints 600 and DOI descriptors 602, the one or more inputs to pre-trained DefectGPT encoder 606 include the fused DOI features, the determined information includes DOI embedding 608, and the one or more components include decoder 610 configured for generating defect signal images 612 from the DOI embedding. In this manner, inputs including DOI location hints and DOI descriptors (inspection optical parameters, process-related parameters, metrology parameters, etc.) are fused into the DOI features.);
thereby generating a collection of synthetic fault images of the semiconductor specimen, wherein different synthetic fault images in the collection comprise different synthetic DOIs, which differentiate in one or more of planting location, DOI pixel strength, and final DOI shape (Paragraph 74 of Zhang teaches that in general, the defect synthesis embodiments presented herein can be constructed through one or more major steps described herein and shown in FIGS. 6-10 and named as Networks A, B, C1, C2, and D, respectively. All the encoders in Networks A, B, C1, C2, and D use a pre-trained DefectGPT encoder. In addition, in these embodiments, the one or more components include a decoder configured for generating synthesized defect information from the information determined for the specimen by the pre-trained DefectGPT encoder. The decoders may be configured as described further herein. Additionally, paragraph 81 of Zhang teaches that an optimized construction using diffusion process/models is shown in FIG. 10 and named Network D. Network D is configured for generating simulated images with defects. In this embodiment, the one or more inputs include design images 1010 for the specimen, and the one or more components include a first diffusion process model 1016 configured for generating design embedding 1020 from the information determined for the specimen by the pre-trained DefectGPT encoder 1012. In this manner, design images are processed by pre-trained DefectGPT encoder 1012.).
Regarding claim 2, Zhang in view of Yen disclose everything claimed as applied above (see claim 1), in addition, Zhang in view of Yen disclose scanning a semiconductor specimen using an examination tool and generating the examination output images (Paragraph 39 of Zhang teaches that the inspection subsystem may also include a scanning subsystem configured to change the position on the specimen to which the light is directed and from which the light is detected and possibly to cause the light to be scanned over the specimen. For example, the inspection subsystem may include stage 22 on which specimen 14 is disposed during inspection. The scanning subsystem may include any suitable mechanical and/or robotic assembly (that includes stage 22) that can be configured to move the specimen such that the light can be directed to and detected from different positions on the specimen.).
Regarding claim 3, Zhang in view of Yen disclose everything claimed as applied above (see claim 2), in addition, Zhang in view of Yen disclose following generation of the plurality of synthetic fault images:
using the plurality of synthetic fault images for determining whether examination output images comprise any DOIs (Paragraph 80 of Zhang teaches that Network C2 is also configured to generate simulated images with defects. As shown in FIG. 9, in one embodiment, the determined information includes DOI embedding 900 (from Network A), pattern embedding 902 (from Network B), and combined embedding 904 generated from the DOI embedding and the pattern embedding, and the one or more components include a decoder 906 configured for generating simulated images with defects 908 on the specimen from the combined embedding.).
Regarding claim 4, Zhang in view of Yen disclose everything claimed as applied above (see claim 3), in addition, Zhang in view of Yen disclose using the plurality of synthetic fault images for creating a training dataset (Paragraph 88 of Zhang teaches that the training data inputs for Network C1 include defect signals generated by Network A and simulated images without defects generated by Network B. The ground truth for Network C1 may either be images with defects generated by a physical model or real optical images with defects. Additionally, paragraph 89 teaches that the training data inputs for Network C2 include DOI latent generated by Network A and pattern latent generated by Network B. The ground truth for Network C2 can either be images with defects generated by a physical model or real optical images with defects.);
training a machine learning model dedicated for detecting defects in examination output images (Paragraph 90 of Zhang teaches that in the general construction, the models described herein can be trained as follows: (1) Network A; (2) Network B; (3) Network C1 and/or Network C2. In the optimized construction, Network D can be trained directly (without Networks A, B, C1, and C2). Therefore, the training for the different options presented herein include (1) training Networks A, B, and C1, (2) training Networks A, B, and C2, (3) training Networks A, B, C1, and C2 (then selecting the best of C1 and C2 for inference performed with Networks A and B), and (4) training only Network D.);
obtaining at least one additional examination output image comprising a candidate defect (Paragraph 106 of Zhang teaches that for example, the computer subsystem may be configured to determine one or more changes to a process that was performed on a specimen inspected as described herein and/or a process that will be performed on the specimen based on the detected defect(s). The changes to the process may include any suitable changes to one or more parameters of the process. The computer subsystem preferably determines those changes such that the defects can be reduced or prevented on other specimens on which the revised process is performed, the defects can be corrected or eliminated on the specimen in another process performed on the specimen, the defects can be compensated for in another process performed on the specimen, etc. The computer subsystem may determine such changes in any suitable manner known in the art.);
applying the machine learning model to the at least one additional examination output image to thereby obtain machine learning output indicating whether the examination output images comprises one or more DOIs (Paragraph 112 of Zhang teaches that another embodiment relates to a computer-implemented method for determining information for a specimen. The method includes inputting one or more inputs specific to a specimen into a pre-trained DefectGPT encoder configured for determining information for the specimen based on the one or more inputs. The inputting is performed by a computer subsystem. One or more components are executed by the computer subsystem, and the one or more components include the pre-trained DefectGPT encoder.).
Regarding claim 5, Zhang in view of Yen disclose everything claimed as applied above (see claim 3), in addition, Zhang in view of Yen disclose using as part of a semiconductor fabrication process, an examination tool for examining one or more fabricated semiconductor specimens and generating the examination output images (Paragraph 106 of Zhang teaches that the results and information generated by performing the inspection on the specimen or other specimens of the same type may be used in a variety of manners by the embodiments described herein and/or other systems and methods. Such functions include, but are not limited to, altering a process such as a fabrication process or step that was or will be performed on the inspected specimen or another specimen in a feedback or feedforward manner.);
and generating the plurality of synthetic fault images by planting synthetic DOIs on-the-fly in the examination output images to thereby enable to use the plurality of synthetic fault images for real-time detection of DOIs in examination output images (Paragraph 99 of Zhang teaches that in another embodiment, the pre-trained DefectGPT encoder is configured for determining the information by jointly generating pattern and defect information for the specimen at the same time. For example, as described further herein, (1) the combination of Networks A, B, and C1, (2) the combination of Networks A, B, and C2, and (3) Network D can jointly generate pattern and defect information for a specimen at the same time.).
Regarding claim 9, Zhang in view of Yen disclose everything claimed as applied above (see claim 1), in addition, Zhang in view of Yen disclose wherein the primary DOI shape is selected from a database comprising a collection of optional defect shapes which are a product of commonly used optical configurations (Paragraph 62 of Zhang teaches that the input(s) may include a variety of different inputs, which as described further herein is one of the advantages of the embodiments described herein. In some embodiments, the computer subsystem may generate one or more of the inputs, but more generally, the computer subsystem may acquire one or more of the inputs, e.g., from an inspection recipe, from a storage medium, from one or more other systems and/or methods, from a user, etc. For example, inputs such as DOI location hints and DOI descriptors may be acquired from user input or from an inspection recipe, previously generate inspection results. and the like. Inputs such as optical modes and process parameters may be acquired from an inspection recipe and a process recipe, respectively, that are stored in some storage medium. Other inputs like design images may be acquired from a storage medium in which the design is stored and/or in which design images that have been generated from design data are stored. Additional inputs described herein may be acquired in similar manners. Additionally, paragraph 86 of Zhang teaches that the setup procedure for the embodiments described herein may be performed in a variety of ways described herein. The training data inputs for Network A may include DOI location hints and DOI descriptors. The DOI location hints may be a binary map that represents the location of defects. The DOI descriptors include the feature attributes of DOIs (optics, process, tool condition, etc.). The ground truth for Network A can either be defects generated by a physical model or real defects.).
Regarding claim 10, Zhang in view of Yen disclose everything claimed as applied above (see claim 1), in addition, Zhang in view of Yen disclose wherein the optical configuration of the examination tool includes actual optical configuration and/or presumed optical configuration (Paragraph 93 of Zhang teaches that in another embodiment, the pre-trained DefectGPT encoder is configured for determining the information by synthesizing DOIs for more than one optical mode. For example, another advantage of the embodiments described herein is the ability to synthesize DOIs in a multi-optical mode context, with consideration of process condition and tool condition. Additionally, paragraph 98 of Zhang teaches that in an additional embodiment, the pre-trained DefectGPT encoder is configured for determining the information for single mode or multiple mode optics conditions. Training an AI-based defect model for optical inspection with the defect images generated as described herein provides a number of benefits such as improving sensitivity of the defect detection model in single mode optical inspection and/or multiple mode optical inspection. Another benefit of such training is that it can improve defect discovery for out-of-distribution defects. Here, an out-of-distribution (OOD) defect refers to the possible defects that an application or customers expect to exist on a wafer, but are difficult to discover or identify through other means. Because these types of defects are not commonly available for downstream tasks like defect detection recipe setup, they are not part of the candidates or dataset during setup, and that is why they are OOD. By using the embodiments described herein, the proposed approach can simulate these “expected” defects and use them for the downstream tasks like training a defect detector. Then, the detector can be used to increase the chance to discover these OOD defects in practical experiments. An additional benefit is that such training can improve the generalizability of the AI detection model, e.g., among different wafers and/or different processes.).
Regarding claim 11, the system steps correlate to and are rejected similarly to the method steps of claim 1 (see claim 1 above). In addition, Zhang discloses a computer system (Paragraph 32 teaches that one embodiment of a system configured for determining information for a specimen is shown in FIG. 1. In some embodiments, system 10 includes an inspection subsystem such as inspection subsystem 100. The inspection subsystem includes and/or is coupled to a computer subsystem, e.g., computer subsystem 36 and/or one or more computer systems 102.) configured and operable to automatically generate synthetic fault images of a semiconductor specimen, wherein a synthetic fault image comprises at least one synthetic defect of interest (DOI);
the computer system comprising at least one processing circuitry (Paragraph 46 teaches that in general, the term “computer system” may be broadly defined to encompass any device having one or more processors, which executes instructions from a memory medium. The computer subsystem(s) or system(s) may also include any suitable processor known in the art such as a parallel processor. In addition, the computer subsystem(s) or system(s) may include a computer platform with high speed processing and software, either as a standalone or a networked tool.).
Regarding claim 12, the system steps correlate to and are rejected similarly to the method steps of claim 2 (see claim 2 above).
Regarding claim 13, the system steps correlate to and are rejected similarly to the method steps of claim 3 (see claim 3 above).
Regarding claim 14, the system steps correlate to and are rejected similarly to the method steps of claim 4 (see claim 4 above).
Regarding claim 15, the system steps correlate to and are rejected similarly to the method steps of claim 5 (see claim 5 above).
Regarding claim 19, the non-transitory computer readable medium steps correlate to and are rejected similarly to the method steps of claim 1 (see claim 1 above) and the system steps of claim 11 (see claim 11 above). In addition, Zhang discloses a non-transitory computer readable medium comprising instructions (Paragraph 27 teaches that FIG. 11 is a block diagram illustrating one embodiment of a non-transitory computer-readable medium storing program instructions for causing a computer system to perform a computer-implemented method described herein.).
Claims 6, 8, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Yen as applied to claims 1 and 11 above, and further in view of Brauer et al. (Pub. No.: US 2018/0157933 A1), hereinafter Brauer.
Regarding claim 6, Zhang in view of Yen disclose everything claimed as applied above (see claim 1), however, Zhang in view of Yen fail to disclose augmenting the at least one synthetic DOI.
Brauer discloses augmenting the at least one synthetic DOI (Paragraph 87 teaches that in one embodiment of the present disclosure may be described as a method 100 for providing an augmented input data to a convolutional neural network (CNN), which is seen in FIG. 5. The augmented input data may comprise a plurality of training images or a plurality of training sets. The augmented input data may come in a variety of formats suitable for the CNN.). Since Zhang in view of Yen teach the ability to make modifications to the overall process of the creation of synthetic DOI images for use in training a machine learning model in relation to an inspection type system and Brauer teaches augmenting and modifying images that can be used in training a machine learning model for use in an inspection type system , it would have been obvious to a person having ordinary skill in the art to combine the functions together so that any of the images or data being generating for the machine learning model, could also use data that has been augmented in a way or have its data be augmented.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Yen to incorporate the teachings of Brauer, so that any data being used to train the machine learning model could be data that has been augmented in some way, which would help improve overall model performance by using a more diverse set of training data which tends to lead to better model accuracy.
Furthermore, Zhang in view of Yen and Brauer disclose augmenting the at least one synthetic DOI comprising:
scaling the final DOI shape in one or two dimensions and/or applying rotation of the final DOI shape (Paragraph 97 of Brauer teaches that in another embodiment of the method 100, the method may further comprise creating 119 a plurality of transposed images using the processor. The transposed images are created 119 by transposing the plurality of reference images and the plurality of difference images with respect to the received one or more test images. For example, the transposition may be a sub pixel offset with respect to the received one or more test images. In another example, the transposition may be a multi-pixel offset with respect to the received one or more test images. The transposition for each reference image and difference image may be the same for the entire plurality or may be varied. Additionally, paragraph 11 of Brauer teaches that in order to avoid overfitting, it is necessary to use additional techniques, such as data augmentation. Data augmentation takes existing data, such as existing wafer images, and applies mathematical functions to the data in order to create new, but similarly indicative images. For example, currently used data augmentation techniques include rotation, translation, zooming, flipping, and cropping of images.).
Regarding claim 8, Zhang in view of Yen discloses everything claimed as applied above (see claim 1), however, Zhang in view of Yen fail to disclose wherein each shape in the collection of predefined secondary shapes is represented as a kernel having a certain size with pixels of different colors distributed within the kernel.
Brauer discloses wherein each shape in the collection of predefined secondary shapes is represented as a kernel having a certain size with pixels of different colors distributed within the kernel (Paragraph 54 teaches that a CNN architecture may be formed by a stack of distinct layers that transform the input volume into an output volume (e.g., holding class scores) through a differentiable function. A few distinct types of layers may be used. The convolutional layer has a variety of parameters that consist of a set of learnable filters (or kernels), which have a small receptive field, but extend through the full depth of the input volume. Additionally, paragraph 56 teaches that the three hyperparameters control the size of the output volume of the convolutional layer: the depth, stride and zero-padding. Depth of the output volume controls the number of neurons in the layer that connect to the same region of the input volume. All of these neurons will learn to activate for different features in the input. For example, if the first CNN Layer takes the raw image as input, then different neurons along the depth dimension may activate in the presence of various oriented edges, or blobs of color.). Since Zhang in view of Yen teach the initial method steps for collecting DOI data in relation to a DOI’s shape and pixel values for use in a machine learning model within a defect inspection system and Brauer teaches using a convolutional neural network in an defect inspection system that consists of layers of kernels that can activate different layers, such as those that consist of different colors, it would have been obvious to a person having ordinary skill in the art to combine the functions together so that kernels could be implemented within the defect inspection system to help improve the machine learning process by potentially improving overall detection accuracy and efficiency.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Yen to incorporate the teachings of Brauer, so that by incorporating kernels into the machine learning process, overall detection accuracy and efficiency could improve due to the kernel being able to potentially adjust its size in accordance with various colors, which would allow for better overall detection and recognition of various sized DOIs.
Regarding claim 16, the system steps correlate to and are rejected similarly to the method steps of claim 6 (see claim 6 above).
Regarding claim 18, the system steps correlate to and are rejected similarly to the method steps of claim 8 (see claim 8 above).
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Yen as applied to claims 1 and 11 above, and further in view of Huang (Pub. No.: US 2014/0270475 A1).
Regarding claim 7, Zhang in view of Yen discloses everything claimed as applied above (see claim 1), however, Zhang in view of Yen fail to disclose wherein fusing the primary DOI shape and secondary DOI shape comprises convolving the primary DOI shape with the secondary DOI shape.
Huang discloses wherein fusing the primary DOI shape and secondary DOI shape comprises convolving the primary DOI shape with the secondary DOI shape (Paragraph 44 of Huang teaches that in one embodiment, portions of the additional image data that correspond to the defects have greater signal-to-noise ratios (S/Ns) than portions of the first and second image data that are combined to create the portions of the additional image data. For example, the cell fusion described above can keep the defect signal, which is not impacted much by the different cell sizes, while cancelling out or suppressing the residual noise with different patterns. In addition, by combining (or fusing) information to at the pixel level, weak signal strengths from defects of interest (DOI) may be enhanced because the noise is greatly suppressed. For example, fusing information at the pixel level thereby leveraging both magnitude (intensity) and phase (correlation) information allows one to extract defects with weak signals by suppressing noise and nuisance events through exploitation of their respective coincidence and non-coincidence. In this manner, one advantage of the embodiments described herein is that pattern noise can be greatly reduced in the additional image data compared to the first and second image data while defect S/Ns in the additional image data are improved compared to the first and second image data. As such, a defect that is not detectable in either of the first and second image data may become detectable in the corresponding additional image data created by image correlation.). Since Zhang in view of Yen teach the initial method steps for fusing data together in relation to the DOIs and Huang teaches a method step for fusing and combining information from a first DOI image and a second DOI image including the pixel values of DOIs in relation to their pattern or shape, it would have been obvious to a person having ordinary skill in the art to combine the functions together so that when fusing DOI images together, the pixel values could also be accounted for to help in merging and combining different DOI shapes together.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang in view of Yen to incorporate the teachings of Huang, so that fusing different DOI shapes together would help ensure a more robust and larger data set to use in training and potentially allow for better recognition of more complex shapes.
Regarding claim 17, the system steps correlate to and are rejected similarly to the method steps of claim 7 (see claim 7 above).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Nalem Venkat et al. (Pub. No.: US 2024/0037723 A1) teaches methods of detecting defects in machine vision systems.
Hirai et al. (Pub. No.: US 2022/0292317 A1) teaches a defect classification apparatus and method and program
Koronel et al. (Pub. No.: US 2022/0207682 A1) teaches an inspection system for a semiconductor specimen
Any inquiry concerning this communication or earlier communications from the examiner should be directed to George Renze whose telephone number is (703)756-5811. The examiner can normally be reached Monday-Friday 9:00am - 6:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/G.R./Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613