DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are currently pending in this application.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 9, 11, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Houben et al. (US 2024/0054669 A1) (hereinafter, “Houben”) in view of Kim et al. (US 2024/0161322 A1) (hereinafter, “Kim”).
Regarding claim 1, Houben discloses a computer system configured to generate height maps of a semiconductor specimen, the computer system comprising at least one processing circuitry configured to (Paragraph [0007] "the system includes an image capture device such as a scanning electron microscope (SEM) having an electron beam optics configured to capture an image of a patterned substrate; and one or more processors including a trained model stored in a memory that can receive the captured image and execute the model to determine depth information form the captured image."):
obtain one or more examination output images (SEM image in Paragraph [0082] equates to the examination output images) generated by a semiconductor examination tool (Paragraph [0082] " the method 500 may include a process for obtaining, via the metrology tool, a SEM image of a patterned substrate at the first e-beam tilt setting of the metrology tool. The SEM image may be a normal image obtained by directing an e-beam approximately perpendicular to the patterned substrate. ");
apply a first machine learning (ML) model (model M1 in Paragraph [0082] equates to the first machine learning model) on the one or more examination output images (SEM image in Paragraph [0082] equates to the examination output images) to generate one or more respective height maps (depth information in Paragraph [0082] equates to the height maps) (Paragraph [0009] " a model (e.g., CNN) configured to predict depth data from an inputted image; obtaining a captured image (e.g., SEM image) and observed depth data (e.g., measured height map) of the structure patterned on a substrate" Paragraph [0082] "The method 500 may further include executing the model M1 using the SEM image as input to generate disparity data associated with the SEM image; and applying a conversion function (e.g., a linear function, a constant conversion factor, or a non-linear function) to the disparity data to generate depth information of the structure in the SEM image. "); wherein the first machine learning model is a model trained with a training dataset that includes examination output images of a semiconductor specimen and respective height maps (Paragraph [0120] "the model is trained using training data generated as a pair of SEM images and corresponding simulated 3D profiles that may be formed on a substrate. The simulated profiles may be generated via a calibrated process model (e.g., a calibrated resist model)... the process model calibration may involve comparing height of 3D simulated geometry to height extracted from atomic force microscopy (AFM) data or optical metrology data."), and is dedicated to converting examination output images to height maps while avoiding artifacts commonly found in height maps generated based on examination output images (Paragraph [0113] in the SEM image 902 of line-space features, SEM charging artifacts are seen just right of trenches where the SEM signal is reduced compared to at the edge of the trench. When such SEM image 902 having charging effect is inputted to the initial model M70 (e.g., in FIG. 7 ), the predicted height maps are affected by this reduction in SEM intensity, as shown (see right of the trenches) in the predicted height map 904 in FIG. 9B. To correct for this, a fine-tuning or further calibration step was employed in the training process of the model M70, as discussed earlier. In such calibration step, the predicted data is adjusted to assume the lines for the new training set to have a flat surface. When such adjusted predicted data is used to generate calibrated model 710 (e.g., in FIG. 7 ), the predicted height maps (by the model 701) do not suffer from charging anymore, but do have a flat surface on top of the lines, as shown the predicted height map 906 in FIG. 9C."), by utilizing a loss function (Paragraph [0067] "In an embodiment, the performance function comprises a similarity loss Lsim(ƒ, m∘ϕ) indicative of similarity between the reconstructed image and the inputted SEM image (e.g., represented as a function ƒ)") [dedicated to maintaining consistency between the examination output images and the respective height maps].
However, Houben fails to teach dedicated to maintaining consistency between the examination output images and the respective height maps.
Kim teaches dedicated to maintaining consistency between the examination output images (simulated images in Paragraph [0008] equate to the examination output images) and the respective height maps (depth map in Paragraph [0008] equate to the height map) (Paragraph [0008] "The method may further include retraining the trained artificial neural network model based on the second depth map and the second simulated image."; Paragraph [0054] "the depth information may be estimated by receiving the depth information 220, inputting an output of a simulator estimating the image 210 into an ANN model, and inputting an output of the ANN model back to the simulator even in a situation in which it is difficult to obtain the depth information 220 sufficient to train the ANN model").
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben’s reference to include dedicated to maintaining consistency between the examination output images and the respective height maps taught by Kim’s reference. The motivation for doing so would have been to train the model to reduce the differences between a depth map and the data obtained from the SEM image as suggested by Kim (see Kim, Paragraph [0066]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Kim with Houben to obtain the invention specified in claim 1.
Regarding claim 9, which claim 1 is incorporated, Houben discloses wherein the examination tool is a Scanning Electron Microscope (SEM) and the examination output images are SEM output images (Paragraph [0082] " the method 500 may include a process for obtaining, via the metrology tool, a SEM image of a patterned substrate at the first e-beam tilt setting of the metrology tool. The SEM image may be a normal image obtained by directing an e-beam approximately perpendicular to the patterned substrate. ").
Regarding claim 11, Houben discloses a computer-implemented method of generating height maps of a semiconductor specimen (Paragraph [0007] "the system includes an image capture device such as a scanning electron microscope (SEM) having an electron beam optics configured to capture an image of a patterned substrate; and one or more processors including a trained model stored in a memory that can receive the captured image and execute the model to determine depth information form the captured image."), the method comprising:
obtaining one or more examination output images (SEM image in Paragraph [0082] equates to the examination output images) generated by a semiconductor examination tool (Paragraph [0082] " the method 500 may include a process for obtaining, via the metrology tool, a SEM image of a patterned substrate at the first e-beam tilt setting of the metrology tool. The SEM image may be a normal image obtained by directing an e-beam approximately perpendicular to the patterned substrate. ");
applying a first machine learning (ML) model (model M1 in Paragraph [0082] equates to the first machine learning model) on the one or more examination output images (SEM image in Paragraph [0082] equates to the examination output images) to generate one or more respective height maps (depth information in Paragraph [0082] equates to the height maps) (Paragraph [0009] " a model (e.g., CNN) configured to predict depth data from an inputted image; obtaining a captured image (e.g., SEM image) and observed depth data (e.g., measured height map) of the structure patterned on a substrate" Paragraph [0082] "The method 500 may further include executing the model M1 using the SEM image as input to generate disparity data associated with the SEM image; and applying a conversion function (e.g., a linear function, a constant conversion factor, or a non-linear function) to the disparity data to generate depth information of the structure in the SEM image. "); wherein the first machine learning model is a model trained with a training dataset that includes examination output images of a semiconductor specimen and respective height maps (Paragraph [0120] " the model is trained using training data generated as a pair of SEM images and corresponding simulated 3D profiles that may be formed on a substrate. The simulated profiles may be generated via a calibrated process model (e.g., a calibrated resist model)... the process model calibration may involve comparing height of 3D simulated geometry to height extracted from atomic force microscopy (AFM) data or optical metrology data."), and is dedicated to converting examination output images to height maps while utilizing a loss function (Paragraph [0065] "the tilted image 302 may be represented as a function m and the disparity data 311 (e.g., a map) may be represented as another function ϕ. By combining the disparity data 311 with the tilted image 302, a reconstructed image may be obtained. For example, the reconstructed image is represented by a function obtained from a composition of functions m and ϕ. In an embodiment, the reconstructed image may be represented as m∘ϕ, where the symbol a denotes the composition operation between functions e.g., m(x)∘ϕ(x)=m(ϕ(x)), x being a vector of xy-coordinates of an image."; Paragraph [0067] "In an embodiment, the performance function comprises a similarity loss Lsim(ƒ, m∘ϕ) indicative of similarity between the reconstructed image and the inputted SEM image (e.g., represented as a function ƒ)") [dedicated to maintaining consistency between the examination output images and the respective height maps] to thereby reduce artifacts commonly found in height maps generated based on examination output images (Paragraph [0113] FIG. 9A is an exemplary SEM image 902 including charging artifacts. For example, in the SEM image 902 of line-space features, SEM charging artifacts are seen just right of trenches where the SEM signal is reduced compared to at the edge of the trench. When such SEM image 902 having charging effect is inputted to the initial model M70 (e.g., in FIG. 7 ), the predicted height maps are affected by this reduction in SEM intensity, as shown (see right of the trenches) in the predicted height map 904 in FIG. 9B. To correct for this, a fine-tuning or further calibration step was employed in the training process of the model M70, as discussed earlier. In such calibration step, the predicted data is adjusted to assume the lines for the new training set to have a flat surface. When such adjusted predicted data is used to generate calibrated model 710 (e.g., in FIG. 7 ), the predicted height maps (by the model 701) do not suffer from charging anymore, but do have a flat surface on top of the lines, as shown the predicted height map 906 in FIG. 9C.").
However, Houben fails to teach dedicated to maintaining consistency between the examination output images and the respective height maps.
Kim teaches dedicated to maintaining consistency between the examination output images (simulated images in Paragraph [0008] equate to the examination output images) and the respective height maps (depth map in Paragraph [0008] equate to the height map) (Paragraph [0008] "The method may further include retraining the trained artificial neural network model based on the second depth map and the second simulated image."; Paragraph [0054] "the depth information may be estimated by receiving the depth information 220, inputting an output of a simulator estimating the image 210 into an ANN model, and inputting an output of the ANN model back to the simulator even in a situation in which it is difficult to obtain the depth information 220 sufficient to train the ANN model").
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben’s reference to include dedicated to maintaining consistency between the examination output images and the respective height maps taught by Kim’s reference. The motivation for doing so would have been to train the model to reduce the differences between a depth map and the data obtained from the SEM image as suggested by Kim (see Kim, Paragraph [0066]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Kim with Houben to obtain the invention specified in claim 11.
Regarding claim 20, Houben discloses a computer program product comprising a non-transitory computer-readable medium having computer-executable instructions stored thereon, which, when executed by a processor, cause the processor to execute a method of generating height maps of a semiconductor specimen (Paragraph [0192] “the procedures may be distributed across a plurality of processors (e.g., parallel computation) to improve computing efficiency. In an embodiment, the computer program product comprising a non-transitory computer readable medium has instructions recorded thereon, the instructions when executed by a computer hardware system implementing the method”), the method comprising:
obtaining one or more examination output images (SEM image in Paragraph [0082] equates to the examination output images) generated by a semiconductor examination tool (Paragraph [0082] " the method 500 may include a process for obtaining, via the metrology tool, a SEM image of a patterned substrate at the first e-beam tilt setting of the metrology tool. The SEM image may be a normal image obtained by directing an e-beam approximately perpendicular to the patterned substrate. ");
applying a first machine learning (ML) model (model M1 in Paragraph [0082] equates to the first machine learning model) on the one or more examination output images (SEM image in Paragraph [0082] equates to the examination output images) to generate one or more respective height maps (depth information in Paragraph [0082] equates to the height maps) (Paragraph [0009] " a model (e.g., CNN) configured to predict depth data from an inputted image; obtaining a captured image (e.g., SEM image) and observed depth data (e.g., measured height map) of the structure patterned on a substrate" Paragraph [0082] "The method 500 may further include executing the model M1 using the SEM image as input to generate disparity data associated with the SEM image; and applying a conversion function (e.g., a linear function, a constant conversion factor, or a non-linear function) to the disparity data to generate depth information of the structure in the SEM image. "); wherein the first machine learning model is a model trained with a training dataset that includes examination output images of a semiconductor specimen and respective height maps (Paragraph [0120] " the model is trained using training data generated as a pair of SEM images and corresponding simulated 3D profiles that may be formed on a substrate. The simulated profiles may be generated via a calibrated process model (e.g., a calibrated resist model)... the process model calibration may involve comparing height of 3D simulated geometry to height extracted from atomic force microscopy (AFM) data or optical metrology data."), and is dedicated to converting examination output images to height maps while utilizing a loss function (Paragraph [0065] "the tilted image 302 may be represented as a function m and the disparity data 311 (e.g., a map) may be represented as another function ϕ. By combining the disparity data 311 with the tilted image 302, a reconstructed image may be obtained. For example, the reconstructed image is represented by a function obtained from a composition of functions m and ϕ. In an embodiment, the reconstructed image may be represented as m∘ϕ, where the symbol a denotes the composition operation between functions e.g., m(x)∘ϕ(x)=m(ϕ(x)), x being a vector of xy-coordinates of an image."; Paragraph [0067] "In an embodiment, the performance function comprises a similarity loss Lsim(ƒ, m∘ϕ) indicative of similarity between the reconstructed image and the inputted SEM image (e.g., represented as a function ƒ)") [dedicated to maintaining consistency between the examination output images and the respective height maps] to thereby reduce artifacts commonly found in height maps generated based on examination output images (Paragraph [0113] FIG. 9A is an exemplary SEM image 902 including charging artifacts. For example, in the SEM image 902 of line-space features, SEM charging artifacts are seen just right of trenches where the SEM signal is reduced compared to at the edge of the trench. When such SEM image 902 having charging effect is inputted to the initial model M70 (e.g., in FIG. 7 ), the predicted height maps are affected by this reduction in SEM intensity, as shown (see right of the trenches) in the predicted height map 904 in FIG. 9B. To correct for this, a fine-tuning or further calibration step was employed in the training process of the model M70, as discussed earlier. In such calibration step, the predicted data is adjusted to assume the lines for the new training set to have a flat surface. When such adjusted predicted data is used to generate calibrated model 710 (e.g., in FIG. 7 ), the predicted height maps (by the model 701) do not suffer from charging anymore, but do have a flat surface on top of the lines, as shown the predicted height map 906 in FIG. 9C.").
However, Houben fails to teach dedicated to maintaining consistency between the examination output images and the respective height maps.
Kim teaches dedicated to maintaining consistency between the examination output images (simulated images in Paragraph [0008] equate to the examination output images) and the respective height maps (depth map in Paragraph [0008] equate to the height map) (Paragraph [0008] "The method may further include retraining the trained artificial neural network model based on the second depth map and the second simulated image."; Paragraph [0054] "the depth information may be estimated by receiving the depth information 220, inputting an output of a simulator estimating the image 210 into an ANN model, and inputting an output of the ANN model back to the simulator even in a situation in which it is difficult to obtain the depth information 220 sufficient to train the ANN model").
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben’s reference to include dedicated to maintaining consistency between the examination output images and the respective height maps taught by Kim’s reference. The motivation for doing so would have been to train the model to reduce the differences between a depth map and the data obtained from the SEM image as suggested by Kim (see Kim, Paragraph [0066]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Kim with Houben to obtain the invention specified in claim 20.
Claims 2-7, 10, 12-17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Houben et al. (US 2024/0054669 A1) (hereinafter, “Houben”) in view of Kim et al. (US 2024/0161322 A1) (hereinafter, “Kim”), and further in view of Shaubi et al. (US 2020/0226420 A1) (hereinafter, “Shaubi”).
Regarding claim 2, which claim 1 is incorporated, Houben fails to teach wherein the at least one processing circuitry is configured to: modify the one or more respective height maps by adding at least one topographic feature, thereby generating one or more modified height maps; and apply a second machine learning (ML) model to the one or more modified height maps to convert the one or more modified height maps to one or more respective modified synthetic examination output images of the semiconductor specimen.
Kim teaches wherein the at least one processing circuitry is configured to: modify the one or more respective height maps [by adding at least one topographic feature], thereby generating one or more modified height maps (modified depth maps in Paragraph [0017] equate to modified height maps) (Paragraph [0017] “The processor may generate, from the second depth map, a plurality of modified depth maps based on data augmentation including any one or any combination of any two or more of random noise addition, image rotation, scale adjustment, image movement, random crop, and color distortion”); and
[apply a second machine learning (ML) model] to the one or more modified height maps to convert the one or more modified height maps to one or more respective modified synthetic examination output images (simulated SEM image in Paragraph [0055] equates to the synthetic examination output images) of the semiconductor specimen (Paragraph [0055] “the system for measuring a depth of a semiconductor sample may receive an SEM image, a TEM image, and/or an AFM image and estimate the depth information of a semiconductor sample.”; Paragraph [0075] “The depth information estimation system may obtain a simulated SEM image by inputting the plurality of modified depth maps 520 into a simulator (e.g., the simulator 310 of FIG. 3 ) and train an ANN model based on the modified depth maps and the simulated SEM image.”)
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben’s reference to include wherein the at least one processing circuitry is configured to: modify the one or more respective height maps [by adding at least one topographic feature], thereby generating one or more modified height maps and [apply a second machine learning (ML) model] to the one or more modified height maps to convert the one or more modified height maps to one or more respective modified synthetic examination output images of the semiconductor specimen taught by Kim’s reference. The motivation for doing so would have been to train the model to reduce the differences between a depth map and the data obtained from the SEM image as suggested by Kim (see Kim, Paragraph [0066]).
However, Houben and Kim both fail to teach by adding at least one topographic feature and apply a second machine learning (ML) model.
Shaubi teaches by adding at least one topographic feature and apply a second machine learning (ML) model (Paragraph [0087] “generating the training set can further include using the obtained "real world" training samples to generate (403) one or more augmented images specific for the given examination-related application and including the generated augmented images into the training set”; Paragraph [0083] “a captured image from a "real world" training sample can be augmented using segmentation, defect contour extraction and/or height map calculation, and/or can be obtained by processing together with a corresponding CAD-based image”; Paragraph [0077] “plant a new defect in an image, amplify the defectiveness of a pre-existing defect in the image, remove a defect from the image, disguise a defect in the image”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben in view of Kim to include by adding at least one topographic feature and apply a second machine learning (ML) model taught by Shaubi’s reference. The motivation for doing so would have been to increase the effectiveness of defect detection and classification by automating the examination process as suggested by Shaubi (see Shaubi, Paragraph [0006]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Shaubi with Houben and Kim to obtain the invention specified in claim 2.
Regarding claim 3, which claim 2 is incorporated, Houben discloses wherein the first ML model and the second ML model are trained concurrently in a cyclic training process (Paragraph [0141] " using a cycle-GAN training, the first model (e.g., G1 of FIG. 14 ) may be further trained in conjunction with third model (e.g., another generator model G2 of FIG. 14 ). The third model may be configured to generate a SEM image from the estimated depth data. In other words, the first model and the third model are cycle-consistent.").
Regarding claim 4, which claim 2 is incorporated, Houben discloses wherein the first machine learning model and the second machine learning model each apply a loss function [dedicated to maintaining consistency between the examination output images and the respective height maps] (Paragraph [0141] " the first model and the third model are trained so that the input of the first model closely matches the output of the third model using a loss function (e.g., a difference between images 1402 and 1406 in FIG. 14 )."; Paragraph [0142] "The training of the first model and the third model may be an iterative process. Each iteration includes inputting a SEM image of the plurality of SEM images 1301 to the first model; estimating, using the first model, the depth data for inputted SEM image; generating, via the third model using the estimated depth data as input, a predicted SEM image; and updating model parameters of both the first model and the third model to cause a difference between the inputted SEM image and the predicted SEM image to be within a specified difference threshold."); wherein the loss function includes at least a loss component configured to maintain correlation between gradients [in the one or more examination output images and the respective one or more height maps] to thereby avoid the insertion of artifacts (Paragraph [0066] "the adjusting comprises determining a gradient map of the performance function by taking a derivative of the performance function with respect to the one or more model parameters. The gradient map guides the adjustment of the one or more model parameters in a direction that minimizes or brings the performance function values within a specified threshold. For example, the specified threshold may be that the difference of the performance function values between a current iteration and a subsequent iteration is less than 1%.").
However, Houben and Shaubi both fail to teach dedicated to maintaining consistency between the examination output images and the respective height maps and [maintain correlation…] in the one or more examination output images and the respective one or more height maps.
Kim teach dedicated to maintaining consistency between the examination output images and the respective height maps and [maintain correlation…] in the one or more examination output images and the respective one or more height maps (Paragraph [0008] "The method may further include retraining the trained artificial neural network model based on the second depth map and the second simulated image."; Paragraph [0054] "the depth information may be estimated by receiving the depth information 220, inputting an output of a simulator estimating the image 210 into an ANN model, and inputting an output of the ANN model back to the simulator even in a situation in which it is difficult to obtain the depth information 220 sufficient to train the ANN model")
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben’s reference to include dedicated to maintaining consistency between the examination output images and the respective height maps and [maintain correlation…] in the one or more examination output images and the respective one or more height maps taught by Kim’s reference. The motivation for doing so would have been to train the model to reduce the differences between a depth map and the data obtained from the SEM image as suggested by Kim (see Kim, Paragraph [0066]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Kim with Houben and Shaubi to obtain the invention specified in claim 4.
Regarding claim 5, which claim 2 is incorporated, Houben fails to teach, wherein a new topographic feature is an artificially generated 3D defect, wherein the one or more modified height maps include at least one fault height map that includes at least one artificially generated 3D defect, and wherein the one or more modified examination output images include one or more synthetic fault images, exhibiting at least one artificially generated 3D defect.
Kim teaches [wherein a new topographic feature is an artificially generated 3D defect], wherein the one or more modified height maps include at least one fault height map [that includes at least one artificially generated 3D defect] (Paragraph [0017] “The processor may generate, from the second depth map, a plurality of modified depth maps based on data augmentation including any one or any combination of any two or more of random noise addition, image rotation, scale adjustment, image movement, random crop, and color distortion”), and wherein the one or more modified examination output images include one or more synthetic fault images, [exhibiting at least one artificially generated 3D defect] (Paragraph [0075] “The depth information estimation system may obtain a simulated SEM image by inputting the plurality of modified depth maps 520 into a simulator (e.g., the simulator 310 of FIG. 3 ) and train an ANN model based on the modified depth maps and the simulated SEM image.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben’s reference to include [wherein a new topographic feature is an artificially generated 3D defect], wherein the one or more modified height maps include at least one fault height map [that includes at least one artificially generated 3D defect], and wherein the one or more modified examination output images include one or more synthetic fault images, [exhibiting at least one artificially generated 3D defect] taught by Kim’s reference. The motivation for doing so would have been to train the model to reduce the differences between a depth map and the data obtained from the SEM image as suggested by Kim (see Kim, Paragraph [0066]).
However, Houben and Kim both fail to teach wherein a new topographic feature is an artificially generated 3D defect, that includes at least one artificially generated 3D defect, and exhibiting at least one artificially generated 3D defect.
Shaubi teaches teach wherein a new topographic feature is an artificially generated 3D defect, that includes at least one artificially generated 3D defect (Paragraph [0087] “generating the training set can further include using the obtained "real world" training samples to generate (403) one or more augmented images specific for the given examination-related application and including the generated augmented images into the training set”; Paragraph [0083] “a captured image from a "real world" training sample can be augmented using segmentation, defect contour extraction and/or height map calculation, and/or can be obtained by processing together with a corresponding CAD-based image”), and exhibiting at least one artificially generated 3D defect (Paragraph [0077] “plant a new defect in an image, amplify the defectiveness of a pre-existing defect in the image, remove a defect from the image, disguise a defect in the image”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben in view of Kim to include wherein a new topographic feature is an artificially generated 3D defect, that includes at least one artificially generated 3D defect, and exhibiting at least one artificially generated 3D defect taught by Shaubi’s reference. The motivation for doing so would have been to increase the effectiveness of defect detection and classification by automating the examination process as suggested by Shaubi (see Shaubi, Paragraph [0006]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Shaubi with Houben and Kim to obtain the invention specified in claim 5.
Regarding claim 6, which claim 5 is incorporated, Houben and Kim both fail to teach wherein the at least one processing circuitry is configured to generate the one or more synthetic fault images as part of execution of a defect detection process, dedicated for detecting 3D defects in examination output images of a semiconductor; wherein the at least one processing circuitry is further configured to execute as part of the defect detection process: generate a training dataset comprising a collection of examination output images including the one or more synthetic fault images and a plurality of fault-free images; utilize the training dataset for training a third machine learning model dedicated for receiving examination output images of a semiconductor specimen and detecting defects therein; and apply the third machine learning model to examination output images of the semiconductor specimen acquired by the examination tool, to thereby detect defects in the semiconductor specimens.
Shaubi teaches wherein the at least one processing circuitry is configured to generate the one or more synthetic fault images as part of execution of a defect detection process, dedicated for detecting 3D defects in examination output images of a semiconductor; wherein the at least one processing circuitry is further configured to execute as part of the defect detection process (Paragraph [0072] " training set generator 111 comprises a generator of augmented images 301, a generator of synthetic images 302 and an output training set module 303 operatively coupled to generator 301 and generator 302 and configured to generate a training set comprising “real world” training sample, synthetic training sample and/or augmented training sample. The generated training set can be stored in the memory of PMC 104."; Paragraph [0077] "image from a “real world” training sample can be augmented using synthetic data (e.g. defect-related data, simulated connectors or other objects, implants from other images, etc.)."; Paragraph [0038] "to DNN 112, PMC can comprise Automatic Defect Review Module (ADR) and/or Automatic Defect Classification Module (ADC) and/or other examination modules usable, after being trained, for examination of a semiconductor specimen."):
generate a training dataset comprising a collection of examination output images including the one or more synthetic fault images and a plurality of fault-free images (“real world” training sample, synthetic training sample and/or augmented training sample in Paragraph [0072] equate to the synthetic fault images and fault-free images) (Paragraph [0072] " training set generator 111 comprises a generator of augmented images 301, a generator of synthetic images 302 and an output training set module 303 operatively coupled to generator 301 and generator 302 and configured to generate a training set comprising “real world” training sample, synthetic training sample and/or augmented training sample. The generated training set can be stored in the memory of PMC 104."; Paragraph [0077] "image from a “real world” training sample can be augmented using synthetic data (e.g. defect-related data, simulated connectors or other objects, implants from other images, etc.).");
utilize the training dataset for training a third machine learning model (DNN 112 equates to a third machine learning model) dedicated for receiving examination output images of a semiconductor specimen and detecting defects therein (Paragraph [0038] "to DNN 112, PMC can comprise Automatic Defect Review Module (ADR) and/or Automatic Defect Classification Module (ADC) and/or other examination modules usable, after being trained, for examination of a semiconductor specimen."); and
apply the third machine learning model to examination output images of the semiconductor specimen acquired by the examination tool, to thereby detect defects in the semiconductor specimens (Paragraph [0010] "the training set can be usable for at least one of automatic defect review (ADR), automatic defect classification (ADC), and automatic examination of the specimen using a Deep Neural Network (DNN) trained for the given examination-related application.").
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben in view of Kim to include wherein the at least one processing circuitry is configured to generate the one or more synthetic fault images as part of execution of a defect detection process, dedicated for detecting 3D defects in examination output images of a semiconductor; wherein the at least one processing circuitry is further configured to execute as part of the defect detection process: generate a training dataset comprising a collection of examination output images including the one or more synthetic fault images and a plurality of fault-free images; utilize the training dataset for training a third machine learning model dedicated for receiving examination output images of a semiconductor specimen and detecting defects therein; and apply the third machine learning model to examination output images of the semiconductor specimen acquired by the examination tool, to thereby detect defects in the semiconductor specimens taught by Shaubi’s reference. The motivation for doing so would have been to increase the effectiveness of defect detection and classification by automating the examination process as suggested by Shaubi (see Shaubi, Paragraph [0006]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Shaubi with Houben and Kim to obtain the invention specified in claim 6.
Regarding claim 7, which claim 6 is incorporated, Houben and Kim both fail to teach wherein the defect detection process is executed as part of a semiconductor examination process for detecting defects in real-time during examination.
Shaubi discloses wherein the defect detection process is executed as part of a semiconductor examination process for detecting defects in real-time during examination (Paragraph [0068] "FP images can be selected from images of specimen (e.g. wafer or parts thereof) captured during the manufacturing process"; Paragraph [0069] "application-specific examination-related data can represent a per-pixel map of values whose meaning depends on an application (e.g. binary map for defect detection; discrete map for nuisance family prediction indicating the family type or general class; discrete map for defect type classification; continuous values for cross modality or die-to model (D2M) regression, etc.).").
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben in view of Kim to wherein the defect detection process is executed as part of a semiconductor examination process for detecting defects in real-time during examination taught by Shaubi’s reference. The motivation for doing so would have been to increase the effectiveness of defect detection and classification by automating the examination process as suggested by Shaubi (see Shaubi, Paragraph [0006]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Shaubi with Houben and Kim to obtain the invention specified in claim 7.
Regarding claim 10, which claim 1 is incorporated, Houben discloses wherein the one or more examination output images include one or more fault-free images; wherein the processing circuitry is further configured to (Paragraph [0082] " the method 500 may include a process for obtaining, via the metrology tool, a SEM image of a patterned substrate at the first e-beam tilt setting of the metrology tool. The SEM image may be a normal image obtained by directing an e-beam approximately perpendicular to the patterned substrate. "):
apply the first ML model on the one or more fault-free images, thereby obtaining one or more respective height maps (Paragraph [0009] " a model (e.g., CNN) configured to predict depth data from an inputted image; obtaining a captured image (e.g., SEM image) and observed depth data (e.g., measured height map) of the structure patterned on a substrate"; Paragraph [0082] "The method 500 may further include executing the model M1 using the SEM image as input to generate disparity data associated with the SEM image; and applying a conversion function (e.g., a linear function, a constant conversion factor, or a non-linear function) to the disparity data to generate depth information of the structure in the SEM image. ");
[apply a second machine learning (ML) model to the one or more respective height maps to convert the one or more height maps to one or more respective synthetic fault-free images of the semiconductor specimen].
However, Houben and Kim both fail to teach apply a second machine learning (ML) model to the one or more respective height maps to convert the one or more height maps to one or more respective synthetic fault-free images of the semiconductor specimen.
Shaubi teaches apply a second machine learning (ML) model to the one or more respective height maps to convert the one or more height maps to one or more respective synthetic fault-free images of the semiconductor specimen (Paragraph [0087] “generating the training set can further include using the obtained "real world" training samples to generate (403) one or more augmented images specific for the given examination-related application and including the generated augmented images into the training set”; Paragraph [0083] “a captured image from a "real world" training sample can be augmented using segmentation, defect contour extraction and/or height map calculation, and/or can be obtained by processing together with a corresponding CAD-based image”; Paragraph [0077] “plant a new defect in an image, amplify the defectiveness of a pre-existing defect in the image, remove a defect from the image, disguise a defect in the image”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben in view of Kim to include apply a second machine learning (ML) model to the one or more respective height maps to convert the one or more height maps to one or more respective synthetic fault-free images of the semiconductor specimen taught by Shaubi’s reference. The motivation for doing so would have been to increase the effectiveness of defect detection and classification by automating the examination process as suggested by Shaubi (see Shaubi, Paragraph [0006]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Shaubi with Houben and Kim to obtain the invention specified in claim 10.
Regarding claim 12 (drawn to a method), claim 12 is rejected the same as claim 2 and the arguments similar to that presented above for claim 2 are equally applicable to the claim 12, and all the other limitations similar to claim 2 are not repeated herein, but incorporated by reference.
Regarding claim 13 (drawn to a method), claim 13 is rejected the same as claim 3 and the arguments similar to that presented above for claim 3 are equally applicable to the claim 13, and all the other limitations similar to claim 3 are not repeated herein, but incorporated by reference.
Regarding claim 14, which claim 13 is incorporated, Houben discloses wherein the first machine learning model and the second machine learning model each apply a loss function [dedicated to maintaining consistency between the examination output images and the respective height maps] (Paragraph [0141] " the first model and the third model are trained so that the input of the first model closely matches the output of the third model using a loss function (e.g., a difference between images 1402 and 1406 in FIG. 14 )."; Paragraph [0142] "The training of the first model and the third model may be an iterative process. Each iteration includes inputting a SEM image of the plurality of SEM images 1301 to the first model; estimating, using the first model, the depth data for inputted SEM image; generating, via the third model using the estimated depth data as input, a predicted SEM image; and updating model parameters of both the first model and the third model to cause a difference between the inputted SEM image and the predicted SEM image to be within a specified difference threshold."); wherein the loss function includes at least a loss component configured to maintain correlation between gradients [in the one or more examination output images and the respective one or more height maps] to thereby avoid the insertion of artifacts (Paragraph [0066] "the adjusting comprises determining a gradient map of the performance function by taking a derivative of the performance function with respect to the one or more model parameters. The gradient map guides the adjustment of the one or more model parameters in a direction that minimizes or brings the performance function values within a specified threshold. For example, the specified threshold may be that the difference of the performance function values between a current iteration and a subsequent iteration is less than 1%.").
However, Houben and Shaubi both fail to teach dedicated to maintaining consistency between the examination output images and the respective height maps and [maintain correlation…] in the one or more examination output images and the respective one or more height maps.
Kim teach dedicated to maintaining consistency between the examination output images and the respective height maps and [maintain correlation…] in the one or more examination output images and the respective one or more height maps (Paragraph [0008] "The method may further include retraining the trained artificial neural network model based on the second depth map and the second simulated image."; Paragraph [0054] "the depth information may be estimated by receiving the depth information 220, inputting an output of a simulator estimating the image 210 into an ANN model, and inputting an output of the ANN model back to the simulator even in a situation in which it is difficult to obtain the depth information 220 sufficient to train the ANN model")
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Houben’s reference to include dedicated to maintaining consistency between the examination output images and the respective height maps and [maintain correlation…] in the one or more examination output images and the respective one or more height maps taught by Kim’s reference. The motivation for doing so would have been to train the model to reduce the differences between a depth map and the data obtained from the SEM image as suggested by Kim (see Kim, Paragraph [0066]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Kim with Houben and Shaubi to obtain the invention specified in claim 14.
Regarding claim 15 (drawn to a method), claim 15 is rejected the same as claim 5 and the arguments similar to that presented above for claim 5 are equally applicable to the claim 15, and all the other limitations similar to claim 5 are not repeated herein, but incorporated by reference.
Regarding claim 16 (drawn to a method), claim 16 is rejected the same as claim 6 and the arguments similar to that presented above for claim 6 are equally applicable to the claim 16, and all the other limitations similar to claim 6 are not repeated herein, but incorporated by reference.
Regarding claim 17 (drawn to a method), claim 17 is rejected the same as claim 7 and the arguments similar to that presented above for claim 7 are equally applicable to the claim 17, and all the other limitations similar to claim 7 are not repeated herein, but incorporated by reference.
Regarding claim 19 (drawn to a method), claim 19 is rejected the same as claim 10 and the arguments similar to that presented above for claim 10 are equally applicable to the claim 19, and all the other limitations similar to claim 10 are not repeated herein, but incorporated by reference.
Allowable Subject Matter
Claims 8 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 8 and 18 contain subject matter that is not disclosed or made obvious in the cited art:
In regards to claim 8, when considering claim 8 as a whole, prior art fails to disclose or render obvious, alone or in combination:
“ […] obtain one or more material maps of the semiconductor specimen; wherein a material map represents the distribution and composition of different materials across the semiconductor specimen; the one or more material maps are modified material maps that each include at least one artificially generated material defect;
apply the second machine learning (ML) model to the one or more material maps in addition to the modified height maps, to convert the one or more modified height maps and material maps to one or more respective modified synthetic examination output images that include at least one 3-dimensional defect and at least one material defect.”
In regards to claim 18, when considering claim 18 as a whole, prior art fails to disclose or render obvious, alone or in combination:
“[…] obtaining one or more material maps of the semiconductor specimen; wherein a material map represents the distribution and composition of different materials across the semiconductor specimen; the one or more material maps are modified material maps that each include at least one artificially generated material defect;
applying the second machine learning (ML) model to the one or more material maps in addition to the modified height maps to convert the one or more modified height maps and material maps to one or more respective modified synthetic examination output images that include at least one 3-dimensional defect and at least one material defect.”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kong et al. (US 2019/0259145 A1) discloses capturing SEM images and scatterometry scans from training wafers, training a machine learning model to correlate the scatterometry scans with defect type and density identified from the SEM images, and using the trained model to detect defectivity in production wafers from the scans.
Wang et al. (US 2023/0144331 A1) discloses a system for measuring surface topography of a semiconductor chip using low coherent light, beam splitters, a detector for interference signals, and a spectrometer for spectrum signals corresponding to positions on the surface of the chip.
Lin et al. (US 2019/0384236 A1) discloses an adaptive machine learning system for auto defect screening that acquires SEM-validated defect data and generates updated defect screening models using training and validation data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UROOJ FATIMA whose telephone number is (571)272-2096. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/UROOJ FATIMA/ Examiner, Art Unit 2676
/CHINEYERE WILLS-BURNS/ Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights