DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested:
SYSTEMS AND METHODS FOR ADAPTIVE PROBING OF PIECEWISE CONTINUOUS SURFACES USING A MACHINE LEARNING ALGORITHM TO PERFORM THE ADAPTIVE PROBING TO OBTAIN A RECONSTRUCTED IMAGE.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 16 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Substeps i)-iv) are present in claim 12 and not claim 11. Since the claim depends on claim 11 instead of claim 12, this is considered as having a lack of antecedent basis. It is suggested to change the dependency in claim 16 to claim 12 from claim 11. Claim 17 is rejected based on its dependency.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-4, 9, 11-14 and 18 is/are rejected under 35 U.S.C. 102(a1 and/or a2) as being anticipated by Houben (US Pub 2024/0054669 (filing date: 11/24/2021)).
Re claim 1: A system for reconstructing an image of a sample, the system comprising:
a processor; and a machine-readable medium in operable communication with the processor (e.g. the invention discloses a processor that executes instructions on a memory, which is taught in ¶ [83].) and having instructions thereon that, when executed, perform the following steps:
[0083] In an embodiment, the processes described herein may be stored in the form of instructions on a non-transitory computer-readable medium that, when executed by one or more processors, cause steps of the present methods. For example, the medium includes operations including receiving, via a metrology tool (e.g., SEM tool), a normal SEM image of a structure on a patterned substrate. The normal SEM image being associated with an e-beam directed perpendicular to a pattered substrate. Further, the medium includes operations to execute a model using the normal SEM image to determine depth information of the structure on the patterned substrate. As discussed herein, the model may be trained to estimate depth information from only a single SEM image and stored on the medium. For example, a model M1 (e.g., a CNN) may be trained using the method of FIG. 5.
receiving first data corresponding to a plurality of probe points of the sample (e.g. the invention discloses a probe that inspects a plurality of areas of an object and acquires an image in the form of the inspected object, which is taught in ¶ [190] and [191]. The invention discloses a model that is able to analyze the image data in order to determine disparity data, which is taught in ¶ [62], [63], [65] and [72].);
[0062] FIG. 3 is a block diagram of a training process of a model M0 according to an embodiment of the present disclosure. After training, a trained model may be referred as a model M1. As depicted in FIG. 3, a model M0 (e.g., CNN) receives an image 301 of a substrate as input, and outputs disparity data 311 associated with the inputted image. The inputted image 301 may be either a normal SEM image or a tilted SEM image of a patterned substrate. In the present embodiment, the term “normal image” or “normal SEM image” refers to a top-view image of a substrate that captures a first topology of the structure viewed from a top. In an embodiment, the normal image can be obtained via a metrology tool configured to direct e-beam perpendicular to the substrate. The term “tilted image” or “tilted SEM image” refers to an angled-view image of the substrate that captures a second topology of the structure viewed at an angle with respect to perpendicular to the substrate. In an embodiment, tilted image can be obtained via the metrology tool configured to direct an e-beam at an angle with respect to a substrate.
[0063] In FIG. 3, the model M0 is configured to output data that characterizes a difference between two images of the same structure. From such difference data, depth information of the structure can be determined. In an embodiment, the model M0 generates disparity data 311 that can be converted to depth information 315 of the structure on the substrate. For example, the disparity data 311 can be converted to depth information 315 by multiplying the magnitude of the disparity data 311 with a conversion function such as a scaling factor k. In an embodiment, the conversion function may be a linear function, a non-linear function, or a constant determined by comparing the result of applying the conversion function with one of the two images. In FIG. 3, the disparity data 311 is visually represented as a disparity map or image, as an example. In the present disclosure, disparity data 311 refers to a difference in coordinates of similar features within two stereo images (e.g., two SEM images of the same structure). In an embodiment, a magnitude of displacement vectors in the disparity map, is directly proportional to a depth of a structure.
[0064] For training the model M0, the disparity data 311 may be transformed, via a transform operation T1, to another image 321 that correspond to the input image 301 of the model M0. The transform operation T1 represents any transformation function that modified a tilted image 302 using the disparity data 311. For example, the transformation function can be composition, or convolution operating between the disparity data 311 and the tilted image 302 that results in another image that should correspond to the normal image.
[0065] In the present example, the disparity data 311 is combined with the tilted image 302, measured at a specified beam tilt (other than normal), to map the pixels of the tilted image 302 with the normal SEM image. In an embodiment, the tilted image 302 may be represented as a function m and the disparity data 311 (e.g., a map) may be represented as another function ϕ. By combining the disparity data 311 with the tilted image 302, a reconstructed image may be obtained. For example, the reconstructed image is represented by a function obtained from a composition of functions m and ϕ. In an embodiment, the reconstructed image may be represented as m∘ϕ, where the symbol a denotes the composition operation between functions e.g., m(x)∘ϕ(x)=m(ϕ(x)), x being a vector of xy-coordinates of an image. If the estimated disparity data 311 is accurate, then the reconstructed image is expected to be very similar to the inputted SEM image. For example, the reconstructed image is more than 95% similar to the normal SEM image inputted to the model M0. If the reconstructed image is not similar, the model M0 is modified or trained to cause the reconstructed image to be similar to the inputted SEM image. The modification of the model M0 may involve adjusting of one or more parameters (e.g., weights and biases) of the model M0 until a satisfactory reconstructed image is generated by the model. In an embodiment, the adjustment of the one or more parameters is based on a difference between the reconstructed image and the inputted SEM image. In an embodiment, a performance function may be used to guide the adjustment of the one or more parameters of the model M0, the performance function being a difference between the reconstructed image and the inputted SEM image.
[0071] According to the present disclosure, referring to FIG. 4, inventors identified, based on an experiment, that depth information of a structure patterned on a substrate can be inferred from a disparity between two or more SEM images captured at different tilt angles. For example, the experiment involved generating a depth map and feed it into a Monte-Carlo SEM simulator to compute a stereo pair of denoised SEM images: one at 0 degrees from the normal (top-view image) and one at 5.7 degrees from the normal (tilted image). An example Monte-Carlo simulator configured to generate SEM images is discussed in “L. van Kessel, and C. W. Hagen, Nebula: Monte Carlo simulator of electron-matter interaction, SoftwareX, Volume 12, 2020, 100605, ISSN 2352-7110.” In the experiment, in two SEM images, a set of points were identified and related disparity data was analyzed. For example, the set of points corresponding to step transitions in a height profile of a structure that results in a peak in the SEM signal. For the set of points, disparity data (e.g., displacement in the x-coordinate) of each point between the two SEM images was calculated. The height or depth of each point is plotted against the disparity, as shown in FIG. 4. Based on a line 401 fitted between the disparity data and the height at the set of points, it can be concluded that there is a strong correlation between the disparity between two SEM images and the height of the structure.
[0072] In an embodiment, once the model M1 is trained predictions of depth information may be performed based on a single image (e.g., SEM image) of any patterned substrate. FIG. 6A illustrates estimating of height or depth information associated with a structure using a single SEM image 351. In FIG. 6A, the model M1 (e.g., trained as in FIG. 3) uses a normal SEM image 351 of the structure as input to estimate disparity data 355. The disparity data 355 is further converted, by applying a conversion function k to the disparity data 355, to height data 360 of a structure patterned on the substrate. For example, the magnitude of the estimated disparity data can be multiplied by a constant to generate the depth information. As such, no tilted image is required to infer height information of the structure.
[0190] In an embodiment, like the electron beam inspection tool of FIG. 16 that uses a probe to inspect a substrate, the electron current in the system of FIG. 17 is significantly larger compared to, e.g., a CD SEM such as depicted in FIG. 16, such that the probe spot is large enough so that the inspection speed can be fast. However, the resolution may not be as high as compared to a CD SEM because of the large probe spot. In an embodiment, the above discussed inspection apparatus may be single beam or a multi-beam apparatus without limiting the scope of the present disclosure.
[0191] The SEM images, from, e.g., the system of FIG. 16 and/or FIG. 17, may be processed to extract contours that describe the edges of objects, representing device structures, in the image. These contours are then typically quantified via metrics, such as CD, at user-defined cut-lines. Thus, typically, the images of device structures are compared and quantified via metrics, such as an edge-to-edge distance (CD) measured on extracted contours or simple pixel differences between images.
generating a first estimate of a piecewise continuous surface based on the first data (e.g. the invention discloses training a model in order to form disparity data from the image data analyzed with a piecewise continuous function when structures are non-vertical walls or walls that are not continuously vertical. After training, the model can evaluate piecewise continuous structures in order to determine a size for reconstruction, which is taught in ¶ [68], [69] and [78].); and
[0068] Additionally, the performance function may include another loss function L.sub.prior (ϕ) determined based on prior information about the disparity characteristic of a pair of stereo SEM images. The prior information includes, but is not limited to, disparity being characterized as piecewise smooth function, disparity being characterized as piecewise constant function, or disparity being characterized as a function allowed to jump at edges of features in the normal image. For example, the edges of the features may be detected applying a gradient operation to an image (e.g., the normal image). The gradient operation identifying a sharp change in slope of an intensity profile of the image (e.g., normal image) at one or more locations, such location may be characterized as edges at which disparity may jump from one function type to another.
[0069] As mentioned earlier, based on prior stereo images of one or more prior patterned substrate a disparity function may be determined. For example, the disparity may be a piecewise smooth function, wherein a derivative of the disparity is piecewise continuous. For example, disparity related to structures having non-vertical walls will be piecewise continuous. In another example, the disparity may be piecewise constant. For example, disparity related to structures having vertical walls will be piecewise constant. In yet another example, the disparity may be a function having a jump at edges of a structure within an image, the edges being detected based a gradient of an intensity profile within the image. For example, disparity related to structures having both vertical and non-vertical walls, where the location of the jumps is per-determined from the SEM images. Thus, the present method enables incorporation of disparity data from prior patterned substrate in form of the disparity function to train the machine learning model. Use of such disparity function enables faster convergence or a faster training of the model M0 along with more accurate results.
[0078] In an embodiment, the performance function further comprises a loss function computed based on disparity characteristics associated with a pair of stereo images of prior one or more patterned substrate, and the disparity data DD1 predicted by the model M1. In an example, the disparity characteristics may include disparity characterized as a piecewise smooth function, wherein a derivative of the disparity is piecewise continuous. In another example, the disparity characteristics may include disparity being piecewise constant. In yet another example, the disparity characteristics may include disparity characterized as a function having a jump at edges of a structure within an image, the edges being detected based a gradient of an intensity profile within the image. An example performance function L is discussed above with respect to FIG. 3.
using a machine learning algorithm to perform adaptive probing on the piecewise continuous surface to obtain a reconstructed image of the sample (e.g. the model may determine the disparity between images when evaluating piecewise continuous walls, which is taught in ¶ [69] and [78] above. The disparity is used by the model to be applied to different images scanned by the probe in order to reconstruct the image of the surface, which is taught in ¶ [74]-[77].).
[0074] Process P502 involves obtaining a pair of images IM1 and IM2 of a structure of a patterned substrate. The pair of images includes a first image IM1 captured at a first angle with respect to the patterned substrate, and a second image IM2 captured at a second angle different from the first angle. As an example, the first image IM1 captures top view details of the structure, and the second image IM2 captures details of an angle-view details of the same structure from another angle (e.g., at an angle between 1 to 15° from a perpendicular to the substrate). An example of the first image IM1 can be a normal image 301 (see FIG. 3) and the second image IM2 can be a tilted image 302 (see FIG. 3). In an embodiment, the first image IM1 is a normal image associated with an e-beam directed perpendicular to the patterned substrate, and the second image IM2 is associated with an e-beam directed at an angle more 90° or less than 90° with respect to the patterned substrate. In an embodiment, a plurality of pair of images may be obtained and used as training dataset. In an embodiment, the pair of images obtained, via a metrology tool, includes a plurality of pairs of SEM images of a patterned substrate. Each pair including a first SEM image associated with a first e-beam tilt setting of the metrology tool, and a second SEM image associated with a second e-beam tilt setting of the metrology tool. In an embodiment, each image of the pair of images is captured from different SEM tools. For example, the first SEM image is captured by a first SEM tool configured to capture an normal image of a structure, and the second SEM image is captured by a second SEM tool configured to capture an image of the same structure at a second e-beam tilt setting.
[0075] Process P504 involves generating, via a model M0 using the first image IM1 as input, disparity data DD1 between the first image IM1 and the second image IM2, the disparity data DD1 being indicative of depth information associated with the first image IM1. In an embodiment, the disparity data DD1 comprises difference in coordinates of similar features within the first image IM1 and the second image IM2. In an embodiment, after the training process, the model M1 is considered as a trained model of M0. In an embodiment, the model M0 or M1 may be a machine learning model configured to predict disparity data using a single image of a substrate. For example, the model M0 or M1 may be a convolutional neural network CNN, deep CNN, or other machine learning model.
[0076] Process P506 involves applying the disparity data DD1 to the second image IM2 to generate a reconstructed image corresponding to the first image IM1. In an embodiment, the reconstructed image is generated by performing a composition operation between the disparity data DD1 and the second image IM2 to generate the reconstructed image. For example, FIG. 3 illustrates the reconstructed image 321 generated based on the disparity data (represented as the disparity map 311) and the tilted image 302.
[0077] Process P508 involves adjusting, based on a performance function one or more parameters of the model M1 causing the performance function to be within a specified performance threshold. The performance function may be a function of the disparity data DD1, the reconstructed image, and the first image IM1. The model M1 is configured to generate data convertible to depth information of a structure of a patterned substrate.
Re claim 2: Houben discloses the system according to claim 1, wherein the using of the machine learning algorithm to perform adaptive probing on the piecewise continuous surface comprises:
i) identifying, by the machine learning algorithm based on the first estimate of the piecewise continuous surface, an updated plurality of probe points of the sample (e.g. a reconstructed image is formed based on the disparity data from the first image and the second image output from the model, which is taught in ¶ [74]-[78] above. In order to update the machine learning model, the process is iterative, which is taught in ¶ [65] and [81].);
[0065] In the present example, the disparity data 311 is combined with the tilted image 302, measured at a specified beam tilt (other than normal), to map the pixels of the tilted image 302 with the normal SEM image. In an embodiment, the tilted image 302 may be represented as a function m and the disparity data 311 (e.g., a map) may be represented as another function ϕ. By combining the disparity data 311 with the tilted image 302, a reconstructed image may be obtained. For example, the reconstructed image is represented by a function obtained from a composition of functions m and ϕ. In an embodiment, the reconstructed image may be represented as m∘ϕ, where the symbol a denotes the composition operation between functions e.g., m(x)∘ϕ(x)=m(ϕ(x)), x being a vector of xy-coordinates of an image. If the estimated disparity data 311 is accurate, then the reconstructed image is expected to be very similar to the inputted SEM image. For example, the reconstructed image is more than 95% similar to the normal SEM image inputted to the model M0. If the reconstructed image is not similar, the model M0 is modified or trained to cause the reconstructed image to be similar to the inputted SEM image. The modification of the model M0 may involve adjusting of one or more parameters (e.g., weights and biases) of the model M0 until a satisfactory reconstructed image is generated by the model. In an embodiment, the adjustment of the one or more parameters is based on a difference between the reconstructed image and the inputted SEM image. In an embodiment, a performance function may be used to guide the adjustment of the one or more parameters of the model M0, the performance function being a difference between the reconstructed image and the inputted SEM image.
[0081] In an embodiment, adjusting the one or more parameters of the model M0 is an iterative process, each iteration includes determining the performance function based on the disparity data DD1, and the reconstructed image. The iteration further includes determining whether the performance function is within the specified performance threshold; and in response to the performance function not being within the specified difference threshold, adjusting the one or more parameters of the model M0 to cause the performance function to be within the specified performance threshold. The adjusting may be based on a gradient of the performance function with respect to the one or more parameters. Once the model M0 is trained, the model M1 can be applied to determine depth information of any structure based on a single image (e.g., a normal SEM image) of any patterned substrate.
ii) receiving, by the machine learning algorithm, updated data corresponding to the updated plurality of probe points of the sample (e.g. the system can receive updated or different pixels of a probe image of the object based on a different angle, which is taught in ¶ [74]-[77] above. With the reconstruction, the invention performs an iterative process that receives an updated image to compare to another in order to modify parameters of the model for a more accurate reconstruction. This is taught in ¶ ]65]-[68].);
[0065] In the present example, the disparity data 311 is combined with the tilted image 302, measured at a specified beam tilt (other than normal), to map the pixels of the tilted image 302 with the normal SEM image. In an embodiment, the tilted image 302 may be represented as a function m and the disparity data 311 (e.g., a map) may be represented as another function ϕ. By combining the disparity data 311 with the tilted image 302, a reconstructed image may be obtained. For example, the reconstructed image is represented by a function obtained from a composition of functions m and ϕ. In an embodiment, the reconstructed image may be represented as m∘ϕ, where the symbol a denotes the composition operation between functions e.g., m(x)∘ϕ(x)=m(ϕ(x)), x being a vector of xy-coordinates of an image. If the estimated disparity data 311 is accurate, then the reconstructed image is expected to be very similar to the inputted SEM image. For example, the reconstructed image is more than 95% similar to the normal SEM image inputted to the model M0. If the reconstructed image is not similar, the model M0 is modified or trained to cause the reconstructed image to be similar to the inputted SEM image. The modification of the model M0 may involve adjusting of one or more parameters (e.g., weights and biases) of the model M0 until a satisfactory reconstructed image is generated by the model. In an embodiment, the adjustment of the one or more parameters is based on a difference between the reconstructed image and the inputted SEM image. In an embodiment, a performance function may be used to guide the adjustment of the one or more parameters of the model M0, the performance function being a difference between the reconstructed image and the inputted SEM image.
[0066] In an embodiment, training of the model M0 involves adjusting model parameters to cause minimization of a performance function. At the end of the training process, the model M0 is referred as the model M1 or the trained model M1. In an embodiment, the adjusting comprises determining a gradient map of the performance function by taking a derivative of the performance function with respect to the one or more model parameters. The gradient map guides the adjustment of the one or more model parameters in a direction that minimizes or brings the performance function values within a specified threshold. For example, the specified threshold may be that the difference of the performance function values between a current iteration and a subsequent iteration is less than 1%.
[0067] In an embodiment, the performance function comprises a similarity loss L.sub.sim(ƒ, m∘ϕ) indicative of similarity between the reconstructed image and the inputted SEM image (e.g., represented as a function ƒ). In an embodiment, the similarity loss function may be computed for a plurality of images of the structure, each image obtained at different angles. For example, the performance function may be modified to include a loss function computed as a sum of similarity between an image of the plurality of images and a corresponding reconstructed image.
[0068] Additionally, the performance function may include another loss function L.sub.prior (ϕ) determined based on prior information about the disparity characteristic of a pair of stereo SEM images. The prior information includes, but is not limited to, disparity being characterized as piecewise smooth function, disparity being characterized as piecewise constant function, or disparity being characterized as a function allowed to jump at edges of features in the normal image. For example, the edges of the features may be detected applying a gradient operation to an image (e.g., the normal image). The gradient operation identifying a sharp change in slope of an intensity profile of the image (e.g., normal image) at one or more locations, such location may be characterized as edges at which disparity may jump from one function type to another.
iii) generating, by the machine learning algorithm, an updated estimate of the piecewise continuous surface based on the updated data (e.g. piecewise continuous surfaces are considered, and disparity data is estimated based on a function used to estimate disparity data based on the piecewise continuous parts of the observed surfaces, which is taught in [74]-[77] above. The disparity data is used to generate the reconstructed surface data based on the initial or new data based on the iterative process, which is taught in ¶ [65]-[68], [76] above and [87].);
[0087] Process P604 involves inputting the SEM image SEM1 to a CNN1 (an example of the model M1 trained according to the method 500) to predict disparity data 605 associated with the SEM image SEM1. In an embodiment, the CNN1 may be trained according to FIG. 5. For example, the CNN1 may be trained by: obtaining, via the SEM tool, a stereo pair of SEM images of a patterned substrate, the stereo pair including a first SEM image obtained at a first e-beam tilt setting of the SEM tool, and a second SEM image obtained at a second e-beam tilt setting of the SEM tool; generating using disparity data between the first SEM image and the second SEM image; combining the disparity data with the second SEM image to generate a reconstructed image of the first SEM image; and comparing the reconstructed image and the first SEM image.
iv) identifying, by the machine learning algorithm based on the updated estimate of the piecewise continuous surface, an updated plurality of probe points of the sample (e.g. after going through an iteration, reconstructing an image using the disparity data and updating the model, the system reconstructs another image that is represented of an updated probe image that includes reconstructed points of the object. This is taught in ¶ [65]-[69] above.); and
v) repeating substeps ii) - iv) at least once (e.g. the invention discloses performing an iterative process that changes reconstructed data based on a trained model, which is taught in ¶ [66] above.).
Re claim 3: Houben discloses the system according to claim 2, wherein substep v) comprises iteratively repeating substeps ii) - iv) until the updated data is sufficient data to generate an accurate reconstructed image (e.g. the system continues to update the reconstructed data until a difference between a reconstructed image and a comparison image is below a threshold, which is taught in ¶ [65] and [66] above.).
Re claim 4: Houben discloses the system according to claim 2, wherein substep v) comprises iteratively repeating substeps ii) - iv) a predetermined number of times, wherein the predetermined number of times is at least two (e.g. the system continues to update the reconstructed data until a difference between a reconstructed image and a comparison image is below a threshold, which is taught in ¶ [65] and [66] above. The number of iterations can be two or more.).
Re claim 9: Houben discloses the system according to claim 1, wherein the instructions when executed further perform the step of training the machine learning algorithm before receiving the first data (e.g. the system discloses applying a trained model M1 to image data captured to perform image reconstruction, which is taught in ¶ [81]-[84].).
[0081] In an embodiment, adjusting the one or more parameters of the model M0 is an iterative process, each iteration includes determining the performance function based on the disparity data DD1, and the reconstructed image. The iteration further includes determining whether the performance function is within the specified performance threshold; and in response to the performance function not being within the specified difference threshold, adjusting the one or more parameters of the model M0 to cause the performance function to be within the specified performance threshold. The adjusting may be based on a gradient of the performance function with respect to the one or more parameters. Once the model M0 is trained, the model M1 can be applied to determine depth information of any structure based on a single image (e.g., a normal SEM image) of any patterned substrate.
[0082] In an embodiment, the method 500 may further include steps for applying the trained model M1 during measurement or inspection of a patterned substrate. In an embodiment, the method 500 may include a process for obtaining, via the metrology tool, a SEM image of a patterned substrate at the first e-beam tilt setting of the metrology tool. The SEM image may be a normal image obtained by directing an e-beam approximately perpendicular to the patterned substrate. The method 500 may further include executing the model M1 using the SEM image as input to generate disparity data associated with the SEM image; and applying a conversion function (e.g., a linear function, a constant conversion factor, or a non-linear function) to the disparity data to generate depth information of the structure in the SEM image. The method 500 includes determining, based on the depth information, physical characteristics of the structure of the patterned substrate. In an embodiment, the physical characteristics may include a shape, a size, or relative positioning of polygon shapes with respect to each other at one or more depths of features of the structure.
[0083] In an embodiment, the processes described herein may be stored in the form of instructions on a non-transitory computer-readable medium that, when executed by one or more processors, cause steps of the present methods. For example, the medium includes operations including receiving, via a metrology tool (e.g., SEM tool), a normal SEM image of a structure on a patterned substrate. The normal SEM image being associated with an e-beam directed perpendicular to a pattered substrate. Further, the medium includes operations to execute a model using the normal SEM image to determine depth information of the structure on the patterned substrate. As discussed herein, the model may be trained to estimate depth information from only a single SEM image and stored on the medium. For example, a model M1 (e.g., a CNN) may be trained using the method of FIG. 5.
[0084] In an embodiment, the medium is further configured to determine, based on the depth information, physical characteristics of the structure of the patterned substrate. For example, the physical characteristics include, but not limited to, a shape, a size, or relative positioning of polygon shapes with respect to each other at one or more depth of features of the structure.
Re claim 11: Houben discloses a method for reconstructing an image of a sample, the method comprising:
receiving first data corresponding to a plurality of probe points of the sample (e.g. the invention discloses a probe that inspects a plurality of areas of an object and acquires an image in the form of the inspected object, which is taught in ¶ [190] and [191] above. The invention discloses a model that is able to analyze the image data in order to determine disparity data, which is taught in ¶ [62], [63], [65] and [72] above.);
generating a first estimate of a piecewise continuous surface based on the first data (e.g. the invention discloses training a model in order to form disparity data from the image data analyzed with a piecewise continuous function when structures are non-vertical walls or walls that are not continuously vertical. After training, the model can evaluate piecewise continuous structures in order to determine a size for reconstruction, which is taught in ¶ [68], [69] and [78] above.); and
using a machine learning algorithm to perform adaptive probing on the piecewise continuous surface to obtain a reconstructed image of the sample (e.g. the model may determine the disparity between images when evaluating piecewise continuous walls, which is taught in ¶ [69] and [78] above. The disparity is used by the model to be applied to different images scanned by the probe in order to reconstruct the image of the surface, which is taught in ¶ [74]-[77] above.).
Re claim 12: The method according to claim 11, wherein the using of the machine learning algorithm to perform adaptive probing on the piecewise continuous surface comprises:
i) identifying, by the machine learning algorithm based on the first estimate of the piecewise continuous surface, an updated plurality of probe points of the sample (e.g. a reconstructed image is formed based on the disparity data from the first image and the second image output from the model, which is taught in ¶ [74]-[78] above. In order to update the machine learning model, the process is iterative, which is taught in ¶ [65] and [81] above.);
ii) receiving, by the machine learning algorithm, updated data corresponding to the updated plurality of probe points of the sample (e.g. the system can receive updated or different pixels of a probe image of the object based on a different angle, which is taught in ¶ [74]-[77] above. With the reconstruction, the invention performs an iterative process that receives an updated image to compare to another in order to modify parameters of the model for a more accurate reconstruction. This is taught in ¶ ]65]-[68] above.);
iii) generating, by the machine learning algorithm, an updated estimate of the piecewise continuous surface based on the updated data (e.g. piecewise continuous surfaces are considered, and disparity data is estimated based on a function used to estimate disparity data based on the piecewise continuous parts of the observed surfaces, which is taught in [74]-[77] above. The disparity data is used to generate the reconstructed surface data based on the initial or new data based on the iterative process, which is taught in ¶ [65]-[68], [76] and [87] above.);
iv) identifying, by the machine learning algorithm based on the updated estimate of the piecewise continuous surface, an updated plurality of probe points of the sample (e.g. after going through an iteration, reconstructing an image using the disparity data and updating the model, the system reconstructs another image that is represented of an updated probe image that includes reconstructed points of the object. This is taught in ¶ [65]-[69] above.); and
v) repeating substeps ii) - iv) at least once (e.g. the invention discloses performing an iterative process that changes reconstructed data based on a trained model, which is taught in ¶ [66] above.).
Re claim 13: Houben discloses the method according to claim 12, wherein substep v) comprises iteratively repeating substeps ii) - iv) until the updated data is sufficient data to generate an accurate reconstructed image (e.g. the system continues to update the reconstructed data until a difference between a reconstructed image and a comparison image is below a threshold, which is taught in ¶ [65] and [66] above.).
Re claim 14: Houben discloses the method according to claim 12, wherein substep v) comprises iteratively repeating substeps ii) - iv) a predetermined number of times, wherein the predetermined number of times is at least two (e.g. the system continues to update the reconstructed data until a difference between a reconstructed image and a comparison image is below a threshold, which is taught in ¶ [65] and [66] above. The number of iterations can be two or more.).
Re claim 18: Houben discloses the method according to claim 11, further comprising training the machine learning algorithm before receiving the first data (e.g. the system discloses applying a trained model M1 to image data captured to perform image reconstruction, which is taught in ¶ [81]-[84].).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 5 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Houben in view of Common Knowledge of bias and variance (Official Notice).
Re claim 5: The system according to claim 2, wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified based on bias (e.g. the model is used to update the reconstructed image that represents the pixel points within the image based on the bias, which is taught in ¶ [65]-[67] above.).
However, Houben fails to specifically teach the feature of bias and variance.
However, this is well known in the art as evidenced by Common Knowledge of bias and variance in machine learning models. Similar to the primary reference, Common Knowledge of bias and variance in machine learning models discloses adjusting parameters of the machine learning model for accuracy (same field of endeavor or reasonably pertinent to the problem).
Common Knowledge of bias and variance in machine learning models discloses bias and variance (e.g. the machine learning model within Houben discloses adjusting the bias of the model. However, other models can be used that has a variance and bias that are to be adjusted in order to identify the image comprised of pixels that illustrate the probe points. The adjustment of the variance of a decision tree (i.e. with is type of model mentioned in Houben ¶ [169]) with the use of an ensemble of trees can lower the variance in order to have a low MSE. With this balance, more accurate predictions can occur.).
Therefore, in view of Common Knowledge of bias and variance in machine learning models, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of bias and variance, incorporated in the device of Houben, in order to use bias and variance to identify probe points of a sample in an adjusted manner, which can improve the accuracy of predictions of the model.
Re claim 15: Houben discloses the method according to claim 12, wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified based on bias (e.g. the model is used to update the reconstructed image that represents the pixel points within the image based on the bias, which is taught in ¶ [65]-[67] above.).
However, Houben fails to specifically teach the feature of bias and variance.
However, this is well known in the art as evidenced by Common Knowledge of bias and variance in machine learning models. Similar to the primary reference, Common Knowledge of bias and variance in machine learning models discloses adjusting parameters of the machine learning model for accuracy (same field of endeavor or reasonably pertinent to the problem).
Common Knowledge of bias and variance in machine learning models discloses bias and variance (e.g. the machine learning model within Houben discloses adjusting the bias of the model. However, other models can be used that has a variance and bias that are to be adjusted in order to identify the image comprised of pixels that illustrate the probe points. The adjustment of the variance of a decision tree (i.e. with is type of model mentioned in Houben ¶ [169]) with the use of an ensemble of trees can lower the variance in order to have a low MSE. With this balance, more accurate predictions can occur.).
Therefore, in view of Common Knowledge of bias and variance in machine learning models, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of bias and variance, incorporated in the device of Houben, in order to use bias and variance to identify probe points of a sample in an adjusted manner, which can improve the accuracy of predictions of the model.
Claim(s) 6-8, 16 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Houben in view of Park (NPL titled “Jump Gaussian process model for estimating piecewise continuous regression functions” Public accessibility date considered as the Receipt date: 12/1/2021).
Re claim 6: However, Houben fails to specifically teach the features of the system according to claim 2, wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP).
However, this is well known in the art as evidenced by Park. Similar to the primary reference, Park discloses the Jump Gaussian process model (same field of endeavor or reasonably pertinent to the problem). Park discloses wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP) (e.g. a Jump Gaussian Process is presented to process points of data containing jumps and discontinuities. This is used to estimate piecewise continuous regression surfaces, which is taught on page 3.).
Therefore, in view of Park, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP), incorporated in the device of Houben, in order to estimate piecewise continuous regression surfaces when updating probe points of a same using JGP, which can provide a simpler and more flexible approximation of points at a local level (as stated in Park page 3).
Re claim 7: However, Houben fails to specifically teach the features of the system according to claim 6, wherein the JGP uses mean square error (MSE).
However, this is well known in the art as evidenced by Park. Similar to the primary reference, Park discloses the Jump Gaussian process model (same field of endeavor or reasonably pertinent to the problem). Park discloses wherein the JGP uses mean square error (MSE) (e.g. the means square prediction error is used by the JGP in order to determine the performance of the process, which is taught on page 24.).
Therefore, in view of Park, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of wherein the JGP uses mean square error (MSE), incorporated in the device of Houben, in order to evaluate the models using the MSE, which allows for increasing the accuracy of the prediction of the model using adjusting parameters to optimize the MSE (as stated in Park pages 24 and 27).
Re claim 8: However, Houben fails to specifically teach the features of the system according to claim 6, wherein the JGP uses mean square prediction error (MSPE).
However, this is well known in the art as evidenced by Park. Similar to the primary reference, Park discloses the Jump Gaussian process model (same field of endeavor or reasonably pertinent to the problem). Park discloses wherein the JGP uses mean square prediction error (MSPE) (e.g. the means square prediction error is used by the JGP in order to determine the performance of the process, which is taught on page 24.).
Therefore, in view of Park, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of wherein the JGP uses mean square prediction error (MSPE), incorporated in the device of Houben, in order to evaluate the models using the MSE, which allows for increasing the accuracy of the prediction of the model using adjusting parameters to optimize the MSE (as stated in Park pages 24 and 27).
Re claim 16: However, Houben fails to specifically teach the features of the method according to claim 11, wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP).
However, this is well known in the art as evidenced by Park. Similar to the primary reference, Park discloses the Jump Gaussian process model (same field of endeavor or reasonably pertinent to the problem). Park discloses wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP) (e.g. a Jump Gaussian Process is presented to process points of data containing jumps and discontinuities. This is used to estimate piecewise continuous regression surfaces, which is taught on page 3.).
Therefore, in view of Park, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP), incorporated in the device of Houben, in order to estimate piecewise continuous regression surfaces when updating probe points of a same using JGP, which can provide a simpler and more flexible approximation of points at a local level (as stated in Park page 3).
Re claim 17: However, Houben fails to specifically teach the features of the method according to claim 16, wherein the JGP uses mean square prediction error (MSPE).
However, this is well known in the art as evidenced by Park. Similar to the primary reference, Park discloses the Jump Gaussian process model (same field of endeavor or reasonably pertinent to the problem). Park discloses wherein the JGP uses mean square prediction error (MSPE) (e.g. the means square prediction error is used by the JGP in order to determine the performance of the process, which is taught on page 24.).
Therefore, in view of Park, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of wherein the JGP uses mean square prediction error (MSPE), incorporated in the device of Houben, in order to evaluate the models using the MSE, which allows for increasing the accuracy of the prediction of the model using adjusting parameters to optimize the MSE (as stated in Park pages 24 and 27).
Claim(s) 10 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Houben in view of Kunio (US Pub 2022/0346885).
Re claim 10: Houben discloses the system according to claim 1, further comprising a display in operable communication with the processor, wherein the instructions when executed further perform the step of displaying (e.g. a display device is connected to an image processing system, which is taught in ¶ [183].).
[0183] A signal detected by secondary electron detector SED is converted to a digital signal by an analog/digital (A/D) converter ADC, and the digital signal is sent to an image processing system IPU. In an embodiment, the image processing system IPU may have memory MEM to store all or part of digital images for processing by a processing unit PU. The processing unit PU (e.g., specially designed hardware or a combination of hardware and software) is configured to convert or process the digital images into datasets representative of the digital images. Further, image processing system IPU may have a storage medium STOR configured to store the digital images and corresponding datasets in a reference database. A display device DIS may be connected with the image processing system IPU, so that an operator can conduct necessary operation of the equipment with the help of a graphical user interface.
However, Houben fails to specifically teach the features of displaying the reconstructed image of the sample on the display.
However, this is well known in the art as evidenced by Kunio. Similar to the primary reference, Kunio discloses displaying a reconstructed image (same field of endeavor or reasonably pertinent to the problem).
Kunio discloses displaying the reconstructed image of the sample on the display (e.g. the system discloses displaying reconstructed images, which is taught in ¶ [169] above.). Therefore, in view of Kunio, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of displaying the reconstructed image of the sample on the display, incorporated in the device of Houben, in order to display a reconstructed image, which improves the control of imaging techniques and knowledge of other information (as stated in Kunio ¶ [77]).
Re claim 19: However, Houben fails to specifically teach the features of the method according to claim 11, further displaying the reconstructed image of the sample on a display.
However, this is well known in the art as evidenced by Kunio. Similar to the primary reference, Kunio discloses displaying a reconstructed image (same field of endeavor or reasonably pertinent to the problem).
Kunio discloses further displaying the reconstructed image of the sample on a display (e.g. the system discloses displaying reconstructed images, which is taught in ¶ [169].).
Therefore, in view of Kunio, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of further displaying the reconstructed image of the sample on a display, incorporated in the device of Houben, in order to display a reconstructed image, which improves the control of imaging techniques and knowledge of other information (as stated in Kunio ¶ [77]).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Houben in view of Park and Kunio.
Re claim 20: Houben discloses a system for reconstructing an image of a sample, the system comprising:
a processor; a display in operable communication with the processor; and a machine-readable medium in operable communication with the processor (e.g. the invention discloses a processor that executes instructions on a memory, which is taught in ¶ [83].) and the display (e.g. a display device is connected to an image processing system, which is taught in ¶ [183] above.) and
having instructions thereon that, when executed, perform the following steps:
receiving first data corresponding to a plurality of probe points of the sample (e.g. the invention discloses a probe that inspects a plurality of areas of an object and acquires an image in the form of the inspected object, which is taught in ¶ [190] and [191] above. The invention discloses a model that is able to analyze the image data in order to determine disparity data, which is taught in ¶ [62], [63], [65] and [72] above.);
generating a first estimate of a piecewise continuous surface based on the first data (e.g. the invention discloses training a model in order to form disparity data from the image data analyzed with a piecewise continuous function when structures are non-vertical walls or walls that are not continuously vertical. After training, the model can evaluate piecewise continuous structures in order to determine a size for reconstruction, which is taught in ¶ [68], [69] and [78] above.);
using a machine learning algorithm to perform adaptive probing on the piecewise continuous surface to obtain a reconstructed image of the sample (e.g. the model may determine the disparity between images when evaluating piecewise continuous walls, which is taught in ¶ [69] and [78] above. The disparity is used by the model to be applied to different images scanned by the probe in order to reconstruct the image of the surface, which is taught in ¶ [74]-[77] above.); and
wherein the using of the machine learning algorithm to perform adaptive probing on the piecewise continuous surface comprises:
i) identifying, by the machine learning algorithm based on the first estimate of the piecewise continuous surface, an updated plurality of probe points of the sample (e.g. a reconstructed image is formed based on the disparity data from the first image and the second image output from the model, which is taught in ¶ [74]-[78] above. In order to update the machine learning model, the process is iterative, which is taught in ¶ [65] and [81] above.);
ii) receiving, by the machine learning algorithm, updated data corresponding to the updated plurality of probe points of the sample (e.g. the system can receive updated or different pixels of a probe image of the object based on a different angle, which is taught in ¶ [74]-[77] above. With the reconstruction, the invention performs an iterative process that receives an updated image to compare to another in order to modify parameters of the model for a more accurate reconstruction. This is taught in ¶ ]65]-[68] above.);
iii) generating, by the machine learning algorithm, an updated estimate of the piecewise continuous surface based on the updated data (e.g. piecewise continuous surfaces are considered, and disparity data is estimated based on a function used to estimate disparity data based on the piecewise continuous parts of the observed surfaces, which is taught in [74]-[77] above. The disparity data is used to generate the reconstructed surface data based on the initial or new data based on the iterative process, which is taught in ¶ [65]-[68], [76] and [87] above.);
iv) identifying, by the machine learning algorithm based on the updated estimate of the piecewise continuous surface, an updated plurality of probe points of the sample (e.g. after going through an iteration, reconstructing an image using the disparity data and updating the model, the system reconstructs another image that is represented of an updated probe image that includes reconstructed points of the object. This is taught in ¶ [65]-[69] above.); and
v) iteratively repeating substeps ii) - iv) until the updated data is sufficient data to generate an accurate reconstructed image (e.g. the invention discloses performing an iterative process that changes reconstructed data based on a trained model, which is taught in ¶ [66] above.).
However, Houben fails to specifically teach the features of wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP), and
wherein the JGP uses mean square prediction error (MSPE).
However, this is well known in the art as evidenced by Park. Similar to the primary reference, Park discloses the Jump Gaussian process model (same field of endeavor or reasonably pertinent to the problem). Park discloses wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP) (e.g. a Jump Gaussian Process is presented to process points of data containing jumps and discontinuities. This is used to estimate piecewise continuous regression surfaces, which is taught on page 3.), and
wherein the JGP uses mean square prediction error (MSPE) (e.g. the means square prediction error is used by the JGP in order to determine the performance of the process, which is taught on page 24.).
Therefore, in view of Park, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of wherein in substeps i) and iv), the updated plurality of probe points of the sample are identified using a jump Gaussian process (JGP), and wherein the JGP uses mean square prediction error (MSPE), incorporated in the device of Houben, in order to estimate piecewise continuous regression surfaces when updating probe points of a same using JGP, which can provide a simpler and more flexible approximation of points at a local level (as stated in Park page 3).
However, the combination above fails to specifically teach the features of displaying the reconstructed image of the sample on the display.
However, this is well known in the art as evidenced by Kunio. Similar to the primary reference, Kunio discloses displaying a reconstructed image (same field of endeavor or reasonably pertinent to the problem).
Kunio discloses displaying the reconstructed image of the sample on the display (e.g. the system discloses displaying reconstructed images, which is taught in ¶ [169] above.). Therefore, in view of Kunio, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of displaying the reconstructed image of the sample on the display, incorporated in the device of Houben, in order to display a reconstructed image, which improves the control of imaging techniques and knowledge of other information (as stated in Kunio ¶ [77]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Piestun discloses reconstruction of a probe image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAD S DICKERSON whose telephone number is (571)270-1351. The examiner can normally be reached Monday-Friday 10AM-6PM EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at 571-270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHAD DICKERSON/ Primary Examiner, Art Unit 2683