Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 16-30 rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more. The claim(s) recite(s) significantly more. The subject matter eligibility test for products and process is describe below for claim 1 in view of dependent claims.
Regarding claim 19:
Step 19: Is the claim to a process machine manufacture or composition of matter?
Yes – Claim 1 recites a method, which is a method that falls under the statutory categories.
Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes – The claim recites the following:
“ an encoder of the modular autoencoder model to encode an input to generate a low dimensional representation of the input in a latent space;” - The limitations recites a mathematical process of generating a low dimensional representation of the input in a latent space (see MPEP 2106.04(a)(2)I).
“causing the decoder of the modular autoencoder model to generate an output corresponding to the input by decoding the low dimensional representation,” - The limitations recites a mathematical process of decoding a low dimensional representation(see MPEP 2106.04(a)(2)I).
“and wherein a parameter of interest is estimated based on the output and/or the low dimensional representation of the input in the latent space”- The limitation recites a mental process of estimating a parameter of interest (see MPEP 2106.04(a)(2)III).
Step 2 Prong 2: Does the claim recite additional elements that integrate the judicial exception into a particular application? No –
The claim includes the additional element(s):
“A method for estimating, with a modular autoencoder model having an extended range of applicability, parameters of interest for optical metrology operations by enforcing known properties of inputs to the modular autoencoder model in a decoder of the modular autoencoder model, the method comprising:”
The additional elements fall under “apply it” as using a generic computer to implement a modular autoencoder. (see MPEP 2106.05(f)).
“during decoding, a known property of the encoded input to generate the output, wherein the known property is associated with a known physical relationship between the low dimensional representation in the latent space and the output,”
The additional elements fall under “apply it” as using a generic computer to implement a decoder and enforce the known properties. (see MPEP 2106.05(f)).
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
No - The claim does not include additional elements that are sufficient to amount to a significantly more than the judicial exemption. As an order whole, the claim is directed to estimating parameters of interest. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of enforcing fall under using generic computer to apply an exemption and mere data gathering. The method does not improve on the function of a computer, transforms an article into another article, nor is it applied by a particular machine, making the claim not patent eligible.
Regarding claim 20:
Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes – The claim recites the following:
“The method of claim 19, wherein enforcing comprises penalizing differences between the output and an output that should be generated according to the known property using a penalty term in a cost function associated with the decoder.” - The limitations recites a mathematical process of cost function with a penalty term (see MPEP 2106.04(a)(2)I).
Step 2A Prong 2, Step 2B: The additional element(s):
No additional elements. The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
Regarding claim 21:
Step 2A Prong 2, Step 2B: The additional element(s):
“The method of claim 20, wherein the penalty term comprises a difference between decoded versions of the low dimensional representation of the input that are related to each other through physical priors.”
The additional elements fall under Insignificant Extra-Solution Activity. See MPEP 2106.5(g). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
Regarding claim 22:
Step 2A Prong 2, Step 2B: The additional element(s):
“The method of claim 21, wherein the known property is a known symmetry property, and wherein the penalty term comprises a difference between decoded versions of the low dimensional representation of the input that are reflected across, or rotated around, a point of symmetry, relative to each other.”
The additional elements fall under Insignificant Extra-Solution Activity. See MPEP 2106.5(g). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
Regarding claim 23:
Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes – The claim recites the following:
“The method of claim 21, wherein the encoder and/or the decoder are configured to be adjusted based on any differences between the decoded versions of the low dimensional representation, wherein adjusting comprises adjusting at least one weight associated with a layer of the encoder and/or the decoder.” - The limitations recites a mathematical process of adjusting the weight of the encoder or decoder (see MPEP 2106.04(a)(2)I).
Step 2A Prong 2, Step 2B: The additional element(s):
No additional elements. The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
Regarding claim 24:
Step 2A Prong 2, Step 2B: The additional element(s):
“The method of claim 19, wherein the input comprises a sensor signal associated with a sensing operation in a semiconductor manufacturing process, the low dimensional representation of the input is a compressed representation of the sensor signal, and the output is an approximation of the input sensor signal.”
The additional elements fall under Insignificant Extra-Solution Activity. See MPEP 2106.5(g). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
Regarding claim 25:
Step 2A Prong 2, Step 2B: The additional element(s):
“The method of claim 24, wherein the sensor signal comprises a pupil image, and wherein an encoded representation of the pupil image is configured to be used to estimate overlay.”
The additional elements fall under Insignificant Extra-Solution Activity as data gathering. See MPEP 2106.5(g). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
Regarding claim 26:
Step 2A Prong 2, Step 2B: The additional element(s):
“The method of claim 19, the method further comprising: processing, with an input model of the modular autoencoder model, the input to a first level of dimensionality suitable for combination with other inputs, and providing the processed input to the encoder;”
The additional element falls under the “apply it” by using computers to process input using an input model (MPEP 2106.05(f)). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
The additional elements fall under Insignificant Extra-Solution Activity as mere data gathering. See MPEP 2106.5(g). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
“receiving, with an output model of the modular autoencoder model, an expanded version of the input from the decoder
The additional elements fall under Insignificant Extra-Solution Activity as mere data gathering. See MPEP 2106.5(g). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
and generating an approximation of the input based on the expanded version; and”
The additional element falls under the “apply it” by using computers generate an approximation of the input (MPEP 2106.05(f)). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.
“estimating, with a prediction model of the modular autoencoder model, the parameter of interest based on the low dimensional representation of the input in the latent space and/or the output.”
“The additional element falls under the “apply it” by using computers to estimate the parameter of interest using a prediction model (MPEP 2106.05(f)). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.”
Regarding claim 27:
Step 2A Prong 2, Step 2B: The additional element(s):
“ The method of claim 26, wherein the input model, the encoder/ decoder, and the output model are separate from each other and correspond to process physics differences in different parts of a manufacturing process and/or a sensing operation such that each of the input model, the encoder/decoder, and/or the output model can be trained together but individually configured based on the process physics for a corresponding part of the manufacturing process and/or sensing operation, apart from other models in the modular autoencoder model.”
“The additional element falls under the “apply it” by using computers to training the models together separately or configure them based on the manufacturing process (MPEP 2106.05(f)). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.”
Regarding claim 28:
Step 2A Prong 2, Step 2B: The additional element(s):
“The method of claim 19, wherein the decoder is configured to enforce a known symmetry property of the encoded input during a training phase, such that the modular autoencoder model obeys the enforced known symmetry property during an inference phase”
“The additional element falls under the “apply it” by using computers to training using a known symmetry process (MPEP 2106.05(f)). The judicial exemptions do not integrate into a practical application nor provide an improvement. The process does not provide an inventive concept nor provides a practical application.”
Claims 16-18 recite a computer readable medium product and are analogous to the method of claims 19-21. Therefore, the rejections of claim 19-21 above applies to claims 16-18.
Claims 21 recite system and are analogous to the method of claims 19. Therefore, the rejections of claim 19 above applies to claims 21.
Claims 30 recite a computer readable medium product and is analogous to the method of claims 19. Therefore, the rejections of claim 19 above applies to claims 19.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 16, 19, 26, 27, 29, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Bhashkar et al. (US10043261B2) (“Bhashkar”) in view of J. Yu, "Enhanced Stacked Denoising Autoencoder-Based Feature Learning for Recognition of Wafer Map Defects," in IEEE Transactions on Semiconductor Manufacturing, vol. 32, no. 4, pp. 613-624, Nov. 2019, (“Yu”).
Regarding claim 19 and analogous claim 16, 29, and 30, Bhashkar teaches A method for estimating, with a modular autoencoder model having an extended range of applicability, parameters of interest for optical metrology operations [by enforcing known properties of inputs to the modular autoencoder model in a decoder of the modular autoencoder model, the method comprising] (Bhashkar Col 1 line 49-67, Metrology processes are also used at various steps during a semiconductor manufacturing process to monitor and control the process. Metrology processes are different than inspection processes in that, unlike inspection processes in which defects are detected on specimens, metrology processes are used to measure one or more characteristics of the specimens that cannot be determined using currently used inspection tools. For example, metrology processes are used to measure one or more characteristics of specimens such as a dimension (e.g., line width, thickness, etc.) of features formed on the specimens during a process such that the performance of the process can be determined from the one or more characteristics. In addition, if the one or more characteristics of the specimens are unacceptable ( e.g., out) of a predetermined range for the characteristic(s)), the measurements of the one or more characteristics of the specimens may be used to alter one or more parameters of the process such that additional specimens manufactured by the process have acceptable characteristic(s).
Col 25 line 3-8, Nonparametric models constitute an approach to model selection and adaptation, where the sizes of models are allowed to grow with data size. This is as opposed to parametric models which use a fixed number of parameters. For example, a parametric approach to density estimation would be to fit a Gaussian or a mixture of a fixed number o
Col 27 line 55-64, In one such embodiment, the generative model includes an autoencoder variant, a generative adversarial network, a conditional generative adversarial network, or a deep generative model. For example, for learning a transformation under fixed imaging or process parameters as described above, the learning based model may be configured for a generative approach using one or more generative models including autoencoder variations, in which the decoder part will eventually be used for representation conversion, a generative adversarial network (GAN) [A method for estimating, with a modular autoencoder model having an extended range of applicability,],
Col 28 line 8-16, An autoencoder, autoassociator or Diabolo network is an artificial neural network used for unsupervised learning of efficient codings. The aim of an autoencoder is to learn a representation ( encoding) for a set of data, typically for the purpose of dimensionality reduction. Recently, the autoencoder concept has become more widely used for learning generative models of data. Architecturally, the simplest form of an autoencoder is a feedforward, non-recurrent neural network very similar to the multilayer perceptron (MLP)having an input layer, an output layer and one or more hidden layers connecting them-
Col 32 line 14-21, In another embodiment, the one or more computer subsystems are configured for classifying a defect detected in the one or more low resolution images, and the classifying is performed using the high resolution image. For example, as described above, one benefit of the optical image to SEM and/or design transformations described herein is that optical inspection is still the key for high volume production yield in semiconductor manufacturing processes [parameters of interest for optical metrology operations]):
Bhashkar does not explicitly teach [A method for estimating, with a modular autoencoder model having an extended range of applicability, parameters of interest for optical metrology operations] by enforcing known properties of inputs to the modular autoencoder model in a decoder of the modular autoencoder model, the method comprising: causing an encoder of the modular autoencoder model to encode an input to generate a low dimensional representation of the input in a latent space; and causing the decoder of the modular autoencoder model to generate an output corresponding to the input by decoding the low dimensional representation, wherein the decoder is configured to enforce, during decoding, a known property of the encoded input to generate the output, wherein the known property is associated with a known physical relationship between the low dimensional representation in the latent space and the output, and wherein a parameter of interest is estimated based on the output and/or the low dimensional representation of the input in the latent space.
However Yu teaches A method for estimating, with a modular autoencoder model having an extended range of applicability, parameters of interest for optical metrology operations] by enforcing known properties of inputs to the modular autoencoder model, the method comprising: causing an encoder of the modular autoencoder model to encode an input to generate a low dimensional representation of the input in a latent space (Page 614 -615, II. ENHANCED STACKED DENOISING AUTOENCODER A. SDAE para 2, In this section, we present the detailed information about AE, DAE and SDAE. An AE is a three-layer network including an encoder and decoder, as shown in Fig. 2(a). The encoder maps the input data from a high-dimensional space into codes in a low-dimensional space, and the decoder reconstructs the inputs from the corresponding codes [causing an encoder of the modular autoencoder model to encode an input]. Given the training data xi ∈ R, the encoder transforms the input vector x into a hidden representation hi ∈ R through a non-linear mapping:
PNG
media_image1.png
38
351
media_image1.png
Greyscale
[to generate a low dimensional representation of the input in a latent space]
where σ is the non-linear activation function. Typically, the sigmoid function σ(x) = 1/1(1 + exp(−x))(1 + exp(−x)) is used for the non-linear deterministic mapping. Then the decoder maps the hidden representation back to the original representation in a similar way as follows:
PNG
media_image2.png
28
337
media_image2.png
Greyscale
The AE training aims to optimize the parameter set θ = {W1, b1,W2, b2} to minimize the reconstruction error between z and x [by enforcing known properties of inputs to the modular autoencoder model]. One commonly adopted measure for the reconstruction error is the mean square error (MSE),);
and causing the decoder of the modular autoencoder model to generate an output corresponding to the input by decoding the low dimensional representation, wherein the decoder is configured to enforce, during decoding, a known property of the encoded input to generate the output, wherein the known property is associated with a known physical relationship between the low dimensional representation in the latent space and the output, and wherein a parameter of interest is estimated based on the output and/or the low dimensional representation of the input in the latent space (Yu Page 615 Fig. 2. (a) and (b),
PNG
media_image3.png
205
387
media_image3.png
Greyscale
Page 614 -615, II. ENHANCED STACKED DENOISING AUTOENCODER A. SDAE para 2,
Then the decoder maps the hidden representation back to the original representation in a similar way as follows:
PNG
media_image4.png
33
329
media_image4.png
Greyscale
[and causing the decoder of the modular autoencoder model to generate an output corresponding to the input by decoding the low dimensional representation,]
The AE training aims to optimize the parameter set θ = {W1, b1,W2, b2} to minimize the reconstruction error between z and x. One commonly adopted measure for the reconstruction error is the mean square error (MSE), and thus the corresponding optimization can be written:
PNG
media_image5.png
126
410
media_image5.png
Greyscale
Considering the advantages of sparse in learning effective features, the sparsity constraint is imposed on the hidden units, to regularize the objective function:
PNG
media_image6.png
63
448
media_image6.png
Greyscale
Different from AE, DAE [48] takes a corrupted version of data as input and is trained to reconstruct or denoise the original inputs from a corrupted one, which can prevent the AE from just simply learning an identity mapping between the input and the reconstructed output, and captures more informative hidden patterns and obtains robust and powerful representations from the noisy data. DAE also consists of an encoder process and a decoder process. As shown in Fig. 2(b), before encoding, the original input data x are corrupted into
x
~
by means of a stochastic mapping
x
~
= qD(
x
~
|x). DAE can generalize well and produce compounding benefits when it is stacked into a deep network [wherein the decoder is configured to enforce, during decoding, a known property of the encoded input to generate the output,]
B. ESDAE-Based Feature Learning para 3,
For each AE, the output hk is seen as a higher-order representation of the original data achieved by the feature reconstruction process, which is also defined as the decoding stack reverse reconstruction in deep learning. The decoding process is as follows:
PNG
media_image7.png
42
424
media_image7.png
Greyscale
where z, h and
θ
g
denote the output, hidden layer output, and connection parameters of the employed model, respectively. The s(.) denotes the reconstruction function, aiming to enable the output z to be equal to the input data [wherein the known property is associated with a known physical relationship between the low dimensional representation in the latent space and the output].
Page 618, D. ESDAE for Defect Detection and Recognition, After the layer-wise pretraining, all hidden representation layers are stacked, and a logistic regression layer can further be added on top of the stacked DAEs, yielding a deep architecture as shown in Fig. 6(b). The parameters of the whole deep network are first initialized by the corresponding parameters learned in the pretraining phase. After initialization softmax-based fine-tuning essentially consists of a softmax classification layer at the end of the DNN, i.e., the features derived from multiple hidden layers are set to be the input of a classifier (e.g., SVM, BPN).
III. METHODOLOGY
This section presents the methodology for using ESDAE with respect to feature learning and WMPR in wafer manufacturing process. A whole system framework is presented in Fig. 4 for detection and recognition of wafer map defects. This framework consists of two key parts, i.e., off-line system modeling and on-line running. We need to collect historic wafer maps with various defects to learn effective features by using ESDAE. ESDAE–based monitoring chart is developed for defect detection. ESDAE-based classifier is further developed to perform WMPR. The proposed scheme intents to reduce needs of human intervention and then improves applicability in real-world cases. An industrial case is used to illustrate effectiveness of the proposed system. The following subsections present these key techniques employed in this system [and wherein a parameter of interest is estimated based on the output and/or the low dimensional representation of the input in the latent space.].).
Bhashkar and Yu are considered to be analogous to the claim invention because they are in the same field of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to have modified Bhashkar to incorporate the teachings of Yu and disclose a discriminator that takes corrupted input and deconstructs the original inputs. Doing so would allow the autoencoder to capture more informative hidden patters and obtain robust and power representations (Yu page 615 II. ENHANCED STACKED DENOISING AUTOENCODER A. SDAE para 4 line 1-7, Different from AE, DAE [48] takes a corrupted version of data as input and is trained to reconstruct or denoise the original inputs from a corrupted one, which can prevent the AE from just simply learning an identity mapping between the input and the reconstructed output, and captures more informative hidden patterns and obtains robust and powerful representations from the noisy data.).
Regarding claim 26, Bhashkar and Yu teach the method of claim 19 and analogous 16, 29, and 30.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Yu further teaches the method further comprising: processing, with an input model of the modular autoencoder model, the input to a first level of dimensionality suitable for combination with other inputs, and providing the processed input to the encoder (Yu page 615 Figure 2(c),
PNG
media_image8.png
393
247
media_image8.png
Greyscale
[processing, with an input model of the modular autoencoder model]
Page 615 II. ENHANCED STACKED DENOISING AUTOENCODER A. SDAE para 5, In Fig. 2(c), several DAEs can be stacked together to form a deep network and learn high-level representations by feeding the outputs of the l-th layer as inputs to the (l + 1)-th layer [47]. SDAE is a special class of DAE where stochastic noise is added to the input layer of the AE and then trains it to output the same input without noise. In general, the training process of SDAE includes an unsupervised pre-training step and a supervised fine-tune step as shown in Fig. 2(c) and (d). The unsupervised pre-training of such an architecture is done one layer at a time. Each layer is trained as a DAE by minimizing the error in reconstructing its input that is the output code of the previous layer. Once all layers are pretrained, to minimize prediction error, the network goes through a supervised fine-tune by adding a logistic regression layer on top of the network. SDAE greatly improves the generalization performance of ANNs, and can learn more robust and compact features [the input to a first level of dimensionality suitable for combination with other inputs, and providing the processed input to the encoder;]);
receiving, with an output model of the modular autoencoder model, an expanded version of the input from the decoder and generating an approximation of the input based on the expanded version ((Page 618 Fig. 6,
PNG
media_image9.png
719
629
media_image9.png
Greyscale
[an expanded version of the input from the decoder and generating an approximation of the input based on the expanded version;]
As shown in Fig. 6(a), DAE at the bottom layer is firstly trained with the input data to obtain its hidden representations, and then, the obtained hidden representations are used as the input data for training the higher level DAE. This pretraining process is task free and focuses on the hierarchical representation learning from unlabeled data in an unsupervised manner [receiving, with an output model of the modular autoencoder model,]);
and estimating, with a prediction model of the modular autoencoder model, the parameter of interest based on the low dimensional representation of the input in the latent space and/or the output (Yu Page 618 Fig. 6(b),
PNG
media_image10.png
398
621
media_image10.png
Greyscale
[the parameter of interest based on the low dimensional representation of the input in the latent space and/or the output.]
Page 618, D. ESDAE for Defect Detection and Recognition para 4, After the layer-wise pretraining, all hidden representation layers are stacked, and a logistic regression layer can further be added on top of the stacked DAEs, yielding a deep architecture as shown in Fig. 6(b). The parameters of the whole deep network are first initialized by the corresponding parameters learned in the pretraining phase. After initialization softmax-based fine-tuning essentially consists of a softmax classification layer at the end of the DNN, i.e., the features derived from multiple hidden layers are set to be the input of a classifier (e.g., SVM, BPN) [estimating, with a prediction model of the modular autoencoder model,].)).
Regarding claim 27, Bhashkar and Yu teach the method of claim 19 and analogous 26.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Yu further teaches wherein the input model, the encoder/decoder, and the output model are separate from each other and correspond to process physics differences in different parts of a manufacturing process and/or a sensing operation such that each of the input model, the encoder/decoder, and/or the output model can be trained together but individually configured based on the process physics for a corresponding part of the manufacturing process and/or sensing operation, apart from other models in the modular autoencoder model (Yu Page 618 C. Class Imbalance A common problem in real-world applications of DNNs is that some classes have a significantly higher number of examples in the training set than other classes. This significant difference is referred to as class imbalance. It affects both learning in the training phase and prediction of a model on the test set. These is also a class imbalance problem exiting in the used wafer dataset (i.e., WM-811K). The WM-811K dataset consists of the eight defect patterns (i.e., Center, Edge-ring, Edge-local, Random, Local, Scratch, Near-full and Donut) and normal class. It is clear that the class label imbalance exists in this dataset. The Near-full class consists of only 56 samples, while the Edge-ring class consists of 9056 samples. This will bring a big challenge for ESDAE in the defect map recognition. Methods of dealing with class imbalance are well studied for those typical learning models [55], [56], which can be divided into two main categories, i.e., data level and classifier level methods. The data level methods operate on training dataset and change its class distribution. The classifier level methods keep the training data unchanged and adjust training or inference algorithms.
Page 618 D. ESDAE for Defect Detection and Recognition
For each new input, the output of ESDAE quantifies the deviation level of the input vector with the normal data distribution space described by the ESDAE (i.e., ESDAE0) that is constructed by the normal and various defect data.
In general, the setup of confidence level of monitoring charts should consider the balance between Type I and Type II errors to meet the real-world requirements. If the type I error is high, the type II error will be low, while it will be high. In this study, the confidence bound 99% (i.e., the Type I error is 1%) is used to setup the threshold δ [wherein the input model, the encoder/decoder, and the output model are separate from each other and correspond to process physics differences in different parts of a manufacturing process and/or a sensing operation such that each of the input model, the encoder/decoder,].
Page 616,
PNG
media_image11.png
308
497
media_image11.png
Greyscale
[and/or the output model can be trained together but individually configured based on the process physics for a corresponding part of the manufacturing process and/or sensing operation, apart from other models in the modular autoencoder model.]
page 616, III. Methodology, This section presents the methodology for using ESDAE with respect to feature learning and WMPR in wafer manufacturing process. A whole system framework is presented in Fig. 4 for detection and recognition of wafer map defects. This framework consists of two key parts, i.e., off-line system modeling and on-line running. We need to collect historic wafer maps with various defects to learn effective features by using ESDAE. ESDAE–based monitoring chart is developed for defect detection. ESDAE-based classifier is further developed to perform WMPR. The proposed scheme intents to reduce needs of human intervention and then improves applicability in real-world cases. An industrial case is used to illustrate effectiveness of the proposed system. The following subsections present these key techniques employed in this system.).
Claim(s) 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Bhashkar in view of Yu and further in view of Shao, Huajie, et al. "Controlvae: Controllable variational autoencoder." International conference on machine learning. PMLR, 2020 (“Shao”).
Regarding claim 20 and analogous claim 17, Bhashkar and Yu teach the method of claim 19 and analogous 16, 29, and 30.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Bhashkar does not explicitly teach wherein enforcing comprises penalizing differences between the output and an output that should be generated according to the known property using a penalty term in a cost function associated with the decoder.
However Shao teaches wherein enforcing comprises penalizing differences between the output and an output that should be generated according to the known property using a penalty term in a cost function associated with the decoder (Shao Page 3,
PNG
media_image12.png
241
563
media_image12.png
Greyscale
PNG
media_image13.png
66
542
media_image13.png
Greyscale
PNG
media_image14.png
257
555
media_image14.png
Greyscale
[a penalty term in a cost function associated with the decoder]).
Bhashkar and Shao are considered to be analogous to the claim invention because they are in the same field of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to have modified Bhashkar to incorporate the teachings of Shao and disclose a using a penalty term in a cost function. Doing so would allow the autoencoder to achieve much better reconstruction quality (Shao Abstract line 16-29, VAE objective using the output KL-divergence as feedback during model training. The framework is evaluated using three applications; namely, language modeling, disentangled representation learning, and image generation. The results show that ControlVAE can achieve much better reconstruction quality than the competitive methods for the comparable disentanglement performance. For language modeling, it not only averts the KL-vanishing, but also improves the diversity of generated text. Finally, we also demonstrate that ControlVAE improves the reconstruction quality for image generation compared to the original VAE.).
Claim(s) 18, 21, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Bhashkar in view of Yu and Shao and further in view of Guen, Vincent Le, and Nicolas Thome. "Disentangling physical dynamics from unknown factors for unsupervised video prediction." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.
Regarding claim 21 and analogous claim 18, Bhashkar in view of Yu and Shao teach the method of claim 20 and analogous claim 17.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Bhashkar and Shao are combined in the same rationale as in claim 19 and analogous 21 and analogous claim 21.
Bhashkar does not explicitly teach wherein the penalty term comprises a difference between decoded versions of the low dimensional representation of the input that are related to each other through physical priors.
However Guen teaches wherein the penalty term comprises a difference between decoded versions of the low dimensional representation of the input that are related to each other through physical priors (Guen Page 11478,
PNG
media_image15.png
253
486
media_image15.png
Greyscale
PNG
media_image16.png
251
465
media_image16.png
Greyscale
[wherein the penalty term comprises a difference between decoded versions of the low dimensional representation]
Page 11480, Influence of physical regularization We conduct in Table 3 a finer ablation on Moving MNIST to study the impact of the physical regularization Lmoment on the performance of PhyCell and PhyDNet. When we disable Lmoment for training PhyCell, performances improve by 7 points in MSE. This underlines that physical laws alone are too restrictive for learning dynamics in a general context, and that complementary factors should be accounted for. On the other side, when we disable Lmoment for training our disentangled architecture PhyDNet, performances decrease by 5 MSE points (29 vs 24.4) compared to the physically-constrained version. This proves that physical constraints are relevant, but should be incorporated carefully in order to make both branches cooperate. This enables to leverage physical prior, while keeping remaining information necessary for pixel level prediction. Same conclusions can be drawn for the other datasets, see supplementary 2.6 [of the input that are related to each other through physical priors.]).
Bhashkar and Guen are considered to be analogous to the claim invention because they are in the same field of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to have modified Bhashkar to incorporate the teachings of Guen and disclose a using physical priors. Doing so would allow disentangling prior dynamical knowledge from other factors of variation (Guen page 11481, 5. Conclusion line 1-6, We propose PhyDNet, a new model for disentangling prior dynamical knowledge from other factors of variation required for video prediction. PhyDNet enables to apply PDE-constrained prediction beyond fully observed physical phenomena in pixel space, and to outperform state-of-the-art performances on four generalist datasets.)
Regarding claim 23, Bhashkar, Yu, Sha and Guen teach the method of claim 21 and analogous 18.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Bhashkar and Shao are combined in the same rationale as in claim 20 and analogous claim 17.
Bhashkar and Guen are combined in the same rationale as in claim 21 and analogous claim 18.
Yu further teaches wherein the encoder and/or the decoder are configured to be adjusted based on any differences between the decoded versions of the low dimensional representation, wherein adjusting comprises adjusting at least one weight associated with a layer of the encoder and/or the decoder (Yu page 615, II. ENHANCED STACKED DENOISING AUTOENCODER A. SDAE para 3 line 10-16, Therefore, the variant of AE is named SAE. The corresponding optimization function is updated by minimizing the following function:
PNG
media_image17.png
29
386
media_image17.png
Greyscale
The weights and bias of the network can be obtained by using back-propagation method [47], where the gradient descent algorithm is used to compute the W and b [wherein adjusting comprises adjusting at least one weight associated with a layer of the encoder and/or the decoder]).
Claim(s) 22 are rejected under 35 U.S.C. 103 as being unpatentable over Bhashkar in view of Yu and Shao and further in view of Guen and Wu, Shangzhe, Christian Rupprecht, and Andrea Vedaldi. "Unsupervised learning of probably symmetric deformable 3d objects from images in the wild." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020 (“Wu”).
Regarding claim 22, Bhashkar in view of Yu, Shao and Guen teach the method of claim 21 and analogous claim 18.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Bhashkar and Shao are combined in the same rationale as in claim 20 and analogous claim 17.
Bhashkar and Guen are combined in the same rationale as in claim 21 and analogous claim 18.
Bhashkar does not explicitly teach wherein the known property is a known symmetry property, and wherein the penalty term comprises a difference between decoded versions of the low dimensional representation of the input that are reflected across, or rotated around, a point of symmetry, relative to each other.
However Wu teaches wherein the known property is a known symmetry property, and wherein the penalty term comprises a difference between decoded versions of the low dimensional representation of the input that are reflected across, or rotated around, a point of symmetry, relative to each other (Huang Page 3, 3. Method para 1 and 3, Given an unconstrained collection of images of an object category, such as human faces, our goal is to learn a model
φ
that receives as input an image of an object instance and produces as output a decomposition of it into 3D shape, albedo, illumination and viewpoint, as illustrated in Fig. 2.
In order to learn such a decomposition without supervision for any of the components, we use the fact that many object categories are bilaterally symmetric. However, the appearance of object instances is never perfectly symmetric. Asymmetries arise from shape deformation, asymmetric albedo and asymmetric illumination. We take two measures to account for these asymmetries. First, we explicitly model asymmetric illumination. Second, our model also estimates, for each pixel in the input image, a confidence score that explains the probability of the pixel having a symmetric counterpart in the image (see conf
σ
,
σ
'
in Fig. 2) [wherein the known property is a known symmetry property].
Page 4 3.2. Probably symmetric objects para 1, para 2 line 1-6, and para 5
Leveraging symmetry for 3D reconstruction requires identifying symmetric object points in an image. Here we do so implicitly, assuming that depth and albedo, which are reconstructed in a canonical frame, are symmetric about a fixed vertical plane. An important beneficial side effect of this choice is that it helps the model discover a ‘canonical view’ for the object, which is important for reconstruction [41].
To do this, we consider the operator that flips a map
a
∈
R
C
x
W
x
H
along the horizontal
a
x
i
s
2
:
f
l
i
p
a
c
,
u
,
v
=
a
c
,
w
-
1
-
u
,
b
. We then require
d
≈
f
l
i
p
d
'
and
a
≈
f
l
i
p
a
'
. While these constraints could be enforced by adding corresponding loss terms to the learning objective, they would be difficult to balance.
PNG
media_image18.png
206
563
media_image18.png
Greyscale
[and wherein the penalty term comprises a difference between decoded versions of the low dimensional representation of the input that are reflected across, or rotated around, a point of symmetry, relative to each other]).
Bhashkar and Wu are considered to be analogous to the claim invention because they are in the same field of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to have modified Bhashkar to incorporate the teachings of Wu and use symmetry to determine a penalty. Doing so would allow the use of a symmetry probability map and learn end -to-end and recover data accurately for 3d shapes (Wu Abstract We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision. The method is based on an autoencoder that factors each input image into depth, albedo, viewpoint and illumination. In order to disentangle these components without supervision, we use the fact that many object categories have, at least in principle, a symmetric structure. We show that reasoning about illumination allows us to exploit the underlying object symmetry even if the appearance is not symmetric due to shading. Furthermore, we model objects that are probably, but not certainly, symmetric by predicting a symmetry probability map, learned end-to-end with the other components of the model. Our experiments show that this method can re- cover very accurately the 3D shape of human faces, cat faces and cars from single-view images, without any supervision or a prior shape model. On benchmarks, we demonstrate superior accuracy compared to another method that uses supervision at the level of 2D image correspondences.).
Claim(s) 18 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Bhashkar in view of Yu and further in view of Maggipinto, Marco, et al. "DeepVM: A Deep Learning-based approach with automatic feature extraction for 2D input data Virtual Metrology." Journal of Process Control 84 (2019): 24-34 (“Maggipinto”).
Regarding claim 24, Bhashkar in view of Yu and Bhashkar teach the method of claim 19 and analogous 16, 29, and 30.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Bhashkar teaches wherein the input comprises a sensor signal associated with a sensing operation in a semiconductor manufacturing process (Bhashkar Fig. 1,
PNG
media_image19.png
754
614
media_image19.png
Greyscale
Col 15 line 55-64 and Col 16 1-5, The component(s), e.g., component(s) 100 shown in FIG. 1, executed by the computer subsystem(s), e.g., computer subsystem 36 and/or computer subsystem(s) 102, include learning based model 104. The learning based model is configured for mapping a triangular relationship between optical images, electron beam images, and design data. For example, as shown in FIG. 2, the learning based model may be configured for mapping a triangular relationship between three different spaces: design 200, electron beam 202, and optical 204. The embodiments described herein, therefore, provide a generalized representation mapping system between optical, electron beam, and design (e.g., CAD) for semiconductor wafers and masks for inspection, metrology and other use cases. The embodiments described herein, therefore, also provide a systematic approach to perform transformations between different observable representations of specimens such as wafers and reticles in semiconductor process control applications. [wherein the input comprises a sensor signal associated with a sensing operation in a semiconductor manufacturing process]),
Bhashkar does not explicitly teach the low dimensional representation of the input is a compressed representation of the sensor signal, and the output is an approximation of the input sensor signal.
However Maggipinto teaches the low dimensional representation of the input is a compressed representation of the sensor signal, and the output is an approximation of the input sensor signal (Maggipinto Page 28,
PNG
media_image20.png
260
403
media_image20.png
Greyscale
[the low dimensional representation of the input is a compressed representation of the sensor signal,]
Page 29 4.1 Semiconductor Manufacturing Case Study,
We propose using the DeepVM algorithm described in the previous sections to build such a VM solution. In particular, different structures corresponding to different choices of the autoencoder and the regression module will be analyzed and their performance compared using a case study provided by an industrial partner involved in the manufacture of storage media. The case study dataset consists of OES spectra and associated etch rate values for N = 1554 wafers processed through a single etch chamber. The OES data, which serves as the VM model input, has a complex 2-dimensional structure, with time and wavelength evolution, as depicted in Figs. 11 and 12 . The 2-dimensional structure of OES suggests the use of Computer Vision inspired technologies, thus motivating the use of models based on CNNs, that have outperformed other methods in many Computer Vision tasks [45] . CNNs are able to extract hierarchical sparse features [36] from complex data like images. As such, the proposed method is expected to provide a powerful feature extraction model for OES data.).
Bhashkar and Maggipinto are considered to be analogous to the claim invention because they are in the same field of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to have modified Bhashkar to incorporate the teachings of Maggipinto and disclose using an autoencoder to process OES spectra. Doing so would allow the autoencoder to extract hierarchical sparse features from complex data like images and extract powerful features from OES data (Maggipinto Page 29 4.1 Semiconductor Manufacturing Case Study, We propose using the DeepVM algorithm described in the previous sections to build such a VM solution. In particular, different structures corresponding to different choices of the autoencoder and the regression module will be analyzed and their performance compared using a case study provided by an industrial partner involved in the manufacture of storage media. The case study dataset consists of OES spectra and associated etch rate values for N = 1554 wafers processed through a single etch chamber. The OES data, which serves as the VM model input, has a complex 2-dimensional structure, with time and wavelength evolution, as depicted in Figs. 11 and 12 . The 2-dimensional structure of OES suggests the use of Computer Vision inspired technologies, thus motivating the use of models based on CNNs, that have outperformed other methods in many Computer Vision tasks [45] . CNNs are able to extract hierarchical sparse features [36] from complex data like images. As such, the proposed method is expected to provide a powerful feature extraction model for OES data.).
Claim(s) 24 are rejected under 35 U.S.C. 103 as being unpatentable over Bhashkar in view of Yu and further in view of Maggipinto and Fu et al. (US20160123894A1).
Regarding claim 25 , Bhashkar in view of Yu and Maggipinto teach the method of claim 24.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Bhashkar and Maggipinto are combined in the same rationale as in claim 24.
Bhashkar does not explicitly teach wherein the sensor signal comprises a pupil image, and wherein an encoded representation of the pupil image is configured to be used to estimate overlay.
However Fu teaches wherein the sensor signal comprises a pupil image, and wherein an encoded representation of the pupil image is configured to be used to estimate overlay (Fu para 0040, In one example, an imaging detector is employed to perform pupil image measurements ( e.g., pupil imaging detector 117) and an imaging detector is employed to simultaneously perform field image measurements ( e.g., field imaging detector 114). Depending on the desired parameter to be measured, both field and pupil images of one or more known film specimens or grating specimens are collected for a series of different z-positions. For each z-position, computing system 130 determines the intensity distribution of the collected field and pupil images [wherein the sensor signal comprises a pupil image,].
para 0044, In some examples, field images acquired, for example, by field imaging detector 114, are processed by computing system 130 to identify the boundary of the measurement target with sub-pixel accuracy. Based on the identified boundary, computing system 130 determines the center of the measurement target. During overlay measurement, the estimate of target location is employed to select particular pixel signals for analysis and reduce the required target placement accuracy and corresponding alignment effort. During CD measurements and thin film measurements, the estimate of target location is employed to determine whether the measurement target is centered within the placement tolerance. If not, the associated pupil images are marked with the location offset determined based on the field images. Based on the measured location offset, computing system 130 communicates command signals, for example, to a wafer positioning system, to reposition specimen 107 in any of the x and y directions, and locate the measurement target within the placement tolerance [and wherein an encoded representation of the pupil image is configured to be used to estimate overlay.]).
Bhashkar and Fu are considered to be analogous to the claim invention because they are in the same field of semiconductor manufacturing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to have modified Bhashkar to incorporate the teachings of Fu and encoded representation is used to estimate overly. Doing so would allow proper placement to a target location based on the placement tolerance (Fu para 0044 line 9-19, During CD measurements and thin film measurements, the estimate of target location is employed to determine whether the measurement target is centered within the placement tolerance. If not, the associated pupil images are marked with the location offset determined based on the field images. Based on the measured location offset, computing system 130 communicates command signals, for example, to a wafer positioning system, to reposition specimen 107 in any of the x and y directions, and locate the measurement target within the placement tolerance.).
Claim(s) 28 are rejected under 35 U.S.C. 103 as being unpatentable over Bhashkar in view of Yu and further in view of Wu, Shangzhe, Christian Rupprecht, and Andrea Vedaldi. "Unsupervised learning of probably symmetric deformable 3d objects from images in the wild." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020 (“Wu”).
Regarding claim 28, Bhashkar and Yu teach the method of claim 19 and analogous 16, 29, and 30.
Bhashkar and Yu are combined in the same rationale as in claim 19 and analogous 16, 29, and 30.
Bhashkar does not explicitly teach wherein the decoder is configured to enforce a known symmetry property of the encoded input during a training phase, such that the modular autoencoder model obeys the enforced known symmetry property during an inference phase.
However Wu teaches wherein the decoder is configured to enforce a known symmetry property of the encoded input during a training phase, such that the modular autoencoder model obeys the enforced known symmetry property during an inference phase (Huang Page 3, 3. Method para 1 and 3, Given an unconstrained collection of images of an object category, such as human faces, our goal is to learn a model
φ
that receives as input an image of an object instance and produces as output a decomposition of it into 3D shape, albedo, illumination and viewpoint, as illustrated in Fig. 2.
In order to learn such a decomposition without supervision for any of the components, we use the fact that many object categories are bilaterally symmetric. However, the appearance of object instances is never perfectly symmetric. Asymmetries arise from shape deformation, asymmetric albedo and asymmetric illumination. We take two measures to account for these asymmetries. First, we explicitly model asymmetric illumination. Second, our model also estimates, for each pixel in the input image, a confidence score that explains the probability of the pixel having a symmetric counterpart in the image (see conf
σ
,
σ
'
in Fig. 2).
Page 4 3.2. Probably symmetric objects para 1, para 2 line 1-6, and para 5
Leveraging symmetry for 3D reconstruction requires identifying symmetric object points in an image. Here we do so implicitly, assuming that depth and albedo, which are reconstructed in a canonical frame, are symmetric about a fixed vertical plane. An important beneficial side effect of this choice is that it helps the model discover a ‘canonical view’ for the object, which is important for reconstruction [41].
To do this, we consider the operator that flips a map
a
∈
R
C
x
W
x
H
along the horizontal
a
x
i
s
2
:
f
l
i
p
a
c
,
u
,
v
=
a
c
,
w
-
1
-
u
,
b
. We then require
d
≈
f
l
i
p
d
'
and
a
≈
f
l
i
p
a
'
. While these constraints could be enforced by adding corresponding loss terms to the learning objective, they would be difficult to balance.
PNG
media_image18.png
206
563
media_image18.png
Greyscale
[wherein the decoder is configured to enforce a known symmetry property of the encoded input during a training phase]
Page 6, 4.2. Results Comparison with baselines. Table 2 uses the BFM dataset to compare the depth reconstruction quality obtained by our method, a fully-supervised baseline and two baselines. The supervised baseline is a version of our model trained to regress the ground-truth depth maps using an L1 loss. The trivial baseline predicts a constant uniform depth map, which provides a performance lower-bound. The third baseline is a constant depth map obtained by averaging all ground-truth depth maps in the test set. Our method largely outperforms the two constant baselines and approaches the results of supervised training. Improving over the third baseline (which has access to GT information) confirms that the model learns an instance specific 3D representation.
Page 7 Qualitative results. In Fig. 4 we show reconstruction results of human faces from CelebA and 3DFAW, cat faces from [66, 42] and synthetic cars from ShapeNet. The 3D shapes are recovered with high fidelity. The reconstructed 3D face, for instance, contain fine details of the nose, eyes and mouth even in the presence of extreme facial expression [such that the modular autoencoder model obeys the enforced known symmetry property during an inference phase].).
Bhashkar and Wu are considered to be analogous to the claim invention because they are in the same field of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filling date of the claimed invention to have modified Bhashkar to incorporate the teachings of Wu and use symmetry to determine a penalty. Doing so would allow the use of a symmetry probability map and learn end -to-end and recover data accurately for 3d shapes (Wu Abstract We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision. The method is based on an autoencoder that factors each input image into depth, albedo, viewpoint and illumination. In order to disentangle these components without supervision, we use the fact that many object categories have, at least in principle, a symmetric structure. We show that reasoning about illumination allows us to exploit the underlying object symmetry even if the appearance is not symmetric due to shading. Furthermore, we model objects that are probably, but not certainly, symmetric by predicting a symmetry probability map, learned end-to-end with the other components of the model. Our experiments show that this method can re- cover very accurately the 3D shape of human faces, cat faces and cars from single-view images, without any supervision or a prior shape model. On benchmarks, we demonstrate superior accuracy compared to another method that uses supervision at the level of 2D image correspondences.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALFREDO CAMPOS whose telephone number is (571)272-4504. The examiner can normally be reached 7:00 - 4:00 pm M - F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALFREDO CAMPOS/ Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights