DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-16 and 19-27are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Samer BANNA (US 20200110390 A1), hereinafter BANNA.
Regarding claim 1
BANNA discloses
A computer program product for adaptive model training, the computer program product comprising a non-transitory computer readable medium on which is provided computer-executable instructions for
(BANNA, p. 10, [0095] “The present disclosure may be provided as a computer program product, or software, that may include a machine-readable medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to the present disclosure.”)
(BANNA, p. 10, [0085] “The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a web appliance, a server, a network router, a switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.”)
receiving, from a plurality of process chambers ex situ data associated with wafers fabricated using the plurality of process chambers and in situ measurements
(BANNA, p. 1, [0007] “… Specifically, a computer-implemented method is described, where, for each current chamber in a multi-chamber processing platform, a spatial model for a wafer is obtained.”
(BANNA, p. 1, [0010] “FIG. 1B is a block diagram of a machine-learning based spatial model generator, according to an embodiment of this disclosure;”)
(BANNA, p. 4, [0050] “FIG. 1B shows a system 100 that outputs a final spatial model 112. Input to the system 100 includes characterization data from physical DoE wafers using recipes around a baseline recipe. Characterization data includes on-tool metrology data 101, including VM raw data 103a and OBM raw data 103b, as well as off-tool data 102, including in-line metrology data 104a and non-inline metrology data 104b.”)
(BANNA, p. 3, [0036] “… These optical or other measurements can be inside the chamber (in-situ), or outside the chamber (ex-situ), but still under vacuum, or, at the factory interface (not necessarily under vacuum) on a process platform that may have multiple chambers.”)
(BANNA, p. 3, [0052] “… In addition to metrology data, the machine-learning engine 108 also receives information 111 about various recipes and knobs, as well as information 113 about the process and the equipment.”)
wherein the plurality of process chambers use a first machine learning model for process control during fabrication of wafers by the plurality of process chambers
(BANNA, p. 1, [0007] “… The spatial model is created by a first machine-learning engine based on a first set of metrology data on one or more dimensions of interest in the device. The spatial model can be global, covering physical behavior of the process, or could be chamber-specific, accounting for chamber variability.”)
(BANNA, p. 4, [0051] “The characterization data is then fed to the machine-learning engine 108. The data is filtered by an additional data filtering and features extraction module 106 that precedes the machine-learning engine 108. The module 106 is a key module that extracts meaningful features from the data set and draw inference to optimize machine-learning engine performance.”)
(BANNA, p. 2, [0025] “FIG. 8 shows application of the model for wafer-to-wafer process control, according to an embodiment of the present disclosure;”)
wherein the first machine learning model is used to predict the ex situ data using the in situ measurements, and wherein the ex situ data for a wafer indicates a characteristic of the wafer post-fabrication
(BANNA, p. 1, [0011] “FIG. 2 and FIG. 3 show two fundamental capabilities of the model, predicting spatial measurement and recommending recipes, according to embodiments of the present disclosure;”)
(BANNA, p. 5, [0055] “FIG. 2 and FIG. 3 show the two basic capabilities of the machine-learning based model. When metrology data 202 is used to generate final spatial model(s) 112, the machine-learning based model can predict spatial dimensions of interest 215 based on various process recipes and control knob information 211.”)
calculating a metric indicating an error associated with the first machine learning model using the ex situ data from the plurality of process chambers
(BANNA, p. 1, [0007] “Aspects of the disclosure describe a method and a corresponding system for controlling chamber-to-chamber variability during manufacturing of a device on wafers. Specifically, a computer-implemented method is described, where, for each current chamber in a multi-chamber processing platform, a spatial model for a wafer is obtained. The spatial model is created by a first machine-learning engine based on a first set of metrology data on one or more dimensions of interest in the device. The spatial model can be global, covering physical behavior of the process, or could be chamber-specific, accounting for chamber variability.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric.”)
determining whether to update the first machine learning model based on the metric indicating the error
(BANNA, p. 7, [0075] “FIG. 9 shows how to adaptively adjust the wafer-to-wafer control model after each lot. Specifically, FIG. 9 shows a block diagram representing the adaptive process for model update.”)
(BANNA, p. 1, “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric.”)
and in response to determining that the first machine learning model is to be updated, generating a second machine learning model using the ex situ data and the in situ measurements received from the plurality of process chambers
(BANNA, p. 1, [0007] “… The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber.”)
(BANNA, p. 7, [0071] “FIG. 7D shows various information utilized to create a multi-input multi-output adaptive empirical process control model 760 (also referred to as “adaptive model”). “Multi-input” refers to metrology data about various dimensions of interest, and “multi-output” refers to various spatial profiles predicted by the model. To implement the adaptive feature, actual DoE wafers are characterized by VM data (shown as block 750), OBM data (shown as block 752) and in-line metrology data (shown as block 754). Other data (shown as block 756), such as TEM data, data available from customers or other sources, device parametric and yield data from the electrical test, may also be fed to the model. In addition, chamber information and process information (shown collectively as block 758) are used to create the adaptive model.”)
wherein the first machine learning model and the second machine learning model are evaluated using a test set that includes ex situ data collected before the determination that the first machine learning model is to be updated, and ex situ data collected after the determination that the first machine learning model is to be updated.
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “…The machine-learning engine 108 then generates intermediate spatial model 109 for each measurement on the wafer. Each measurement can have data about one or more dimensions of interest. The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. … Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
(BANNA, p. 2, [0021] “FIG. 7B shows correlation between various test and measurement techniques and device performance, in accordance with embodiments of the present disclosure;”)
(BANNA, p. 3, [0036] “… Collectively, on-tool metrology performed on devices or test structures is referred to as “on-board metrology” (OBM). OBM can be based on optical measurements (e.g., collecting optical emission spectra in-situ from devices or test structures, or macro 2D mapping using optical targets) or other types of measurements. These optical or other measurements can be inside the chamber (in-situ), or outside the chamber (ex-situ), but still under vacuum, or, at the factory interface (not necessarily under vacuum) on a process platform that may have multiple chambers. One such example is integrated metrology, such as optical critical dimension (OCD).”)
(BANNA, p. 7, [0075] “… Specifically, FIG. 9 shows a block diagram representing the adaptive process for model update. In block 902 and 904, VM raw data and OBM raw data respectively, are collected from the one or more samples of the current lot of wafers (
L
n
-
1
). In-line metrology data is collected for the current lot in block 906 by sampling a few wafers in the current lot. All the collected data goes through the data filtering and features extraction module 910. The spatial model 912 creates a spatial predicted measurement 914, which is fed to the machine-learning engine 918. The machine-learning engine 918 compares predictive measurement to a reference measurement 916 provided by in-line or other off-tool metrology. Process recipe 920 and process and equipment information 922 are provided to the machine-learning engine. An updated spatial model is created in block 924 to be used for controlling the next lot (
L
n
). “)
Regarding claim 2
BANNA teaches all aspects of claim 1 as disclosed above and further discloses
The computer program product of claim 1, wherein the ex situ data is ex situ metrology data measured post-fabrication for a subset of fabricated wafers.
(BANNA, p. 1, [0007] “… The spatial model is created by a first machine-learning engine based on a first set of metrology data on one or more dimensions of interest in the device.”)
(BANNA, p. 3, [0036] “… Collectively, on-tool metrology performed on devices or test structures is referred to as “on-board metrology” (OBM). OBM can be based on optical measurements (e.g., collecting optical emission spectra in-situ from devices or test structures, or macro 2D mapping using optical targets) or other types of measurements. These optical or other measurements can be inside the chamber (in-situ), or outside the chamber (ex-situ), but still under vacuum, or, at the factory interface (not necessarily under vacuum) on a process platform that may have multiple chambers.”)
(BANNA, p. 3, [0042] “FIG. 1A is a flow diagram of an example method 10 to enable creation and use of a spatial model, in accordance with some embodiments of the present disclosure.”)
(BANNA, p. 3, [0043] “Referring back to FIG. 1A, at block 20, the process is profiled. Process profiling involves understanding the recipe structure to achieve certain dimensions and profiles of a device fabricated on a wafer.”)
Regarding claim 3
BANNA teaches all aspects of claim 1 as disclosed above and further discloses
The computer program product of claim 1, wherein the ex situ data includes geometric information related to features of a wafer.
(BANNA, p. 3, [0041] “… The dimensions of interest may be geometrical dimensions of physical features on the wafer, e.g., a line width, a height of a structure, a sidewall angle, a top critical dimension (TCD), a bottom critical dimension (BCD) or any other feature-level three-dimensional profile information.”)
Regarding claim 4
BANNA teaches all aspects of claim 3 as disclosed above and further discloses
The computer program product of claim 3, wherein the ex situ data includes Optical Critical Dimension (OCD) information that indicates a depth of the features of the wafer.
(BANNA, p. 3, [0036] “… These optical or other measurements can be inside the chamber (in-situ), or outside the chamber (ex-situ), but still under vacuum, or, at the factory interface (not necessarily under vacuum) on a process platform that may have multiple chambers. One such example is integrated metrology, such as optical critical dimension (OCD).”)
(BANNA, p. 3, [0041] “… The dimensions of interest may be geometrical dimensions of physical features on the wafer, e.g., a line width, a height of a structure, a sidewall angle, a top critical dimension (TCD), a bottom critical dimension (BCD) or any other feature-level three-dimensional profile information.”)
Regarding claim 5
BANNA teaches all aspects of claim 4 as disclosed above and further discloses
The computer program product of claim 4, wherein the ex situ data comprises an etch depth.
(BANNA, p. 3, [0041] “… The dimensions of interest may be geometrical dimensions of physical features on the wafer, e.g., a line width, a height of a structure, a sidewall angle, a top critical dimension (TCD), a bottom critical dimension (BCD) or any other feature-level three-dimensional profile information.”)
(BANNA, p. 2, [0031] “… During production ramp leading to high volume manufacturing (HVM), the disclosed systems and methods enables tighter control of the process window not only intra-wafer, but also between wafers in a single lot (wafer-to-wafer control), or between different lots of wafers (lot-to-lot control). The process control frequency and frequency of model adaptation may vary depending on whether it is wafer-to-wafer (higher frequency), lot-to-lot (medium frequency) or PM-to-PM (i.e. once at each periodic maintenance (PM)).”)
(BANNA, p. 2, [0032] “Examples of the semiconductor processing equipment may include, but are not limited to, advanced plasma etchers. As an illustration, a typical plasma etch process may have more than twenty steps, and there may be twenty or more knobs available to control each process step by controlling various process parameters that can be varied (also known as process variables).”)
(BANNA, p. 3, [0036] “… For example a platform may have multiple process chambers performing the same process, or may have some chambers where one process (e.g., etch) is performed, and other chambers where a different process (e.g. deposition) is performed.”)
(BANNA, p. 5, [0062] “… Example of DoE processes may include but are not limited to an etch process for tapered bottom profile in a device structure to match process of record (POR) for wafer acceptance test (WAT).”)
Regarding claim 6
BANNA teaches all aspects of claim 4 as disclosed above and further discloses
The computer program product of claim 4, wherein the first machine learning model and the second machine learning model are each used to generate predicted OCD values using the in situ measurements.
(BANNA, p. 2, [0023] “FIG. 7D shows the different inputs used in an adaptive version of the machine-learning based model, according to an embodiment of this disclosure;”)
(BANNA, p. 6, [0071] “FIG. 7D shows various information utilized to create a multi-input multi-output adaptive empirical process control model 760 (also referred to as “adaptive model”). “Multi-input” refers to metrology data about various dimensions of interest, and “multi-output” refers to various spatial profiles predicted by the model.”)
(BANNA, p. 2, [0027] “FIG. 10 shows a flow diagram of an example process control method using an adaptive model to maintain tighter process control, including chamber-to-chamber variability control (long term and short term), during high volume manufacturing, in accordance with some embodiments of the present disclosure;”)
(BANNA, p. 7, [0079] “The method 1000 may have two complementary flows, namely, a model-building and updating flow 1005 for the adaptive model and a within-lot run-to-run process control flow 1010.”)
(BANNA, p. 8, [0080] “The run-to-run process control flow 1010 starts at block 1060, where on-wafer metrology (i.e. spatial map of dimensions of interest) is predicted based on VM and OBM data.”)
(BANNA, p. 3, [0036] “… One such example is integrated metrology, such as optical critical dimension (OCD).”)
Regarding claim 7
BANNA teaches all aspects of claim 1 as disclosed above and further discloses
The computer program product of claim 1, wherein the metric indicating the error comprises a cumulative sum of errors of the plurality of process chambers.
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer.”)
(BANNA, p. 6, [0065] “Presently, the primary focus in process control is lot-to-lot control rather than wafer-to-wafer control within a lot. Inventors of the present disclosure recognize that temporal changes due to chamber condition drift, chamber wall changes and degradation of consumable parts over time may introduce device performance variation between wafers even within a single lot. Within a multi-chamber fabrication platform, the process control ecosystem requires big-data-analytics based process control model for advanced chamber matching to minimize wafer-to-wafer device performance variability.”)
(BANNA, p. 3, [0036] “… The term “platform” broadly encompasses a system including multiple process and/or metrology tools which are all identical or some of the tools may be different from the others. For example, a platform may have multiple process chambers performing the same process or may have some chambers where one process (e.g., etch) is performed, and other chambers where a different process (e.g. deposition) is performed. In some other embodiments, a platform may include different types of process chambers (e.g., conductor or dielectric etch). A platform may also include metrology tools. Persons skilled in the art would appreciate that the scope of the disclosure is not limited by the configuration of the chamber and/or the platform.”)
Regarding claim 8
BANNA teaches all aspects of claim 7 as disclosed above and further discloses
The computer program product of claim 7, wherein determining whether to update the first machine learning model comprises determining whether the cumulative sum of errors exceeds a control threshold.
(BANNA, p. 4, [0050] “FIG. 1B shows a system 100 that outputs a final spatial model 112.”)
(BANNA, p. 4, [0051] “… The module 106 is a key module that extracts meaningful features from the data set and draw inference to optimize machine-learning engine performance.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition.”)
(BANNA, p. 4, [0052] “…Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
Regarding claim 9
BANNA teaches all aspects of claim 1 as disclosed above and further discloses
The computer program product of claim 1, wherein the metric indicating the error comprises a variance of errors of the plurality of process chambers.
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition.”)
(BANNA, p. 1, [0007] “Aspects of the disclosure describe a method and a corresponding system for controlling chamber-to-chamber variability during manufacturing of a device on wafers. Specifically, a computer-implemented method is described, where, for each current chamber in a multi-chamber processing platform, a spatial model for a wafer is obtained.”)
(BANNA, p. 1, [0007] “… The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber.”)
(BANNA, p. 3, [0036] “… The term “platform” broadly encompasses a system including multiple process and/or metrology tools which are all identical or some of the tools may be different from the others. For example, a platform may have multiple process chambers performing the same process or may have some chambers where one process (e.g., etch) is performed, and other chambers where a different process (e.g. deposition) is performed. In some other embodiments, a platform may include different types of process chambers (e.g., conductor or dielectric etch). A platform may also include metrology tools. Persons skilled in the art would appreciate that the scope of the disclosure is not limited by the configuration of the chamber and/or the platform.”)
Regarding claim 10
BANNA teaches all aspects of claim 9 as disclosed above and further discloses
The computer program product of claim 9, wherein determining whether to update the first machine learning model comprises determining whether the variance of errors exceeds a control threshold.
(BANNA, p. 1, [0007] “Aspects of the disclosure describe a method and a corresponding system for controlling chamber-to-chamber variability during manufacturing of a device on wafers.”)
(BANNA, p. 4, [0050] “FIG. 1B shows a system 100 that outputs a final spatial model 112.”)
(BANNA, p. 4, [0051] “… The module 106 is a key module that extracts meaningful features from the data set and draw inference to optimize machine-learning engine performance.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition.”)
(BANNA, p. 4, [0052] “…Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
Regarding claim 11
BANNA teaches all aspects of claim 1 as disclosed above and further discloses
The computer program product of claim 1, wherein determining whether to update the first machine learning model comprises determining that a cumulative sum of errors of the plurality of process chambers exceeds a control threshold and that a variance of errors of the plurality of process chambers exceeds the control threshold.
(BANNA, p. 1, [0007] “Aspects of the disclosure describe a method and a corresponding system for controlling chamber-to-chamber variability during manufacturing of a device on wafers.”)
(BANNA, p. 4, [0050] “FIG. 1B shows a system 100 that outputs a final spatial model 112.”)
(BANNA, p. 4, [0051] “… The module 106 is a key module that extracts meaningful features from the data set and draw inference to optimize machine-learning engine performance.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition.”)
(BANNA, p. 4, [0052] “…Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
Regarding claim 12
BANNA teaches all aspects of claim 1 as disclosed above and further discloses
The computer program product of claim 1, wherein generating the second machine learning model comprises training a machine learning model using a training set constructed from the ex situ data received from the plurality of process chambers and the in situ measurements received from the plurality of process chambers.
(BANNA, p. 1, [0007] “Aspects of the disclosure describe a method and a corresponding system for controlling chamber-to-chamber variability during manufacturing of a device on wafers.”)
(BANNA, p. 2, [0023] “FIG. 7D shows the different inputs used in an adaptive version of the machine-learning based model, according to an embodiment of this disclosure;”)
(BANNA, p. 4, [0046] “… The model is built based on the following building blocks: 1) a machine-learning engine that processes on-tool and off-tool customized metrology data 70 from a finite set of actual DoE wafers;”)
(BANNA, p. 4, [0050] “… Input to the system 100 includes characterization data from physical DoE wafers using recipes around a baseline recipe. Characterization data includes on-tool metrology data 101, including VM raw data 103a and OBM raw data 103b, as well as off-tool data 102, including in-line metrology data 104a and non-inline metrology data 104b.”)
(BANNA, p. 3, [0036] “On-tool metrology can include measurements performed on the devices themselves within a die or on test structures having features similar to the devices. Depending on the measurement techniques used, the test structures may include, but are not limited to, structures similar to logic or memory devices that are on the wafers. Collectively, on-tool metrology performed on devices or test structures is referred to as “on-board metrology” (OBM). OBM can be based on optical measurements (e.g., collecting optical emission spectra in-situ from devices or test structures, or macro 2D mapping using optical targets) or other types of measurements. These optical or other measurements can be inside the chamber (in-situ), or outside the chamber (ex-situ), but still under vacuum, or, at the factory interface (not necessarily under vacuum) on a process platform that may have multiple chambers. One such example is integrated metrology, such as optical critical dimension (OCD). The term “platform” broadly encompasses a system including multiple process and/or metrology tools which are all identical or some of the tools may be different from the others. For example a platform may have multiple process chambers performing the same process, or may have some chambers where one process (e.g., etch) is performed, and other chambers where a different process (e.g. deposition) is performed. In some other embodiments, a platform may include different types of process chambers (e.g., conductor or dielectric etch). A platform may also include metrology tools. Persons skilled in the art would appreciate that the scope of the disclosure is not limited by the configuration of the chamber and/or the platform.”)
Regarding claim 13
BANNA teaches all aspects of claim 12 as disclosed above and further discloses
The computer program product of claim 12, wherein the in situ measurements comprise reflectance data.
(BANNA, p. 3, [0036] “… Collectively, on-tool metrology performed on devices or test structures is referred to as “on-board metrology” (OBM). OBM can be based on optical measurements (e.g., collecting optical emission spectra in-situ from devices or test structures, or macro 2D mapping using optical targets) or other types of measurements. These optical or other measurements can be inside the chamber (in-situ), or outside the chamber (ex-situ), but still under vacuum, or, at the factory interface (not necessarily under vacuum) on a process platform that may have multiple chambers. One such example is integrated metrology, such as optical critical dimension (OCD).”)
Regarding claim 14
BANNA teaches all aspects of claim 1 as disclosed above and further discloses
The computer program product of claim 1, further comprising instructions for
determining whether the second machine learning model satisfies criteria to be deployed to the plurality of process chambers
(BANNA, p. 7, [0075] “FIG. 9 shows how to adaptively adjust the wafer-to-wafer control model after each lot. Specifically, FIG. 9 shows a block diagram representing the adaptive process for model update. In block 902 and 904, VM raw data and OBM raw data respectively, are collected from the one or more samples of the current lot of wafers (
L
n
-
1
). In-line metrology data is collected for the current lot in block 906 by sampling a few wafers in the current lot. All the collected data goes through the data filtering and features extraction module 910. The spatial model 912 creates a spatial predicted measurement 914, which is fed to the machine-learning engine 918. The machine-learning engine 918 compares predictive measurement to a reference measurement 916 provided by in-line or other off-tool metrology. Process recipe 920 and process and equipment information 922 are provided to the machine-learning engine. An updated spatial model is created in block 924 to be used for controlling the next lot (
L
n
). A process similar to the process shown in the block diagram of FIG. 8 is used to recommend a recipe for the next lot ((
L
n
).”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition. Optimization routines (including, but not limited to swarm optimization or swarm variants, are designed to minimize non-convex multi-minima hyper-surfaces. Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
and in response to determining that the second machine learning model satisfies the criteria to be deployed to the plurality of process chambers, transmitting the second machine learning model to each of the plurality of process chambers.
(BANNA, p. 7, [0075] “FIG. 9 shows how to adaptively adjust the wafer-to-wafer control model after each lot. Specifically, FIG. 9 shows a block diagram representing the adaptive process for model update. In block 902 and 904, VM raw data and OBM raw data respectively, are collected from the one or more samples of the current lot of wafers (
L
n
-
1
). In-line metrology data is collected for the current lot in block 906 by sampling a few wafers in the current lot. All the collected data goes through the data filtering and features extraction module 910. The spatial model 912 creates a spatial predicted measurement 914, which is fed to the machine-learning engine 918. The machine-learning engine 918 compares predictive measurement to a reference measurement 916 provided by in-line or other off-tool metrology. Process recipe 920 and process and equipment information 922 are provided to the machine-learning engine. An updated spatial model is created in block 924 to be used for controlling the next lot (
L
n
). A process similar to the process shown in the block diagram of FIG. 8 is used to recommend a recipe for the next lot ((
L
n
).”)
(BANNA, p. 2, [0026] “FIG. 9 shows adaptive extension of the model for periodic updating, including but not limited to lot-to-lot updating, according to an embodiment of the present disclosure;”)
(BANNA, p. 2, [0031] “… During production ramp leading to high volume manufacturing (HVM), the disclosed systems and methods enables tighter control of the process window not only intra-wafer, but also between wafers in a single lot (wafer-to-wafer control), or between different lots of wafers (lot-to-lot control). The process control frequency and frequency of model adaptation may vary depending on whether it is wafer-to-wafer (higher frequency), lot-to-lot (medium frequency) or PM-to-PM (i.e. once at each periodic maintenance (PM)).”)
(BANNA, p. 8, [0083] “FIG. 12 shows a technical architecture block diagram showing a single chamber for simplicity of illustrations, though persons skilled in the art would appreciate that multiple chambers may be part of a single platform. The process platform 1220 has a process modeling mini-server 1208. The process model captures physics of the process within the chamber that creates the spatial variation in one or more dimensions of interest across the wafer. The model may be chamber-specific, or a common model may be used for multiple chambers.”)
Regarding claim 15
BANNA teaches all aspects of claim 14 as disclosed above and further discloses
The computer program product of claim 14, wherein determining whether the second machine learning model satisfies the criteria to be deployed comprises evaluating the first machine learning model and the second machine learning model on the test set, and wherein the test set comprises the ex situ data and in situ measurements.
(BANNA, p. 3, [0036] “On-tool metrology can include measurements performed on the devices themselves within a die or on test structures having features similar to the devices. Depending on the measurement techniques used, the test structures may include, but are not limited to, structures similar to logic or memory devices that are on the wafers. Collectively, on-tool metrology performed on devices or test structures is referred to as “on-board metrology” (OBM). OBM can be based on optical measurements (e.g., collecting optical emission spectra in-situ from devices or test structures, or macro 2D mapping using optical targets) or other types of measurements. These optical or other measurements can be inside the chamber (in-situ), or outside the chamber (ex-situ), but still under vacuum, or, at the factory interface (not necessarily under vacuum) on a process platform that may have multiple chambers.”)
Regarding claim 16
BANNA teaches all aspects of claim 15 as disclosed above and further discloses
The computer program product of claim 15, wherein the criteria comprises better predictive performance of the second machine learning model on the test set of ex situ data and in situ measurements compared to the first machine learning model.
(BANNA, p. 1, [0007] “… The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber.”)
(BANNA, p. 2, [0021] “FIG. 7B shows correlation between various test and measurement techniques and device performance, in accordance with embodiments of the present disclosure;”)
(BANNA, p. 2, [0022] “FIG. 7C shows how an empirical spatial model, which is the foundation for process optimization and control, can be calibrated to both device performance and in-line metrology, in accordance with an embodiment of the present disclosure;”)
(BANNA, p. 4, “… The module 106 is a key module that extracts meaningful features from the data set and draw inference to optimize machine-learning engine performance.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The machine-learning engine 108 then generates intermediate spatial model 109 for each measurement on the wafer. Each measurement can have data about one or more dimensions of interest. The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition. Optimization routines (including, but not limited to swarm optimization or swarm variants, are designed to minimize non-convex multi-minima hyper-surfaces. Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
Regarding claim 19
BANNA teaches all aspects of claim 14 as disclosed above and further discloses
The computer program product of claim 14, wherein determining whether the second machine learning model satisfies the criteria to be deployed comprises determining that an error of the second machine learning model in predicting ex situ data included in a test set is below a threshold.
(BANNA, p. 4, [0050] “FIG. 1B shows a system 100 that outputs a final spatial model 112.”)
(BANNA, p. 4, [0051] “… The module 106 is a key module that extracts meaningful features from the data set and draw inference to optimize machine-learning engine performance.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition.”)
(BANNA, p. 4, [0052] “… The machine-learning engine 108 then generates intermediate spatial model 109 for each measurement on the wafer. Each measurement can have data about one or more dimensions of interest. The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition. Optimization routines (including, but not limited to swarm optimization or swarm variants, are designed to minimize non-convex multi-minima hyper-surfaces. Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
Regarding claim 20
BANNA teaches all aspects of claim 14 as disclosed above and further discloses
The computer program product of claim 14, further comprising instructions for
(i) in response to determining that the second machine learning model does not satisfy criteria to be deployed to the plurality of process chambers, generating a third machine learning model
(BANNA, p. 4, [0052] “The machine-learning method used by the machine-learning engine 108 can be based on neural network, deep learning or any other known techniques used for regression analysis. In addition to metrology data, the machine-learning engine 108 also receives information 111 about various recipes and knobs, as well as information 113 about the process and the equipment. The machine-learning engine 108 then generates intermediate spatial model 109 for each measurement on the wafer. Each measurement can have data about one or more dimensions of interest. The model's performance is evaluated by the evaluation module 110.”)
(ii) determining whether the third machine learning model satisfies the criteria to be deployed to the plurality of process chambers; repeating (i) and (ii) until it is determined that the third machine learning model satisfies the criteria to be deployed to the plurality of process chambers
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition. Optimization routines (including, but not limited to swarm optimization or swarm variants, are designed to minimize non-convex multi-minima hyper-surfaces. Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
and in response to determining that the third machine learning model satisfies the criteria to be deployed to the plurality of process chambers, transmitting the third machine learning model to each of the plurality of process chambers.
(BANNA, p. 7, [0075] “FIG. 9 shows how to adaptively adjust the wafer-to-wafer control model after each lot. Specifically, FIG. 9 shows a block diagram representing the adaptive process for model update.”)
(BANNA, p. 2, [0031] “… During production ramp leading to high volume manufacturing (HVM), the disclosed systems and methods enables tighter control of the process window not only intra-wafer, but also between wafers in a single lot (wafer-to-wafer control), or between different lots of wafers (lot-to-lot control). The process control frequency and frequency of model adaptation may vary depending on whether it is wafer-to-wafer (higher frequency), lot-to-lot (medium frequency) or PM-to-PM (i.e. once at each periodic maintenance (PM)).”)
(BANNA, p. 8, [0083] “FIG. 12 shows a technical architecture block diagram showing a single chamber for simplicity of illustrations, though persons skilled in the art would appreciate that multiple chambers may be part of a single platform. The process platform 1220 has a process modeling mini-server 1208. The process model captures physics of the process within the chamber that creates the spatial variation in one or more dimensions of interest across the wafer. The model may be chamber-specific, or a common model may be used for multiple chambers.”)
Regarding claim 21
BANNA teaches all aspects of claim 20 as disclosed above and further discloses
The computer program product of claim 20, wherein repeating (i) and (ii) until it is determined that the third machine learning model satisfies the criteria to be deployed comprises repeating (i) and (ii) until it is determined that the third machine learning model is optimal.
(BANNA, p. 1, [0007] “… The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber.”)
(BANNA, p. 2, [0020] “FIG. 7A shows DoEs during process optimization and process control, respectively, in accordance with an embodiment of the present disclosure;”)
(BANNA, p. 2, [0022] “FIG. 7C shows how an empirical spatial model, which is the foundation for process optimization and control, can be calibrated to both device performance and in-line metrology, in accordance with an embodiment of the present disclosure;”)
(BANNA, p. 4, [0051] “The characterization data is then fed to the machine-learning engine 108. The data is filtered by an additional data filtering and features extraction module 106 that precedes the machine-learning engine 108. The module 106 is a key module that extracts meaningful features from the data set and draw inference to optimize machine-learning engine performance. The output of module 106 is multi-input metrology data 107a (derived from on-tool and off-tool metrology data) from the currently used DoE wafers.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The machine-learning engine 108 then generates intermediate spatial model 109 for each measurement on the wafer. Each measurement can have data about one or more dimensions of interest. The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition. Optimization routines (including, but not limited to swarm optimization or swarm variants, are designed to minimize non-convex multi-minima hyper-surfaces. Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
Regarding claim 22
BANNA teaches all aspects of claim 20 as disclosed above and further discloses
The computer program product of claim 20, wherein a training set used to generate the second machine learning model is smaller than a training set used to generate the third machine learning model
(BANNA, p. 1, [0007] “… The spatial model is created by a first machine-learning engine based on a first set of metrology data on one or more dimensions of interest in the device. “)
(BANNA, p. 10, [0096] “… The method of claim 1, wherein the second machine-learning engine adaptively refines an empirical process model by getting trained with the first set of metrology data and a second set of metrology data on one or more dimensions of interest in the device.”)
A person having ordinary skill in the art knows very well that different machine learning engines utilize different set of training data.
Regarding claim 23
BANNA teaches all aspects of claim 22 as disclosed above and further discloses
The computer program product of claim 22, wherein the training set used to generate the third machine learning model comprises newer ex situ data and in situ measurements than the training set used to generate the second machine learning model.
(BANNA, p. 1, [0007] “… The spatial model is created by a first machine-learning engine based on a first set of metrology data on one or more dimensions of interest in the device. The spatial model can be global, covering physical behavior of the process, or could be chamber-specific, accounting for chamber variability.”)
(BANNA, p. 3, [0039] “Off-tool metrology may also include data available from any additional non-inline or off-line metrology, such as TEM, previously performed on a similar set of devices.”)
(BANNA, p. 4, [0048] “During the process model's training phase, a subset of the physical DoE wafers (e.g. 20-100 wafers) is processed using a finite number of recipes around the baseline recipe.”)
(BANNA, p. 4, [0052] “… Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary.”)
Regarding claim 24
BANNA discloses
A computer program product for using adaptively trained models, the computer program product comprising a non-transitory computer-readable medium on which is provided computer-executable instructions for
(BANNA, p. 10, [0095] “The present disclosure may be provided as a computer program product, or software, that may include a machine-readable medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to the present disclosure.”)
(BANNA, p. 10, claim 3, “The method of claim 1, wherein the second machine-learning engine adaptively refines an empirical process model by getting trained with the first set of metrology data and a second set of metrology data on one or more dimensions of interest in the device.”)
(BANNA, p. 9, [0085] “The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a web appliance, a server, a network router, a switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.”)
(BANNA, p. 9, [0089] “The data storage device 1316 may include a machine-readable storage medium 1324 (also known as a computer-readable medium) on which is stored one or more sets of instructions or software embodying any one or more of the methodologies or functions described herein.”)
transmitting, to a model training system, ex situ metrology data corresponding to a wafer fabricated using a first machine learning model received from the model training system
(BANNA, p. 1, [0007] “Aspects of the disclosure describe a method and a corresponding system for controlling chamber-to-chamber variability during manufacturing of a device on wafers. Specifically, a computer-implemented method is described, where, for each current chamber in a multi-chamber processing platform, a spatial model for a wafer is obtained. The spatial model is created by a first machine-learning engine based on a first set of metrology data on one or more dimensions of interest in the device.”)
wherein the first machine learning model is used for process control of a process chamber that fabricated the wafer
(BANNA, p. 2, [0025] “FIG. 8 shows application of the model for wafer-to-wafer process control, according to an embodiment of the present disclosure;”)
(BANNA, p. 7, [0079] “The method 1000 may have two complementary flows, namely, a model-building and updating flow 1005 for the adaptive model and a within-lot run-to-run process control flow 1010. The term “run-to-run” is used to mean from one wafer to the next wafer within a lot.”)
(BANNA, p. 7, [0079] “… Next, a process control model is built in block 1030, which captures process variability via VM and OBM, and uses machine-learning engine to suggest process variability correction that may be used for a next set of wafers within a lot.”)
receiving, from the model training system, a second machine learning model for use in process control of the process chamber
(BANNA, p. 1, [0007] “… Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber.”)
wherein the second machine learning model was generated by the model training system using the ex situ metrology data received from a plurality of process chambers and in situ on-wafer optical data measured by the plurality of process chambers
(BANNA, p. 3, [0036] “… Collectively, on-tool metrology performed on devices or test structures is referred to as “on-board metrology” (OBM). OBM can be based on optical measurements (e.g., collecting optical emission spectra in-situ from devices or test structures, or macro 2D mapping using optical targets) or other types of measurements. These optical or other measurements can be inside the chamber (in-situ), or outside the chamber (ex-situ), but still under vacuum, or, at the factory interface (not necessarily under vacuum) on a process platform that may have multiple chambers. One such example is integrated metrology, such as optical critical dimension (OCD). The term “platform” broadly encompasses a system including multiple process and/or metrology tools which are all identical or some of the tools may be different from the others. For example, a platform may have multiple process chambers performing the same process, or may have some chambers where one process (e.g., etch) is performed, and other chambers where a different process (e.g. deposition) is performed. In some other embodiments, a platform may include different types of process chambers (e.g., conductor or dielectric etch). A platform may also include metrology tools. Persons skilled in the art would appreciate that the scope of the disclosure is not limited by the configuration of the chamber and/or the platform.”)
(BANNA, p. 4, [0046] “At block 60, a process model is built. The model is built based on the following building blocks: 1) a machine-learning engine that processes on-tool and off-tool customized metrology data 70 from a finite set of actual DoE wafers; 2) input based on fundamental understanding of process equipment (e.g. chamber) design and physics and chemistry of the process within the equipment (e.g. properties and behavior of plasma within a reactor); 3) input based on equipment hardware specification and allowed range of operation (including process recipe creation rules); and 4) a penalty function quantifying the confidence level in the model's prediction.”)
and replacing the first machine learning model with the second machine learning model, wherein the first machine learning model and the second machine learning model were evaluated using the same test set that includes ex situ data collected before a determination that the first machine learning model is to be updated, and ex situ data collected after the determination that the first machine learning model is to be updated
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 7, [0075] “FIG. 9 shows how to adaptively adjust the wafer-to-wafer control model after each lot. Specifically, FIG. 9 shows a block diagram representing the adaptive process for model update. In block 902 and 904, VM raw data and OBM raw data respectively, are collected from the one or more samples of the current lot of wafers (
L
n
-
1
). In-line metrology data is collected for the current lot in block 906 by sampling a few wafers in the current lot. All the collected data goes through the data filtering and features extraction module 910. The spatial model 912 creates a spatial predicted measurement 914, which is fed to the machine-learning engine 918. The machine-learning engine 918 compares predictive measurement to a reference measurement 916 provided by in-line or other off-tool metrology. Process recipe 920 and process and equipment information 922 are provided to the machine-learning engine. An updated spatial model is created in block 924 to be used for controlling the next lot (
L
n
). A process similar to the process shown in the block diagram of FIG. 8 is used to recommend a recipe for the next lot ((
L
n
).”)
and wherein replacing the first machine learning model with the second machine learning model is responsive to determining the second machine learning model performed better in the evaluation compared to the first machine learning model when evaluated with the same test set.
(BANNA, p. 1, [0007] “… Specifically, a computer-implemented method is described, where, for each current chamber in a multi-chamber processing platform, a spatial model for a wafer is obtained. The spatial model is created by a first machine-learning engine based on a first set of metrology data on one or more dimensions of interest in the device. The spatial model can be global, covering physical behavior of the process, or could be chamber-specific, accounting for chamber variability. One or more parameters of the current chamber are obtained. Using the spatial model and the one or more parameters of the current chamber, spatial measurements of the one or more dimensions of interest in the device across the wafer are predicted. Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber.”)
(BANNA, p. 3, [0040] “Once the machine-learning-based spatial model is tested and validated, the model is used for developing an optimized process for actual production wafers.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The machine-learning engine 108 then generates intermediate spatial model 109 for each measurement on the wafer. Each measurement can have data about one or more dimensions of interest. The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition. Optimization routines (including, but not limited to swarm optimization or swarm variants, are designed to minimize non-convex multi-minima hyper-surfaces. Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted.”)
Regarding claim 25
BANNA teaches all aspects of claim 24 as disclosed above and further discloses
The computer program product of claim 24, further comprising instructions for receiving, from the model training system, a message that an error associated with the first machine learning model has exceeded a threshold.
(BANNA, p. 4, [0050] “FIG. 1B shows a system 100 that outputs a final spatial model 112.”)
(BANNA, p. 4, [0051] “… The module 106 is a key module that extracts meaningful features from the data set and draw inference to optimize machine-learning engine performance.”)
(BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
(BANNA, p. 4, [0052] “… The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. The cost function is sometimes referred to as “objective function,” designed to allow optimization of one or more dimensions of interest. The cost function can be for each location on a wafer, or just one cost function for an entire wafer. Cost function can also be for each DoE condition.”)
(BANNA, p. 4, [0052] “…Error penalties or regularization terms may be added to the cost function to find higher probability solutions in high dimension non-convex multi-minima hyper-surfaces. Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
(BANNA, p. 2, [0030] “FIG. 13 shows a simplified environment within which the methods and systems of the present disclosure may be implemented.”)
(BANNA, p. 7, [0084] “FIG. 13 illustrates an example machine of a computer system 1300 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.”)
Regarding claim 26
BANNA teaches all aspects of claim 24 as disclosed above and further discloses
The computer program product of claim 24, further comprising instructions for transmitting, to the model training system, second ex situ metrology data corresponding to a second wafer fabricated using the first machine learning model prior to receiving the second machine learning model from the model training system.
(BANNA, p. 1, [0007] “… Specifically, a computer-implemented method is described, where, for each current chamber in a multi-chamber processing platform, a spatial model for a wafer is obtained. The spatial model is created by a first machine-learning engine based on a first set of metrology data on one or more dimensions of interest in the device. The spatial model can be global, covering physical behavior of the process, or could be chamber-specific, accounting for chamber variability. One or more parameters of the current chamber are obtained. Using the spatial model and the one or more parameters of the current chamber, spatial measurements of the one or more dimensions of interest in the device across the wafer are predicted. Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber.”)
(BANNA, p. 2, [0025] “FIG. 8 shows application of the model for wafer-to-wafer process control, according to an embodiment of the present disclosure;”)
(BANNA, p. 4, [0050] “FIG. 1B shows a system 100 that outputs a final spatial model 112. Input to the system 100 includes characterization data from physical DoE wafers using recipes around a baseline recipe. Characterization data includes on-tool metrology data 101, including VM raw data 103a and OBM raw data 103b, as well as off-tool data 102, including in-line metrology data 104a and non-inline metrology data 104b.”)
(BANNA, p. 4, [0051] “The characterization data is then fed to the machine-learning engine 108. The data is filtered by an additional data filtering and features extraction module 106 that precedes the machine-learning engine 108.”)
Regarding claim 27
BANNA teaches all aspects of claim 24 as disclosed above and further discloses
The computer program product of claim 26, wherein the ex situ metrology data is used to determine that an error associated with the first machine learning model has exceeded a threshold, and wherein the second ex situ metrology data is used to determine that the second machine learning model is to replace the first machine learning model.
(BANNA, p. 1, [0007] “…The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber.”)
(BANNA, p. 2, [0031] “… During the research and development phase, the disclosed systems and methods provide for faster convergence to target process recipes using only a limited number of test wafers.”)
Response to Arguments
Applicant's arguments filed 07/10/2026 have been fully considered but they are not persuasive.
Regarding argument with respect to Independent claim 1
Argument a. "determining whether to update the first machine learning model"
In rejecting independent claim 1, the Office asserted that Banna discloses "determining whether to update the first machine learning model based on [a] metric indicating [an] error," citing FIG. 9 and paragraph [0075] of Banna. (Office Action, page 6).
Paragraph [0075] of Banna states "FIG. 9 shows how to adaptively adjust the wafer-to- wafer control model after each lot." Banna goes on to state "metrology data is collected for the current lot in block 906 by sampling a few wafers in the current lot," and "the machine-learning engine 918 compares predictive measurement to a reference measurement 916," and "an updated spatial model is created in block 924 for controlling the next lot." (Banna, paragraph [0075]).
In other words, there is no determination in Banna of whether to update the spatial model. Banna describes updating the spatial model after every lot. There is no determination of whether to update the spatial model, let alone "based on [a] metric indicating [an] error."
Response: Two paragraphs were cited from Banna with respect to the above. As correctly elaborated on the citation in the argument "the machine-learning engine 918 compares predictive measurement to a reference measurement 916," A person having ordinary skill in the art to which the claimed invention pertains would know very well that comparisons of model predicted measurements against the reference measurement is to make determination with respect to changes to the model. The application "based on [a] metric indicating [an] error." compares the measurements predicted by the model to the actual measurements.
Banna also compares the model prediction to actual measurements, the reference spatial measurements. (BANNA, p. 1, “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
The second part of the citation in the original rejection (BANNA, p. 4, [0052] “… The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric.”)”, clearly indicate the determination is based on a suitable error metric, and the machine learning engine is utilizing cost function to produce the next model.
Argument b. "wherein the first machine learning model and the second machine learning model are evaluated using a test set that includes ex situ data collected before the determination that the first machine learning model is to be updated and ex situ data collected after the determination that the first machine learning model is to be updated"
In rejecting independent claim 1, the Office asserted that Banna discloses "the first machine learning model and the second machine learning model are evaluated using a test set that includes ex situ data collected before the determination that the first machine learning model is to be updated, and ex situ data collected after the determination that the first machine learning model is to be updated.
First, as discussed above, Banna does not make any determination that the first machine learning model is to be updated.
Response: A person having ordinary skill in the art to which the claimed invention pertains would know very well that comparisons of model predicted measurements against the reference measurement is to make determination. If not, why bother doing comparison for nothing. A person having ordinary skill in the art to which the claimed invention pertains would not waste time doing comparison for nothing. The comparison leads to the error and the determination is based on the error and the associated cost function
Argument b (cont.) Instead, Banna updates the spatial model after every lot,
Response: True; depending on the error and the corresponding cost function the model is tweaked with new test wafer (BANNA, p. 2, [0031] “… During the research and development phase, the disclosed systems and methods provide for faster convergence to target process recipes using only a limited number of test wafers.”)
Argument b (cont.) and there is no determination of whether the model is to be updated.
Response: False, depending on the error and the corresponding cost function the machine learning engine proceeds with tweaking the model using the new test wafer.
Argument b (cont.) Accordingly, Banna cannot and does not disclose or suggest evaluating any spatial model with a test set that includes data collected "before the determination that the first machine learning model is to be updated ... and data collected after the determination that the first machine learning model is to be updated," because Banna has no such point of determination with which to demarcate the test set.
Response: As mentioned before Banna uses reference data as the test data: (BANNA, p. 1, “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber. Based on a multi-constraint optimization of the metric performed by a second machine-learning engine, adjustments for at least some of the one or more parameters of the current chamber are recommended, so that a performance of the current chamber substantially matches a performance of the reference chamber. “)
PNG
media_image1.png
504
708
media_image1.png
Greyscale
(BANNA, p. 4, [0052] “…The machine-learning engine 108 then generates intermediate spatial model 109 for each measurement on the wafer. Each measurement can have data about one or more dimensions of interest. The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. … Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
Argument b (cont.) Second, Banna does not evaluate any model with a test set.
Response: (BANNA, p. 4, [0052] “…the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary.”
Argument b (cont.) During the interview, the Examiner appeared to assert that reference measurement 916 is a test set used to evaluate the model. However, Banna states, "[t]he machine learning engine 918 compares predictive measurement to a reference measurement 916," and the machine learning engine generates the updated spatial model. (Banna, paragraph [0075]). In other words, reference measurement 916 is an input used by the machine learning engine 918 to generate the machine learning model,
Response: (BANNA, p. 1, [0007] “… Reference spatial measurements of the one or more dimensions of interest in the device across the wafer are obtained from a reference chamber (e.g., a golden chamber used to calibrate different chambers). In some embodiments, a model of a golden chamber may be used for calibration. The predicted spatial measurements are compared with the reference spatial measurements to produce a metric indicating variation of performance between the current chamber and the reference chamber.”)
As stated before the reference data used for evaluation as well as for model optimization using the error and the associated cost function.
Argument b (cont.): not a test set used to evaluate the generated model. In other words, Applicant's claim recites that the second machine learning model is generated, and then that the second machine learning model is evaluated. Nowhere does Banna evaluate the updated spatial model.
Response: As stated before the reference data used for evaluation as well as for model optimization using the error and the associated cost function. Also, please see the figure and the associated paragraph below.
PNG
media_image1.png
504
708
media_image1.png
Greyscale
(BANNA, p. 4, [0052] “…The machine-learning engine 108 then generates intermediate spatial model 109 for each measurement on the wafer. Each measurement can have data about one or more dimensions of interest. The model's performance is evaluated by the evaluation module 110. The model's performance is optimized using a penalty function or cost function 105, such as root mean square error (rMSE) or any other suitable metric. … Once the desired value of the cost function is obtained, the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary. Depending on how a cost function is chosen, the test and validation process can be repeated spatially for each data point across a wafer for which metrology was conducted. Alternatively, the spatial model can be optimized to achieve an average dimensional uniformity across the wafer. The final spatial model 112 can combine results from all the data points on the wafer for which metrology was conducted.”)
Argument b (cont.): During the telephonic interview, the Examiner stated that "Banna is always evaluating the model," and because Banna is using optimization, the "updated model will always be better." Applicant respectfully submits that these two statements are inconsistent - if the updated model in Banna is "always better," there is no need to evaluate the updated model, and, in fact, Banna does not evaluate the updated model, let alone evaluating the updated model with "a test set that includes ex situ data collected before the determination that the first machine learning model is to be updated and ex situ data collected after the determination that the first machine learning model is to be updated," as recited in Applicant's claim 1.
Response: A person having ordinary skill in the art to which the claimed invention pertains would be wise enough always to validate the AI model with actual measurements after applying the model even though the model is optimized using the ideal measurement values from a reference chamber. Banna teaches it (BANNA, p. 4, [0052] “…the spatial model may be further validated using metrology data from another set of physical DoE wafers. Number of test and validation wafers may be in the range of tens or twenties but may vary.”)
Claim 1 conclusion: In essence Banna teaches all features of claim 1.
Regarding argument with respect to independent claim 24
Argument: Applicant respectfully submits that independent claim 24 includes similar features as recited in independent claim 1 and is allowable for at least the same reasons.
However, independent claim 24 is allowable for at least a second reason. Independent claim 24 has been amended to recite, inter alia:
replacing the first machine learning model with the second machine learning model, wherein the first machine learning model and the second machine learning model were evaluated using the same test set that includes ex situ data collected before a determination that the first machine learning model is to be updated, and ex situ data collected after the determination that the first machine learning model is to be updated, and wherein replacing the first machine learning model with the second machine learning model is responsive to determining the second machine learning model performed better in the evaluation compared to the first machine learning model when evaluated with the same test set.
(Independent claim 24 as amended; emphasis added).
First, nowhere does Banna disclose or suggest evaluating the original spatial model and the updated spatial model using the same test set, as recited in amended claim 24.
Response: See the responses provided above under Regarding argument with respect to Independent claim 1.
Argument : Second, nowhere does Banna disclose or suggest "wherein replacing the first machine learning model with the second machine learning model is responsive to determining the second machine learning model performed better in the evaluation compared to the first machine learning model when evaluated with the same test set," as recited in amended claim 24. Rather, in Banna, the updated spatial model is always deployed.
Response: Banna teaches that the updated model is an optimized based on the error and the cost function being deployed for tweaking the model. Further the updated model is validated against multiple wafers with actual measurements. Refer to the citations above.
Claim 24 conclusion: Banna teaches all aspects of claim 24.
Conclusion on arguments
Applicant's arguments are not persuasive and the independent claims 1 and 24 are rejected. The dependent claims 2-16, 19-23 and 25-27 are rejected based on their dependency to claims 1 and 24 respectively.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYAPPU SOUNDRANAYAGAM whose telephone number is (571)272-0629. The examiner can normally be reached Mon-Fri:8:00AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jack Chiang can be reached at (571) 272-7483. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.S./Examiner, Art Unit 2851
/JACK CHIANG/Supervisory Patent Examiner, Art Unit 2851
Prosecution Insights