Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The action is in response to the original filing on August 22, 2023 and the Remarks and Amendments filed on August 03, 2026. Claims 1-20 are pending and have been considered below. Claims 1, 10 and 16 are independent claims. Claims 1, 9, 10, 12, 14, 15, 16, 19 are amended.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. KR10-2022-01836011, filed on December 27, 2022.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 8-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03.
Claim 1 is a method claim, thus it falls into one of the four categories of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding independent claim 1, the following limitations recite judicial exceptions:
"determining an out of distribution (OOD) index with respect to the spectrum of the semiconductor for each of the at least one semiconductor based on the first data set"
[Mathematical Calculations / Mental Process] – Determining an out-of-distribution (OOD) index based on spectral data involves calculating statistical distribution metrics, numerical evaluations, and data comparisons capable of being performed in the human mind with the assistance of paper and pen, as well as mathematical calculations.
"performing a data split on the first data set and the second data set by cluster sampling the first data set and the second data set into at least one learning data set with respect to the OOD index according to the at least one semiconductor"
[Mental Process / Mathematical Calculations] – Clustering data, partitioning/splitting data sets based on calculated numerical indexes, and sampling data points represent logical sorting, categorization, and mathematical data manipulation capable of being performed in the human mind with paper and pen.
"training a plurality of AI algorithms using the first data set and the second data set such that at least one of the plurality of AI algorithms is trained using the at least one learning data set"
[Mathematical Calculations / Mental Process] – Training artificial intelligence algorithms relies on mathematical calculations, statistical optimization, matrix operations, and probabilistic updates, which recite an abstract idea.
"providing an optimal AI algorithm, from among the plurality of AI algorithms that have been trained on the at least one learning data set"
[Mental Process / Mathematical Calculations] – Comparing performance metrics of trained algorithms to select and provide an optimal algorithm involves numerical evaluations, logical decision-making, and ranking capable of being performed in the human mind with paper and pen.
"wherein the OOD index represents a distribution of the first data set having a different distribution from a distribution of learning data used to train the plurality of AI algorithms."
[Mental Process / Mathematical Calculations] – Defining or describing the OOD index as representing differences between statistical distributions involves mental observations, statistical comparisons, and conceptual evaluations capable of being performed in the human mind with paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
Regarding Claim 1, the claim recites additional elements of:
"loading a first data set and a second data set, the first data set representing a spectrum of at least one semiconductor, and the second data set representing a structure of the at least one semiconductor"
Loading data sets representing semiconductor spectrums and structures to a computer is mere data gathering recited at a high level of generality, thus is insignificant extra-solution activity (see MPEP 2106.05(g)).
"AI algorithms"
The AI algorithms are recited at a high level of generality and merely add words to "apply it" using generic software or machine-learning components (see MPEP 2106.05(f)).
[Even when viewed in combination, the additional elements do no more than automate the mental processes and mathematical calculations that a person could perform using generic computer components as a tool, thus the claim as a whole does not integrate the judicial exception into a practical application.]
Step 2B – whether the claim as a whole amounts to significantly more than the judicial exception? I.e., Are there any additional elements (features/limitations/step) recited in the claim beyond the abstract idea?
The claim does not provide an inventive concept (significantly more than the abstract idea). The claim is ineligible under 35 U.S.C. 101.
As explained above, the additional element [1] is considered an insignificant extra-solution activity and at best the equivalent of mere data gathering at a high level of generality and amount to receiving data, which is well-understood, routine, conventional activity (see MPEP 2106.05(d), subsection II, MPEP 2106.05(g)).
The additional element [2] is a mere instruction to apply an exception to generic computer components or machine-learning components that simply run mathematical calculations and mental processes (see MPEP 2106.05(f)).
These limitations remain insignificant extra-solution activity and a mere instruction to apply an exception even upon reconsideration. Even when considered in combination, the additional elements represent a mere instruction to apply an exception and insignificant extra-solution activity, which cannot provide an inventive concept.
Regarding Claim 2
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 2 is a dependent claim of 1, thus it falls within the same category of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding dependent claim 2, following limitations recite a judicial exception:
"extracting a first principal component and a second principal component, with respect to the first data set, by performing principal component analysis (PCA) to reduce a dimension of the first data set"
[Mathematical Calculations / Mental Process] – Performing principal component analysis (PCA) to reduce data dimensionality relies on mathematical computations, linear algebra matrix operations, and eigenvector/eigenvalue calculations capable of being performed in the human mind with paper and pen.
"determining, for each of the at least one semiconductor, a value based on a Euclidean distance between vectors and a cosine distance between the vectors and an origin the vectors including the first principal component and the second principal component"
[Mathematical Calculations / Mental Process] – Calculating Euclidean distances and cosine distances between vectors represents mathematical calculations, geometric formulas, and numerical evaluations capable of being performed in the human mind with the assistance of paper and pen.
"extracting a normalized value as the OOD index by normalizing an average value of a product of the Euclidean distance and the cosine distance for each of the at least one semiconductor"
[Mathematical Calculations / Mental Process] – Computing vector products, taking average values, and normalizing numerical outputs to extract a normalized index rely on mathematical computations and statistical formulas capable of being performed in the human mind with paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
The claim 2 does not recite any additional elements other than abstract ideas, so it does not integrate into a practical application. Thus, this claim is directed to the abstract idea.
Regarding Claim 3
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 3 is a dependent claim of 1, thus it falls within the same category of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding dependent claim 3, following limitations recite a judicial exception:
"the at least one semiconductor includes a plurality of semiconductors"
[Mental Process] – Categorizing or specifying data entities as a plurality of semiconductors represents logical grouping, conceptual classification, and observation capable of being performed in the human mind with paper and pen.
"sampling the first data set and the second data set at a preset ratio into a training group, a valid group, and a test group of each allocated learning data set according to the OOD indexes of the plurality of semiconductors"
[Mental Process / Mathematical Calculations] – Partitioning and sampling data sets at preset ratios into training, validation, and test groups based on numerical OOD indexes involves ratio calculations, logical data sorting, and statistical data splitting capable of being performed in the human mind with the assistance of paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
The claim 3 does not recite any additional elements other than abstract ideas, so it does not integrate into a practical application. Thus, this claim is directed to the abstract idea.
Regarding Claim 4
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 4 is a dependent claim of 3, thus it falls within the same category of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding dependent claim 4, following limitations recite a judicial exception:
"sampling the first data set and the second data set into the training group, the valid group, and the test group of learning data set, sequentially according to the OOD indexes of the plurality of semiconductors"
[Mental Process / Mathematical Calculations] – Sequentially sampling and partitioning data sets into training, valid, and test groups in accordance with numerical OOD indexes represents logical sequencing, data sorting, and statistical partitioning capable of being performed in the human mind with the assistance of paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
The claim 4 does not recite any additional elements other than abstract ideas, so it does not integrate into a practical application. Thus, this claim is directed to the abstract idea.
Regarding Claim 5
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 5 is a dependent claim of 4, thus it falls within the same category of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding dependent claim 5, following limitations recite a judicial exception:
"sampling the first data set and the second data set into the training group, the valid group, and the test group in a descending order or an ascending order of the OOD indexes of the plurality of semiconductors"
[Mental Process / Mathematical Calculations] – Sorting, ordering, and sampling data sets into training, validation, and test groups in ascending or descending numerical order based on calculated OOD indexes represents logical sorting, numerical ranking, and data organization capable of being performed in the human mind with the assistance of paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
The claim 5 does not recite any additional elements other than abstract ideas, so it does not integrate into a practical application. Thus, this claim is directed to the abstract idea
Regarding Claim 6
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 6 is a dependent claim of 3, thus it falls within the same category of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding dependent claim 6, following limitations recite a judicial exception:
"the at least one learning data set includes a first learning data set and a second learning data set"
[Mental Process] – Defining, categorizing, or organizing data into multiple distinct data set categories (first and second learning data sets) represents logical grouping and conceptual data classification capable of being performed in the human mind with paper and pen.
"sampling the first data set and the second data set into a training group, a valid group, and a test group of the first learning data set in descending order of decreasing OOD indexes according to the plurality of semiconductors"
[Mental Process / Mathematical Calculations] – Sorting and sampling data sets into training, validation, and test groups in descending order based on calculated OOD indexes represents logical sorting, numerical ranking, and statistical data partitioning capable of being performed in the human mind with the assistance of paper and pen.
"sampling the first data set and the second data set into a training group, a valid group, and a test group of the second learning data set in ascending order of increasing OOD indexes according to the plurality of semiconductors"
[Mental Process / Mathematical Calculations] – Sorting and sampling data sets into training, validation, and test groups in ascending order based on calculated OOD indexes represents logical sorting, numerical ranking, and statistical data partitioning capable of being performed in the human mind with the assistance of paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
The claim 6 does not recite any additional elements other than abstract ideas, so it does not integrate into a practical application. Thus, this claim is directed to the abstract idea.
Regarding Claim 8
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 8 is a dependent claim of 1, thus it falls within the same category of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding dependent claim 8, following limitations recite a judicial exception:
"extracting a first evaluation indicator, from each of the plurality of AI algorithms, by applying test data classified as a test group, of the at least one learning data set in the first data set and the second data set, to each of the plurality of AI algorithms"
[Mathematical Calculations / Mental Process] – Computing or extracting evaluation indicators by applying test data to AI algorithms relies on mathematical calculations, statistical performance metrics, and algorithmic evaluations capable of being performed in the human mind with the assistance of paper and pen.
"extracting a second evaluation indicator from each of the plurality of AI algorithms, by applying valid data classified into a valid group, of the at least one learning data set in the first data set and the second data set, to each of the plurality of AI algorithms"
[Mathematical Calculations / Mental Process] – Computing or extracting evaluation indicators by applying validation data to AI algorithms relies on mathematical calculations, statistical performance metrics, and algorithmic evaluations capable of being performed in the human mind with the assistance of paper and pen.
"determining a ratio of the second evaluation indicator to the first evaluation indicator for each of the plurality of AI algorithms"
[Mathematical Calculations / Mental Process] – Calculating ratios between numerical evaluation indicators represents basic mathematical division/calculations and numerical comparisons capable of being performed in the human mind with the assistance of paper and pen.
"selecting an AI algorithm having a smallest product of the first evaluation indicator and the ratio from among the plurality of AI algorithms as the optimal AI algorithm"
[Mathematical Calculations / Mental Process] – Multiplying indicators, comparing numerical products, and selecting an algorithm with the smallest value involves mathematical multiplication, numerical comparisons, and logical decision-making capable of being performed in the human mind with paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
The claim 8 does not recite any additional elements other than abstract ideas, so it does not integrate into a practical application. Thus, this claim is directed to the abstract idea.
Regarding Claim 9
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 9 is a dependent claim of 8, thus it falls within the same category of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding dependent claim 9, following limitations recite a judicial exception:
"wherein the first evaluation indicator includes a first evaluation indicator within a learning area and a first evaluation indicator outside the learning area"
[Mental Process / Mathematical Calculations] – Categorizing or partitioning evaluation indicators into region-based subsets (within or outside a learning area) represents logical data classification and threshold-based categorization capable of being performed in the human mind with paper and pen.
"extracting the first evaluation indicator within the learning area determined by an OOD index of a trained semiconductor and a first evaluation indicator outside the learning area determined by the OOD index of the trained semiconductor"
[Mathematical Calculations / Mental Process] – Computing or extracting evaluation indicators inside and outside a boundary determined by numerical OOD indexes relies on mathematical calculations, boundary condition evaluations, and statistical metrics capable of being performed in the human mind with the assistance of paper and pen.
"selecting a first AI algorithm having a smallest product of the first evaluation indicator within the learning area and the ratio, and a second AI algorithm having a smallest product of the first evaluation indicator outside the learning area and the ratio as the optimal AI algorithms according to an OOD index of a semiconductor to be predicted"
[Mathematical Calculations / Mental Process] – Calculating products of evaluation indicators and ratios, comparing numerical products across regions, and conditionally selecting first and second AI algorithms based on OOD index comparisons involves mathematical multiplication, numerical ranking, and logical decision-making capable of being performed in the human mind with paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
The claim 9 does not recite any additional elements other than abstract ideas, so it does not integrate into a practical application. Thus, this claim is directed to the abstract idea.
Regarding Claim 10
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 10 is a method claim, thus it falls into one of the four categories of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding independent claim 10, the following limitations recite judicial exceptions:
"generating a plurality of out of distribution (OOD) indexes by determining an OOD index for the spectrum of each of the plurality of semiconductors"
[Mathematical Calculations / Mental Process] – Determining or generating OOD indexes based on spectral data involves calculating statistical distribution metrics, numerical evaluations, and data comparisons capable of being performed in the human mind with paper and pen, as well as mathematical calculations.
"predicting, from the spectrum data and using the AI algorithm, a structure of a semiconductor, of the plurality of semiconductors, when an OOD index of the semiconductor is smaller than a reference value for the AI algorithm"
[Mental Process / Mathematical Calculations] – Comparing numerical OOD indexes against reference values and executing predictive inference via an AI algorithm involves numerical threshold evaluations, logical conditional testing, and mathematical calculations capable of being performed in the human mind with paper and pen.
"predicting the structure of the semiconductor using an optimal AI algorithm, among a plurality of AI algorithms that have been trained, as the AI algorithm to predict the structure of the semiconductor, when the OOD index of the semiconductor is greater than or equal to the reference value"
[Mental Process / Mathematical Calculations] – Comparing numerical OOD indexes against reference values, selecting trained algorithms, and predicting physical structures involves numerical threshold evaluations, logical decision-making, and mathematical model execution capable of being performed in the human mind with paper and pen.
"wherein the optimal AI algorithm is different from the AI algorithm used to predict the structure when the OOD index of the semiconductor is smaller than the reference value"
[Mental Process] – Defining or distinguishing model selection criteria based on conditional index thresholds represents logical decision-making rules and conceptual categorization capable of being performed in the human mind with paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
Regarding Claim 10, the claim recites additional elements of:
"computer", "AI algorithm"
The computer and AI algorithm are recited at a high level of generality and merely add words to "apply it" using generic computer hardware or machine-learning components (see MPEP 2106.05(f)).
"receiving spectrum data indicating information of an actually measured spectrum of each of a plurality of semiconductors"
Receiving spectrum data is mere data gathering recited at a high level of generality, thus is insignificant extra-solution activity (see MPEP 2106.05(g)).
[Even when viewed in combination, the additional elements do no more than automate the mental processes and mathematical calculations that a person could perform using generic computer components as a tool, thus the claim as a whole does not integrate the judicial exception into a practical application.]
Step 2B – whether the claim as a whole amounts to significantly more than the judicial exception? I.e., Are there any additional elements (features/limitations/step) recited in the claim beyond the abstract idea?
The claim does not provide an inventive concept (significantly more than the abstract idea). The claim is ineligible under 35 U.S.C. 101.
As explained above, the additional element [1] is a mere instruction to apply an exception to generic computer components or machine-learning components that simply run mathematical calculations and mental processes (see MPEP 2106.05(f)).
The additional element [2] is considered an insignificant extra-solution activity and at best the equivalent of mere data gathering at a high level of generality and amount to receiving data over a network, which is well-understood, routine, conventional activity (see MPEP 2106.05(d), subsection II, MPEP 2106.05(g)).
These limitations remain insignificant extra-solution activity and a mere instruction to apply an exception even upon reconsideration. Even when considered in combination, the additional elements represent a mere instruction to apply an exception and insignificant extra-solution activity, which cannot provide an inventive concept.
Regarding Claim 11
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 11 is a dependent claim of 10, thus it falls within the same category of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding dependent claim 11, following limitations recite a judicial exception:
"calculating the plurality of OOD indexes based on the first data set"
[Mathematical Calculations / Mental Process] – Calculating out-of-distribution (OOD) indexes based on spectral data involves calculating statistical distribution metrics, numerical evaluations, and data comparisons capable of being performed in the human mind with paper and pen, as well as mathematical calculations.
"performing a data split on the first data set and the second data set, by cluster sampling the first data set and the second data set into at least one learning data set with respect to the OOD indexes according to the plurality of semiconductors"
[Mental Process / Mathematical Calculations] – Partitioning and splitting data sets based on calculated numerical OOD indexes, clustering data points, and sampling data represent logical sorting, categorization, and statistical data manipulation capable of being performed in the human mind with paper and pen.
"wherein the optimal AI algorithm was trained on the at least one learning data set"
[Mathematical Calculations / Mental Process] – Training artificial intelligence algorithms using learning data sets relies on mathematical calculations, statistical optimization, matrix operations, and probabilistic updates, which recite an abstract idea.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
Regarding Claim 11, the claim recites additional elements of:
"loading a first data set including the spectrum data and a second data set including structure measurement data representing information of actually measured structures of the plurality of semiconductors respectively having the plurality of OOD indexes"
loading data is mere data gathering recited at a high level of generality, thus is insignificant extra-solution activity (see MPEP 2106.05(g)).
[Even when viewed in combination, the additional elements do no more than automate the mental processes and mathematical calculations that a person could perform using generic computer components as a tool, thus the claim as a whole does not integrate the judicial exception into a practical application.]
Step 2B – whether the claim as a whole amounts to significantly more than the judicial exception? I.e., Are there any additional elements (features/limitations/step) recited in the claim beyond the abstract idea?
The claim does not provide an inventive concept (significantly more than the abstract idea). The claim is ineligible under 35 U.S.C. 101.
As explained above, the additional element [1] is considered an insignificant extra-solution activity and at best the equivalent of mere data gathering at a high level of generality and amount to receiving data over a network, which is well-understood, routine, conventional activity (see MPEP 2106.05(d), subsection II, MPEP 2106.05(g)).
This limitation remains insignificant extra-solution activity even upon reconsideration. Even when considered in combination, the additional element represents insignificant extra-solution activity, which cannot provide an inventive concept.
Regarding Claims 12-15
Claims 12,13 and 15 have similar limitations of Claims 2, 6, 8 respectively. For the reasons described above with respect to Claim 2, 6, 8, these judicial exceptions are not meaningfully integrated into a practical application, or significantly more than the abstract ideas. The claims do not provide anything more than the abstract ideas of mental processes and mathematical calculations that are practically capable of being performed with the assistance of pen and paper. Therefore, claims 12, 13 and 15 also recite abstract ideas that do not integrate into a practical application or amount to significantly more than judicial exception, and thus are rejected under U.S.C. 101.
Regarding Claim 16
Step 1 – whether the claim falls within any statutory category. See MPEP 2106.03
Claim 16 is a device claim, thus it falls into one of the four categories of statutory subject matter.
Step 2A Prong 1 – whether the claim recites a judicial exception. See MPEP 2106.04, subsection II. Regarding independent claim 16, the following limitations recite judicial exceptions:
"determine an out of distribution (OOD) index with respect to the spectrum of the semiconductor for each of the least one semiconductor based on the first data set"
[Mathematical Calculations / Mental Process] – Determining an out-of-distribution (OOD) index based on spectral data involves calculating statistical distribution metrics, numerical evaluations, and data comparisons capable of being performed in the human mind with the assistance of paper and pen, as well as mathematical calculations.
"perform a data split on the first data set and the second data set, by cluster sampling the first data set and the second data set into at least one learning data set with respect to the OOD index according to the at least one semiconductor"
[Mental Process / Mathematical Calculations] – Clustering data, partitioning/splitting data sets based on calculated numerical indexes, and sampling data points represent logical sorting, categorization, and mathematical data manipulation capable of being performed in the human mind with paper and pen.
"train a plurality of AI algorithms using the first data set and the second data set such that at least one of the plurality of AI algorithms is trained using the at least one learning data set"
[Mathematical Calculations / Mental Process] – Training artificial intelligence algorithms relies on mathematical calculations, statistical optimization, matrix operations, and probabilistic updates, which recite an abstract idea.
"provide an optimal AI algorithm from among the plurality of AI algorithms that have been trained on the at least one learning data set"
[Mental Process / Mathematical Calculations] – Comparing performance metrics of trained algorithms to select and provide an optimal algorithm involves numerical evaluations, logical decision-making, and ranking capable of being performed in the human mind with paper and pen.
"wherein the OOD index represents a distribution of the first data set having a different distribution from a distribution of learning data used to train the plurality of AI algorithms."
[Mental Process / Mathematical Calculations] – Defining or describing the OOD index as representing differences between statistical distributions involves mental observations, statistical comparisons, and conceptual evaluations capable of being performed in the human mind with paper and pen.
Step 2A Prong 2 – whether the claim recites additional elements that integrate the exception into a practical application of the exception?
Regarding Claim 16, the claim recites additional elements of:
"memory", "processor"
The memory and processor are recited at a high level of generality and merely add words to "apply it" using generic computer hardware or machine-learning components (see MPEP 2106.05(f)).
"load a first data set and a second data set, the first data set representing a spectrum of at least one semiconductor, and the second data set representing a structure of the at least one semiconductor"
Loading data sets representing semiconductor spectrums and structures is mere data gathering recited at a high level of generality, thus is insignificant extra-solution activity (see MPEP 2106.05(g)).
[Even when viewed in combination, the additional elements do no more than automate the mental processes and mathematical calculations that a person could perform using generic computer components as a tool, thus the claim as a whole does not integrate the judicial exception into a practical application.]
Step 2B – whether the claim as a whole amounts to significantly more than the judicial exception? I.e., Are there any additional elements (features/limitations/step) recited in the claim beyond the abstract idea?
The claim does not provide an inventive concept (significantly more than the abstract idea). The claim is ineligible under 35 U.S.C. 101.
As explained above, the additional element [1] is considered merely computer components that are just to store and execute code-based instructions which are considered a mere instruction to apply an exception and amount to storing and receiving information in memory, which is well-understood, routine, conventional activity (see MPEP 2106.05(d), subsection II, MPEP 2106.05(f)).
The additional element [2] is considered an insignificant extra-solution activity and at best the equivalent of mere data gathering at a high level of generality and amount to receiving data, which is well-understood, routine, conventional activity (see MPEP 2106.05(d), subsection II, MPEP 2106.05(g)).
These limitations remain insignificant extra-solution activity and a mere instruction to apply an exception even upon reconsideration. Even when considered in combination, the additional elements represent a mere instruction to apply an exception and insignificant extra-solution activity, which cannot provide an inventive concept.
Regarding Claims 17-20
Claims 17-20 have similar limitations of Claims 2, 3, 8 and 9 respectively. For the reasons described above with respect to Claim 2, 3, 8 and 9 these judicial exceptions are not meaningfully integrated into a practical application, or significantly more than the abstract ideas. The claims do not provide anything more than the abstract ideas of mental processes and mathematical calculations that are practically capable of being performed with the assistance of pen and paper or using computer components as a tool. Therefore, claims 17-20 also recite abstract ideas that do not integrate into a practical application or amount to significantly more than judicial exception, and thus are rejected under U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 16, 18 are rejected under 35 U.S.C. 103 as being unpatentable over David et al. (David), US Patent, US 10,430,719 B2, Published in Oct 1, 2019, in view Inkawhich et al. (Inkawhich), Non-Patent Literature(NPL), “Improving Out-of-Distribution Detection by Learning From the Deployment Environment”, published on Jan 27 2022, pages: 17, in further view of Li et al. (Li), NPL, “Improving Image Clustering through Sample Ranking and Its Application to remote-sensing images”, published in 26 Sep 2022, Pages: 39.
As to independent Claim 1,
David teaches a method of providing an artificial intelligence (AI) algorithm, the method comprising:
loading a first data set and a second data set, the first data set representing a spectrum of at least one semiconductor, and the second data set representing a structure of the at least one semiconductor (
David, Pg21, Col6, Lines5-10, "These metrology measurements can also be thickness data determined by each metrology system, or the refractive index and absorption coefficient. In another example, metrology data can be collected during etch processes… Optical emissions spectra or spectral data from photoluminescence can be utilized as input data."
Pg23, Col10, Lines22-28, "These input parameters can include other metrology measurements from earlier process steps, including optical reflectometry or ellipsometry (normal incident, polarized or unpolarized light, oblique angles of incidence, and varying azimuth angles). These metrology measurements can be inputs to the algorithm as an intensity at a given wavelength."
Pg23, Col10, Lines31-38, "If the reflectometry data is collected by illuminating the target… get a total of 301 spectral intensity measurements… These 301 samples would each be an input to the algorithm"
Pg21, Col5, Lines10-13, "… key performance indicators (KPIs) of the process steps, such as the critical dimensions of lithographic and etch steps 202, the dielectric file thickness 204…"
Pg26, Col15, Lines59-67, "CD measurements such as film thickness, trench depth, metal gate recess, high k recess, side wall angle… are used as inputs to the training model"
Pg23, Col9, Lines54-57, "In step 604, the parameters that are useful in evaluating the target are identified, and in step 606, input data relevant to the parameters is collected. Every set of input data is associated with a specific output or target."
Wherein David explicitly teaches loading a first dataset representing optical/spectral measurements (e.g., optical emissions spectra, reflectometry intensities across wavelengths) of semiconductor devices; and a second dataset representing semiconductor structural/target features (e.g., film thickness, critical dimensions, gate structures) for training AI/machine learning algorithms)
training a plurality of AI algorithms using the first data set and the second data set such that at least one of the plurality of AI algorithms is trained using the at least one learning data set; and providing an optimal AI algorithm, from among the plurality of AI algorithms that have been trained (
David, Pg24, Col12, Lines3-13, "In step 612, the data is then fed into the algorithm for training. The algorithm could be one of many different types of algorithms. Examples of machine learning algorithms include Decision Trees, such as CART (Classification and Regression Trees), C5.0, C4.5, and CHAID; Support Vector Regression; Artificial Neural Networks, including Perceptron, Back Propagation, and Deep Learning (BigData enabled); and Ensemble, including Boosting/Bagging, Random Forests, and GBM (Gradient Boosting Machine). The best algorithm may not be a single algorithm, but can be an ensemble of algorithms."
Pg25, Col13, Lines19-24, "As new input data and corresponding target data is generated, the algorithm can be retrained so as to produce a better model that will give better scores. A set of algorithms can be trained simultaneously with the same input and target dataset. The algorithm that gives the best output can be the algorithm that is ultimately deployed."
Wherein David explicitly teaches training a plurality of AI algorithms (e.g., Decision Trees, Neural Networks, Random Forests, GBM) simultaneously using the input dataset and target dataset, and providing/deploying an optimal AI algorithm that yields the best output from among the plurality of trained algorithms.)
However, David does not explicitly teach (Highlighted Part)
determining an out of distribution (OOD) index with respect to the spectrum of the semiconductor for each of the at least one semiconductor based on the first data set
wherein the OOD index represents a distribution of the first data set having a different distribution from a distribution of learning data used to train the plurality of AI algorithms
In the same field of endeavor, Inkawhich teaches them (
Inkawhich, Pg2070, Right Column(RC), Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings [5]–[7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8]–[10]."
Pg2073, LC, P2(Paragraph2), 2), Lines2-5, "In addition to the ID-only empirical training distribution
PNG
media_image1.png
30
42
media_image1.png
Greyscale
, OE introduces another empirical distribution
PNG
media_image2.png
40
60
media_image2.png
Greyscale
which contains samples from the wider OOD set."
Pg2073, RC, Section C, P2, Lines3-10, Eq7, "For input x and temperature T, the ODIN detector probes the DNN's logit vector
PNG
media_image3.png
29
82
media_image3.png
Greyscale
and produces a score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
which is the maximum value from a temperature-scaled softmax vector, as described by Eq7.
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD (and the system will abstain from releasing the classification prediction)."
PNG
media_image7.png
73
481
media_image7.png
Greyscale
Wherein Inkawhich explicitly teaches determining an out-of-distribution (OOD) index/score (e.g.,
PNG
media_image4.png
26
92
media_image4.png
Greyscale
for input data, wherein the OOD index represents whether an input dataset belongs to a distribution different from the training/learning dataset distribution (D_id vs. D_ood).)
David and Inkawhich are analogous to the claimed invention as they are from the same field of endeavor of machine learning algorithms and data processing for semiconductor manufacturing and feature prediction. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date, to combine the teachings of David with the OOD index determination and distribution evaluation of Inkawhich.
The motivation is as recited by Inkawhich (
Inkawhich, Pg2070, Right Column(RC), Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings [5]–[7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8]–[10].")
such that a PHOSITA would be motivated to integrate Inkawhich's OOD index determination into David's semiconductor machine learning pipeline in order to detect anomalous or out-of-distribution input datasets that deviate from the model's training distribution, thereby preventing unreliable or erroneous high-confidence predictions on unseen semiconductor spectrum data.
However, both David and Inkawhich do not explicitly teach (Highlighted Part)
performing a data split on the first data set and the second data set by cluster sampling the first data set and the second data set into at least one learning data set with respect to the OOD index according to the at least one semiconductor
In the same field of endeavor, Li teaches this (
Li, Pg4, Section 3.1, Line1, "Consider K clusters of images where the clusters are formed by deep learning models..."
Pg5, P2, Lines5-7, "We then use majority voting to obtain m groups of samples as Algorithm 1. It is easy to understand that samples in g_i are less likely to be noise than samples in g_{i+1}."
Pg7, Algorithm 1, "Algorithm 1: Sample ranking and robust majority voting... Output: m groups of images... return g_1, g_2, … , g_m"
Pg8, P2, Lines2-3, "For the m groups of data, we should give different weights to their loss since the reliability of the data is different. Therefore, we modify the loss function as Eq9"
PNG
media_image8.png
50
161
media_image8.png
Greyscale
Pg9, EQ10, Pg10, EQ11
PNG
media_image9.png
21
187
media_image9.png
Greyscale
PNG
media_image10.png
53
187
media_image10.png
Greyscale
wherein Li explicitly teaches ranking samples within clusters based on confidence/likelihood metrics (M) computed from distances in feature space, and splitting/partitioning the samples into m learning dataset groups (g_1, g_2, … , g_m) to form weighted loss functions for training AI/neural network models.)
David, Inkawhich, and Li are in the same field of endeavor of machine learning data processing systems, and/or are reasonably pertinent to the particular problem of optimizing training datasets and neural network model training for feature prediction and classification. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine the teachings of David, Inkawhich, and Li.
The motivation to combine is as recited by Li (
Li, Pg2, Introduction, P3, Lines9-11, "keeping inappropriate pseudo labels would mislead model training... make use of most samples in training by developing a method to rank the samples in each cluster based on their confidence of belonging to the cluster..."
Pg8, P2, Lines2-3, "For the m groups of data, we should give different weights to their loss since the reliability of the data is different.")
such that a PHOSITA would have been motivated to integrate Li's cluster-based sample ranking and data splitting methodology into David's semiconductor machine learning framework (incorporating Inkawhich's OOD score calculation) in order to systematically partition semiconductor spectral and structural datasets into optimized learning datasets based on their OOD/confidence distribution characteristics, thereby preventing misleading model training from out-of-distribution samples, optimizing training dataset quality, and enhancing the overall accuracy and prediction performance of the trained AI models.
As to dependent Claim 3,
The combination of David, Inkawhich, and Li teaches all the limitations of Claim 1 as set forth above.
David further teaches the method of claim 1, wherein
the at least one semiconductor includes a plurality of semiconductors (
David, Pg21, Col6, Lines5-10, "These metrology measurements can also be thickness data determined by each metrology system, or the refractive index and absorption coefficient. In another example, metrology data can be collected during etch processes… Optical emissions spectra or spectral data from photoluminescence can be utilized as input data."
Pg26, Col15, Lines48-50, "An overlay process can be performed on one or more training wafers, and the training wafers are then analyzed for actual overlay errors"
Pg30, Col23, Lines49-51, "The input dataset can contain a set of die manufactured throughout the manufacturing process with associated input data for each die..."
wherein David explicitly teaches collecting metrology data and training AI algorithms across a plurality of semiconductors, such as a plurality of wafers and dies.)
Also, the combination of David and Inkawhich further teaches
the performing of the data split includes sampling the dataset at a preset ratio into a training group, a valid group, and a test group of each allocated learning data set according to the OOD indexes of the plurality of semiconductors (
David, Pg26, Col16, Lines60-63, "The training dataset can be partitioned into training, testing, and validation portions to ensure a robust model is built that is not over-fit or over-biased. A typical partition can be 60% training, 30% testing, and 10% validation"
Inkawhich, Pg2070, Right Column(RC), Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings [5]–[7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8]–[10]."
Inkawhich, Pg2073, LC, P2(Paragraph2), 2), Lines2-5, "In addition to the ID-only empirical training distribution
PNG
media_image1.png
30
42
media_image1.png
Greyscale
, OE introduces another empirical distribution
PNG
media_image2.png
40
60
media_image2.png
Greyscale
which contains samples from the wider OOD set."
Inkawhich, Pg2073, RC, Section C, P2, Lines3-10, Eq7, "For input x and temperature T, the ODIN detector probes the DNN's logit vector
PNG
media_image3.png
29
82
media_image3.png
Greyscale
and produces a score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
which is the maximum value from a temperature-scaled softmax vector, as described by Eq7.
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD (and the system will abstain from releasing the classification prediction)."
PNG
media_image7.png
73
481
media_image7.png
Greyscale
wherein David explicitly teaches partitioning training datasets (comprising paired first spectral datasets and second structural datasets) into a training group, a validation group, and a test group at a preset ratio, e.g., 60% training, 30% testing, and 10% validation. Thus, combining David’s partitioning practice with Inkawhich’s OOD standards can partition the first data set and the second data set at a preset ratio according to the OOD indexes of the plurality of semiconductors.)
David and Inkawhich are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing systems, model training, and dataset partitioning for semiconductor manufacturing and feature prediction. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David with the OOD index determination of Inkawhich.
The motivation to combine is as recited by David (
David, Pg26, Col16, Lines60-63, "The training dataset can be partitioned into training, testing, and validation portions to ensure a robust model is built that is not over-fit or over-biased. A typical partition can be 60% training, 30% testing, and 10% validation")
such that a PHOSITA would have been motivated to apply David's preset ratio partitioning practice (e.g., 60% training, 30% testing, and 10% validation) to each learning dataset structured according to Inkawhich's OOD indexes in order to ensure that each OOD-partitioned dataset is evaluated across distinct training, validation, and test groups, thereby preventing model overfitting or over-biasing and enhancing the robustness and prediction accuracy of the trained AI models across varying out-of-distribution spectra.
As to independent Claim 16,
it is a device claim of Claim 1 which contains similar limitations of Claim 1 and thus rejected under the same rationale.
As to dependent Claim 18,
it is a device claim of Claim 3 which contains similar limitations of Claim 3 and thus rejected under the same rationale.
Claims 2 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over David, Inkawhich and Li as discussed above in Claim 1, in view of Guang-qi et al. (Guang), Chinese Patent CN-102039277A, English Translated Version, and further in view of Zhang et al (Zhang), NPL, “Evaluation of Similarity measure methods for hyperspectral remote sensing data”, published on 2012, Pages: 4.
As to dependent Claim 2,
The combination of David, Inkawhich, and Li teaches all the limitations of Claim 1 as set forth above.
However, the combination of David, Inkawhich, and Li does not explicitly teach
extracting a first principal component and a second principal component by performing principal component analysis (PCA) to reduce a dimension of the first data set
In the same field of endeavor, Guang teaches extracting a first principal component and a second principal component by performing principal component analysis (PCA) to reduce a dimension of the first data set (
Guang, Pg1, Abstract, Lines3-5, "WAT parameters are screened from the WAT data of the known wafers; the screened WAT parameters are subject to orthogonal analysis to obtain principal components which are selected"
Pg8, P2, Lines8-11, "Using this dimensionality reduction idea, a principal component analysis was generated, that is, through dimensionality reduction, several synthesis factors (principal components) were found to represent many variables"
wherein Guang explicitly teaches extracting principal components (first and second principal components) with respect to an input dataset by performing principal component analysis (PCA) to reduce the dimension of the dataset).
David, Inkawhich, Li and Guang are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing systems, feature extraction, and semiconductor metrology. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date, to combine the teachings of David, Inkawhich, and Li with Guang, specifically by incorporating Guang's principal component analysis (PCA) dimensionality reduction technique into the spectral data processing pipeline of David, Inkawhich, and Li prior to determining Inkawhich's OOD indices and performing Li's cluster sampling.
The motivation is as recited by Guang (
Guang, Pg8, P2, Lines4-9, "In most practical problems, there is a certain correlation between variables, so that the information reflected by the observed data overlaps to a certain extent. People naturally want to replace the original variables with fewer variables, and these fewer variables reflect the information of the original variables as much as possible. Using this dimensionality reduction idea, a principal component analysis was generated...")
such that a PHOSITA would be motivated to integrate Guang's PCA dimensionality reduction methodology into the combined pipeline of David, Inkawhich, and Li in order to compress high-dimensional semiconductor spectrum data into principal components prior to calculating OOD indices, thereby eliminating redundant feature correlations, reducing noise, and lowering computational complexity without losing critical spectral information.
However, David, Inkawhich, Li and Guang do not explicitly teach (Highlighted Part)
determining a value based on a Euclidean distance between vectors and a cosine distance between the vectors and an origin
extracting a normalized value as the OOD index by normalizing an average value of a product of the Euclidean distance and the cosine distance
In the same field of endeavor, Zhang teaches these (
Zhang, Pg4138, Section 2.1, Lines1-6, "According to the shape and amplitude features of the hyperspectral data, we choose five widely used similarity measure methods for evaluation... The first type is sensitive to spectral amplitude, such as the Euclidean distance (ED) [6]. The second type is sensitive to spectral shape, such as the spectral angel cosine (SAC) [7]..."
Pg4138, Abstract, Lines6-13, "The results indicate that the spectral angle cosine-Euclidean distance method demonstrates the most superior ability to distinguish various land cover types among five methods because it fully utilizes both the spectral amplitude and shape feature in the hyperspectral data. A combination of the spectral amplitude-sensitive method and the shape-sensitive method will effectively improve the identification accuracy of different land cover types"
Pg4141, Conclusion, P2, Lines4-7, "However, this method only considers the product of the spectral angle cosine and the Euclidean distance. In certain case, this integration is more sensitive to the spectral profile shape of the hyperspectral data..."
wherein Zhang explicitly teaches determining values based on Euclidean distance and cosine distance between feature vectors and an origin, and normalizing the product of Euclidean distance and cosine distance to form an index/score for discriminating spectral data).
David, Inkawhich, Li, Guang, and Zhang are analogous to the claimed invention as they are from the same field of endeavor of spectral data analysis and feature index calculation. Therefore, it would have been obvious to a PHOSITA, before the effective filing date of the claimed invention, to combine David, Inkawhich, Li, and Guang with the spectral angle cosine-Euclidean distance (SAC-ED) calculation of Zhang.
The motivation to combine is as recited by Zhang (
Zhang, Pg4138, Abstract, Lines8-13, "A combination of the spectral amplitude-sensitive method and the shape-sensitive method will effectively improve the identification accuracy of different land cover types"
Pg4141, Conclusion, P2, Lines4-7, "the spectral angle cosine-Euclidean distance method has taken the amplitude and shape characteristics of the spectral profile into consideration, and achieves a higher accuracy to distinguish different land covers.")
such that a PHOSITA would be motivated to integrate Zhang's spectral angle cosine-Euclidean distance (SAC-ED) calculation into the combined framework of David, Inkawhich, Li, and Guang (applying it within the dimensionally reduced PCA feature space) in order to comprehensively evaluate both the magnitude (amplitude) and directional (shape) characteristics of the reduced semiconductor spectrum vectors, thereby maximizing the discrimination power and accuracy of the extracted OOD index.
As to dependent Claim 17,
it is a device claim of Claim 2 which contains similar limitations of Claim 2 and thus rejected under the same rationale.
Claims 4, 5, 6 are rejected under 35 U.S.C. 103 as being unpatentable over David, Inkawhich and Li as discussed above in Claim 3, in view of Zhi-huang et al. (Zhihuang), Chinese patent, CN-111178399A, English Translated Version.
As to dependent Claim 4,
The combination of David, Inkawhich, and Li teaches all the limitations of Claim 3 as set forth above.
As mentioned in Claim3, David and Inkawhich teach the method of claim 3,
wherein the performing of the data split includes sampling the first data set and the second data set into the training group, the valid group, and the test group of learning data set, according to the OOD indexes of the plurality of semiconductors (
David, Pg26, Col16, Lines60-63, "The training dataset can be partitioned into training, testing, and validation portions to ensure a robust model is built that is not over-fit or over-biased. A typical partition can be 60% training, 30% testing, and 10% validation"
Inkawhich, Pg2070, Right Column(RC), Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings [5]–[7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8]–[10]."
Inkawhich, Pg2073, LC, P2(Paragraph2), 2), Lines2-5, "In addition to the ID-only empirical training distribution
PNG
media_image1.png
30
42
media_image1.png
Greyscale
, OE introduces another empirical distribution
PNG
media_image2.png
40
60
media_image2.png
Greyscale
which contains samples from the wider OOD set."
Inkawhich, Pg2073, RC, Section C, P2, Lines3-10, Eq7, "For input x and temperature T, the ODIN detector probes the DNN's logit vector
PNG
media_image3.png
29
82
media_image3.png
Greyscale
and produces a score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
which is the maximum value from a temperature-scaled softmax vector, as described by Eq7.
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD (and the system will abstain from releasing the classification prediction)."
PNG
media_image7.png
73
481
media_image7.png
Greyscale
wherein David explicitly teaches partitioning training datasets (comprising paired first spectral datasets and second structural datasets) into a training group, a validation group, and a test group at a preset ratio, e.g., 60% training, 30% testing, and 10% validation. Thus, combining David’s partitioning practice with Inkawhich’s OOD standards can partition the first data set and the second data set at a preset ratio according to the OOD indexes of the plurality of semiconductors.)
However, David, Inkawhich and Li do not explicitly teach (Highlighted Part)
wherein the performing of the data split includes sampling the first data set and the second data set into the training group, the valid group, and the test group of learning data set, sequentially according to the OOD indexes of the plurality of semiconductors
In the same field of endeavor, Zhihuang teaches this (
Zhihuang, Pg3, Claim 4, Lines 4-11, "Arrange the features in descending order according to the relevance of the features to obtain an ordered feature sequence; Select the first M1 features from the ordered feature sequence, where M1 is a positive integer greater than or equal to 1; Clustering the object based on the first M1 features; If after clustering the objects based on the first M1 features, the ratio of the number of objects with a small proportion to the number of objects with a large proportion is 1:N, then the clustering result of the object is used as the clustering result"
Pg22, paragraph 5, Line 4, "Each time a new feature in the ordered feature sequence is added sequentially"
wherein ordering features/indexes and performing stepwise sequential selection and clustering corresponds to the sequential data partitioning.)
David, Inkawhich, Li, and Zhihuang are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing and partitioning for model training. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, and Li with Zhihuang, specifically by sequentially sampling the dataset into training, validation, and test sets according to the ordered OOD indexes.
The motivation to combine is as recited by Zhihuang (
Zhihuang, Pg7, P4, Lines11-13, "By adding minority samples, the problem of imbalance of data categories can be solved to ensure that the information of minority samples is not covered")
such that a PHOSITA would have been motivated to integrate Zhihuang's sequential ordering and feature-based partitioning methodology into the combined framework of David, Inkawhich, and Li in order to sequentially sample semiconductor spectral and structural datasets into training, validation, and test groups according to their OOD index order, thereby ensuring that minority or extreme OOD spectral characteristics are systematically distributed across learning groups without being covered or overlooked, achieving balanced dataset partitioning, and ultimately improving the training stability, generalization, and prediction performance of the trained AI models.
As to dependent Claim 5,
The combination of David, Inkawhich, and Li teaches all the limitations of Claim 3 as set forth above.
As mentioned in Claim 3, David and Inkawhich teach the method of claim 3,
wherein the performing of the data split includes sampling the first data set and the second data set into the training group, the valid group, and the test group of learning data set, according to the OOD indexes of the plurality of semiconductors (
David, Pg26, Col16, Lines60-63, "The training dataset can be partitioned into training, testing, and validation portions to ensure a robust model is built that is not over-fit or over-biased. A typical partition can be 60% training, 30% testing, and 10% validation"
Inkawhich, Pg2070, Right Column(RC), Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings [5]–[7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8]–[10]."
Inkawhich, Pg2073, LC, P2(Paragraph2), 2), Lines2-5, "In addition to the ID-only empirical training distribution
PNG
media_image1.png
30
42
media_image1.png
Greyscale
, OE introduces another empirical distribution
PNG
media_image2.png
40
60
media_image2.png
Greyscale
which contains samples from the wider OOD set."
Inkawhich, Pg2073, RC, Section C, P2, Lines3-10, Eq7, "For input x and temperature T, the ODIN detector probes the DNN's logit vector
PNG
media_image3.png
29
82
media_image3.png
Greyscale
and produces a score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
which is the maximum value from a temperature-scaled softmax vector, as described by Eq7.
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD (and the system will abstain from releasing the classification prediction)."
PNG
media_image7.png
73
481
media_image7.png
Greyscale
wherein David explicitly teaches partitioning training datasets (comprising paired first spectral datasets and second structural datasets) into a training group, a validation group, and a test group at a preset ratio, e.g., 60% training, 30% testing, and 10% validation. Thus, combining David’s partitioning practice with Inkawhich’s OOD standards can partition the first data set and the second data set at a preset ratio according to the OOD indexes of the plurality of semiconductors.)
However, the combination of David, Inkawhich, and Li does not explicitly teach (Highlighted Part)
wherein the performing of the data split includes sampling the first data set and the second data set into the training group, the valid group, and the test group in ascending order of increasing OOD indexes according to the plurality of semiconductors.
In the same field of endeavor, Zhihuang teaches this (
Zhihuang, Pg3, Claim4, Lines 4-11, "Arrange the features in descending order according to the relevance of the features to obtain an ordered feature sequence; Select the first M1 features from the ordered feature sequence, where M1 is a positive integer greater than or equal to 1; Clustering the object based on the first M1 features; If after clustering the objects based on the first M1 features, the ratio of the number of objects with a small proportion to the number of objects with a large proportion is 1:N, then the clustering result of the object is used as the clustering result"
Pg21, P1, Lines 1-5, "It should be noted that, here is an example of sorting in descending order according to the relevance of the features, but in other embodiments, it is also possible to sort them in ascending order according to the relevance of the features, but the features are selected for clustering in the following steps When, you can select the last M1 features or the last M2 features"
wherein arranging features/indexes in ascending order and selecting feature subsets for clustering corresponds to sampling data in ascending order of increasing OOD indexes.)
David, Inkawhich, Li, and Zhihuang are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing and partitioning for model training. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, and Li with Zhihuang, specifically by sampling the dataset into training, validation, and test groups in ascending order of increasing OOD indexes.
The motivation to combine is as recited by Zhihuang (
Zhihuang, Pg7, P4, Lines11-13, "By adding minority samples, the problem of imbalance of data categories can be solved to ensure that the information of minority samples is not covered")
such that a PHOSITA would have been motivated to integrate Zhihuang's ascending order sorting and feature-based partitioning methodology into the combined framework of David, Inkawhich, and Li in order to sample semiconductor spectral and structural datasets into training, validation, and test groups in ascending order of increasing OOD indexes, thereby systematically structuring the dataset based on increasing OOD complexity, preventing extreme OOD data noise from interfering with initial model optimization, achieving balanced and controlled dataset partitioning, and ultimately improving the training stability, generalization, and prediction performance of the trained AI models.
As to dependent Claim 6,
The combination of David, Inkawhich, and Li teaches all the limitations of Claim 3 as set forth above.
The combination of David and Inkawhich further teaches
the at least one learning data set includes a first learning data set and a second learning data set and the performing of the data split includes sampling the first data set and the second data set into the training group, the valid group, and the test group of learning data set, according to the OOD indexes of the plurality of semiconductors (
David, Pg26, Col16, Lines60-63, "The training dataset can be partitioned into training, testing, and validation portions to ensure a robust model is built that is not over-fit or over-biased. A typical partition can be 60% training, 30% testing, and 10% validation"
Inkawhich, Pg2070, Right Column(RC), Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings [5]–[7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8]–[10]."
Inkawhich, Pg2073, LC, P2(Paragraph2), 2), Lines2-5, "In addition to the ID-only empirical training distribution
PNG
media_image1.png
30
42
media_image1.png
Greyscale
, OE introduces another empirical distribution
PNG
media_image2.png
40
60
media_image2.png
Greyscale
which contains samples from the wider OOD set."
Inkawhich, Pg2073, RC, Section C, P2, Lines3-10, Eq7, "For input x and temperature T, the ODIN detector probes the DNN's logit vector
PNG
media_image3.png
29
82
media_image3.png
Greyscale
and produces a score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
which is the maximum value from a temperature-scaled softmax vector, as described by Eq7.
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD (and the system will abstain from releasing the classification prediction)."
PNG
media_image7.png
73
481
media_image7.png
Greyscale
wherein David explicitly teaches partitioning training datasets (comprising paired first spectral datasets and second structural datasets) into a training group, a validation group, and a test group at a preset ratio, e.g., 60% training, 30% testing, and 10% validation. Thus, combining David’s partitioning practice with Inkawhich’s OOD standards can partition the first data set and the second data set at a preset ratio according to the OOD indexes of the plurality of semiconductors.)
However, the combination of David, Inkawhich, and Li does not explicitly teach (Highlighted Part)
the performing of the data split includes sampling the first data set and the second data set into a training group, a valid group, and a test group of the first learning data set in descending order of decreasing OOD indexes according to the plurality of semiconductors
sampling the first data set and the second data set into a training group, a valid group, and a test group of the second learning data set in ascending order of increasing OOD indexes according to the plurality of semiconductors.
In the same field of endeavor, Zhihuang teaches this (
Zhihuang, Pg3, Claim 4, Lines4-11, "Arrange the features in descending order according to the relevance of the features to obtain an ordered feature sequence; Select the first M1 features from the ordered feature sequence, where M1 is a positive integer greater than or equal to 1; Clustering the object based on the first M1 features; If after clustering the objects based on the first M1 features, the ratio of the number of objects with a small proportion to the number of objects with a large proportion is 1:N, then the clustering result of the object is used as the clustering result"
Pg21, P1, Lines1-5, "It should be noted that, here is an example of sorting in descending order according to the relevance of the features, but in other embodiments, it is also possible to sort them in ascending order according to the relevance of the features, but the features are selected for clustering in the following steps When, you can select the last M1 features or the last M2 features"
wherein arranging features/indexes in both descending and ascending orders and selecting feature subsets for clustering corresponds to sampling data in descending order of decreasing indexes and in ascending order of increasing indexes to form first and second learning data sets.)
David, Inkawhich, Li, and Zhihuang are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing and partitioning for model training. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, and Li with Zhihuang. To the extent Zhihuang performs sorting on feature relevance metrics rather than individual semiconductor samples, it would have been obvious to a PHOSITA to apply Zhihuang's ascending and descending sorting/partitioning methodology directly to individual semiconductor samples according to their calculated OOD indexes, specifically by sampling the dataset into a first learning data set in descending order of decreasing OOD indexes and a second learning data set in ascending order of increasing OOD indexes.
The motivation to combine is as recited by Zhihuang (
Zhihuang, Pg7, P4, Lines11-13, "By adding minority samples, the problem of imbalance of data categories can be solved to ensure that the information of minority samples is not covered")
such that a PHOSITA would have been motivated to integrate Zhihuang's ascending and descending order sorting and feature-based partitioning methodology into the combined framework of David, Inkawhich, and Li in order to sample semiconductor spectral and structural datasets into a first learning data set in descending order of decreasing OOD indexes and a second learning data set in ascending order of increasing OOD indexes, thereby systematically structuring the dataset based on varying OOD complexity, preventing extreme OOD data noise from interfering with initial model optimization, achieving balanced and controlled dataset partitioning, and ultimately improving the training stability, generalization, and prediction performance of the trained AI models.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over David, Inkawhich, Li, and Zhihuang as discussed above in Claim 6, in view of Donghua et al. (Donghua), WIPO Application, WO-2017/161646-A1, English Translated Version.
As to dependent Claim 7,
The combination of David, Inkawhich, Li, and Zhihuang teaches all the limitations of Claim 6 as set forth above.
However, the combination of David, Inkawhich, Li, and Zhihuang does not explicitly teach
the providing of the optimal AI algorithm includes extracting a first evaluation indicator for each of first valid data and first test data from each of the plurality of AI algorithms, by applying the first valid data classified into the valid group of the first learning data in the first data set and the second data set and the first test data classified into the test group of the first learning data set in the first data set and the second data set to each of the plurality of AI algorithms
extracting a second evaluation indicator for each of second valid data and second test data from each of the plurality of AI algorithms, by applying the second valid data classified into the valid group of the second learning data set in the first data set and the second data set and the second test data classified into the test group of the second learning data set in the first data set and the second data set to each of the plurality of AI algorithms
selecting the optimal AI algorithm from among the plurality of AI algorithms, based on a valid root mean square error (RMSE) for the first valid data, a test RMSE for the first test data, a valid RMSE for the second valid data, and a test RMSE for the second test data.
In the same field of endeavor, Donghua teaches these limitations.
Specifically, as to extracting a first evaluation indicator for each of first valid data and first test data from each of the plurality of AI algorithms by applying the first valid data and the first test data of the first learning data set (
Donghua, Pg10, Algorithm3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
Pg12, P4, Lines1-6, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model... with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively"
wherein Donghua explicitly teaches evaluating candidate AI models on partitioned validation and prediction/test datasets to calculate error metrics, specifically root mean square error (RMSE) values for the validation set and prediction/test set. In the combined framework of David, Inkawhich, Li, Zhihuang, and Donghua, applying the first valid data and first test data from the first learning dataset (sampled in descending OOD order) to each candidate AI algorithm to extract validation RMSE and test RMSE metrics corresponds directly to extracting the claimed first evaluation indicator for each of the first valid data and first test data.)
As to extracting a second evaluation indicator for each of second valid data and second test data from each of the plurality of AI algorithms by applying the second valid data and the second test data of the second learning data set (
Donghua, Pg10, Algorithm3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
Pg12, P4, Lines1-6, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model... with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively"
wherein Donghua explicitly teaches applying validation data and test/prediction data to candidate AI models to calculate respective validation and test RMSE metrics. Applying this teaching to the second learning dataset (sampled in ascending OOD order) of David, Inkawhich, Li, and Zhihuang, applying the second valid data and second test data to each candidate AI algorithm to extract validation RMSE and test RMSE metrics corresponds directly to extracting the claimed second evaluation indicator for each of the second valid data and second test data.)
As to selecting the optimal AI algorithm from among the plurality of AI algorithms based on a valid RMSE for the first valid data, a test RMSE for the first test data, a valid RMSE for the second valid data, and a test RMSE for the second test data (
Donghua, Pg11, P1, Lines1-2, "On the basis of the six weighting algorithms, the optimal weight algorithm selection algorithm selects the best weight algorithm"
Pg12, P4, Lines4-7, "the prediction error of the training set and the prediction error of the prediction set are calculated respectively... And finally get the comprehensive error value"
Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
wherein Donghua explicitly teaches evaluating model prediction errors across validation and test sets to compute overall/comprehensive error metrics (specifically RMSE) and selecting the optimal algorithm that yields the minimum error across the evaluated datasets. In the combined framework, selecting the optimal AI algorithm based on the evaluated valid RMSE and test RMSE of the first learning dataset alongside the valid RMSE and test RMSE of the second learning dataset corresponds directly to selecting the optimal AI algorithm based on the valid RMSE for the first valid data, test RMSE for the first test data, valid RMSE for the second valid data, and test RMSE for the second test data.)
David, Inkawhich, Li, Zhihuang, and Donghua are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing, model evaluation, and optimal algorithm selection for predictive systems. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, Li, and Zhihuang with Donghua, specifically by applying Donghua's RMSE evaluation indicator extraction and model selection framework to the first (descending OOD) and second (ascending OOD) partitioned learning datasets (comprising respective valid and test groups) of David, Inkawhich, Li, and Zhihuang.
The motivation to combine is as recited by Donghua (
Donghua, Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
Donghua, Pg10, Algorithm3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
such that a PHOSITA would have been motivated to integrate Donghua's RMSE-based evaluation and algorithm selection methodology into the combined framework of David, Inkawhich, Li, and Zhihuang in order to evaluate each trained AI algorithm across the valid and test groups of both the first (descending OOD) and second (ascending OOD) learning datasets, thereby obtaining precise quantitative error indicators (valid RMSE and test RMSE) across varying OOD complexity distributions and accurately selecting the overall optimal AI algorithm to ensure maximum prediction accuracy, robustness, and stability in semiconductor structure metrology.
Claims 8, 9, 19, 20 are rejected under 35 U.S.C. 103 as being unpatentable over David, Inkawhich, and Li as discussed above in Claim 1, in view of Donghua et al. (Donghua), WIPO Application, WO-2017/161646-A1, English Translated Version.
As to dependent Claim 8,
The combination of David, Inkawhich, and Li teaches all the limitations of Claim 1 as set forth above.
However, the combination of David, Inkawhich, and Li does not explicitly teach
the providing of the optimal AI algorithm includes extracting a first evaluation indicator, from each of the plurality of AI algorithms, by applying test data classified as a test group, of the at least one learning data set in the first data set and the second data set, to each of the plurality of AI algorithms
extracting a second evaluation indicator from each of the plurality of AI algorithms, by applying valid data classified into a valid group, of the at least one learning data set in the first data set and the second data set, to each of the plurality of AI algorithms
determining a ratio of the second evaluation indicator to the first evaluation indicator for each of the plurality of AI algorithms
selecting an AI algorithm having a smallest product of the first evaluation indicator and the ratio from among the plurality of AI algorithms as the optimal AI algorithm.
In the same field of endeavor, Donghua teaches these limitations.
Specifically, as to extracting a first evaluation indicator, from each of the plurality of AI algorithms, by applying test data classified as a test group to each of the plurality of AI algorithms (
Donghua, Pg10, Algorithm 3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
Pg12, P4, Lines1-6, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model... with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively"
wherein Donghua explicitly teaches applying test/prediction data to candidate AI models to calculate prediction error metrics, specifically root mean square error (RMSE), which corresponds directly to extracting the claimed first evaluation indicator from test data.)
As to extracting a second evaluation indicator from each of the plurality of AI algorithms, by applying valid data classified into a valid group to each of the plurality of AI algorithms (
Donghua, Pg10, Algorithm 3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
Pg12, P4, Lines1-6, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model... with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively"
wherein Donghua explicitly teaches applying validation data to candidate AI models to extract validation error metrics (RMSE), which corresponds directly to extracting the claimed second evaluation indicator from valid data.)
As to determining a ratio of the second evaluation indicator to the first evaluation indicator for each of the plurality of AI algorithms (
Donghua, Pg6, Algorithm 2, "Algorithm 2: Excludes the relatively poor model of 20% predicted results and give the same weight to the rest of the model"
Pg12, P4, Lines4-7, "the prediction error of the training set and the prediction error of the prediction set are calculated respectively. Respectively, to the three values of weight, take 0.3, 0.3, 0.4. And finally get the comprehensive error value"
wherein Donghua explicitly teaches determining selection and weighting ratios based on model error metrics—such as excluding the worst 20% or assigning specific weighting ratios—which corresponds directly to determining a ratio of the evaluation indicators.)
As to selecting an AI algorithm having a smallest product of the first evaluation indicator and the ratio from among the plurality of AI algorithms as the optimal AI algorithm (
Donghua, Pg6, Algorithm 2, "Algorithm 2: Excludes the relatively poor model of 20% predicted results and give the same weight to the rest of the model"
Donghua, Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
Pg12, P4, Lines4-7, "the prediction error of the training set and the prediction error of the prediction set are calculated respectively. Respectively, to the three values of weight, take 0.3, 0.3, 0.4. And finally get the comprehensive error value"
wherein Donghua explicitly teaches combining error metrics with weighting ratios to calculate a comprehensive error value for each algorithm configuration and selecting the algorithm configuration yielding the minimum comprehensive error, which corresponds directly to selecting an AI algorithm having a smallest product of the first evaluation indicator and the ratio).
David, Inkawhich, Li, and Donghua are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing, model evaluation, and optimal algorithm selection for predictive systems. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, and Li with Donghua, specifically by applying Donghua's RMSE evaluation indicator extraction, ratio determination, and minimum-error algorithm selection framework to the OOD-partitioned learning datasets (comprising test and validation groups) of David, Inkawhich, and Li.
The motivation to combine is as recited by Donghua (
Donghua, Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
Pg10, Algorithm 3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model")
such that a PHOSITA would have been motivated to integrate Donghua's evaluation indicator extraction, ratio calculation, and minimum-error selection methodology into the combined framework of David, Inkawhich, and Li in order to evaluate candidate AI algorithms across validation and test sets using weighted error ratios, thereby identifying and selecting the optimal AI algorithm that minimizes prediction error and maximizes stability and accuracy in semiconductor structure metrology.
As to dependent Claim 9,
The combination of David, Inkawhich, Li, and Donghua teaches all the limitations of Claim 8 as set forth above.
However, the combination of David, Li, and Donghua does not explicitly teach
wherein the first evaluation indicator includes a first evaluation indicator within a learning area and a first evaluation indicator outside the learning area
extracting the first evaluation indicator within the learning area determined by an OOD index of a trained semiconductor and a first evaluation indicator outside the learning area determined by the OOD index of the trained semiconductor
In the same field of endeavor, Inkawhich further teaches these limitations.
Specifically, as to the first evaluation indicator including a first evaluation indicator within a learning area and a first evaluation indicator outside the learning area (
Inkawhich, Pg2070, Abstract, Lines1-5, "Recognition systems in the remote sensing domain often operate in 'open-world' environments, where they must be capable of accurately classifying data from the in-distribution categories while simultaneously detecting and rejecting anomalous/out-of-distribution (OOD) inputs"
Pg2072, Section III, Lines11-16, "The OOD detector then observes the signal from the classifier and produces a real-valued ID/OOD score. If the ID/OOD score is greater than some predefined threshold, the input is predicted to be ID and the classifier releases its prediction. Else, the sample is deemed to be OOD and the system abstains from releasing any prediction"
Pg2074, Section C,P2, Lines7-9, "
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD"
wherein computing a real-valued ID/OOD score and applying a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
classifies samples into inside the learning area (ID) and outside the learning area (OOD), which inherently corresponds to the first evaluation indicator within and outside the learning area.)
As to extracting the first evaluation indicator within the learning area determined by an OOD index of a trained semiconductor and a first evaluation indicator outside the learning area determined by the OOD index of the trained semiconductor (
Inkawhich, Pg2072, Section III, Lines11-16, "The OOD detector then observes the signal from the classifier and produces a real-valued ID/OOD score. If the ID/OOD score is greater than some predefined threshold, the input is predicted to be ID and the classifier releases its prediction. Else, the sample is deemed to be OOD and the system abstains from releasing any prediction"
Pg2074, Section C,P2, Lines7-9, "
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD"
wherein evaluating the real-valued score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
relative to the threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
for trained samples corresponds directly to extracting the evaluation indicator within the learning area (ID) and outside the learning area (OOD) determined by the OOD index.)
However, Inkawhich alone does not explicitly teach
selecting a first AI algorithm having a smallest product of the first evaluation indicator within the learning area and the ratio, and a second AI algorithm having a smallest product of the first evaluation indicator outside the learning area and the ratio as the optimal AI algorithms according to an OOD index of a semiconductor to be predicted
Thus, Inkawhich in view of Donghua teaches this (
Donghua, Pg6, Algorithm 2, "Algorithm 2: Excludes the relatively poor model of 20% predicted results and give the same weight to the rest of the model"
Donghua, Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
Donghua, Pg12, P4, Lines4-7, "the prediction error of the training set and the prediction error of the prediction set are calculated respectively. Respectively, to the three values of weight, take 0.3, 0.3, 0.4. And finally get the comprehensive error value"
Inkawhich, Pg2072, Section III, Lines11-16, "The OOD detector then observes the signal from the classifier and produces a real-valued ID/OOD score. If the ID/OOD score is greater than some predefined threshold, the input is predicted to be ID and the classifier releases its prediction. Else, the sample is deemed to be OOD and the system abstains from releasing any prediction"
wherein Donghua teaches applying candidate weighting ratios to error indicators to select the algorithm configuration yielding the minimum error/smallest product, and Inkawhich teaches threshold-based classification into ID and OOD regimes. Combining these teachings, computing evaluation indicators inside (ID) and outside (OOD) the learning area, applying each candidate ratio to form combined metrics, and selecting the AI algorithm whose product metric is smallest within each region according to the OOD index of a semiconductor to be predicted corresponds directly to selecting the first and second optimal AI algorithms.)
David, Inkawhich, Li, and Donghua are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing, model evaluation, and OOD-aware algorithm selection. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the metrology prediction framework of David, Li, and Donghua with the threshold-based ID/OOD classification framework of Inkawhich, specifically by extracting first evaluation indicators within and outside the learning area using Inkawhich's thresholding criteria and selecting respective optimal AI algorithms for ID and OOD target regimes using Donghua's minimum weighted product selection methodology.
The motivation to combine is as recited by Inkawhich (
Inkawhich, Pg2070, RC, Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings... they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes..."
Pg2070, Abstract, Lines1-5, "Recognition systems in the remote sensing domain often operate in 'open-world' environments, where they must be capable of accurately classifying data from the in-distribution categories while simultaneously detecting and rejecting anomalous/out-of-distribution (OOD) inputs"
Pg2072, Section III, Lines11-16, "The OOD detector then observes the signal from the classifier and produces a real-valued ID/OOD score. If the ID/OOD score is greater than some predefined threshold, the input is predicted to be ID and the classifier releases its prediction. Else, the sample is deemed to be OOD and the system abstains from releasing any prediction")
and as recited by Donghua (
Donghua, Pg5, Lines4-6, "In forecasting, the most suitable model can be dynamically selected to abandon the model with poor prediction effect... to ensure the stability of the effect, on the other hand the error control in a reasonable range"
Donghua, Pg10, Algorithm 3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model")
such that a PHOSITA would have been motivated to integrate Inkawhich's ID/OOD thresholding framework with Donghua's minimum-error algorithm selection methodology within the David/Li pipeline, because machine learning model performance differs substantially between in-distribution and out-of-distribution data domains. Evaluating and selecting optimal AI algorithms separately for inputs inside the learning area (ID) and outside the learning area (OOD) ensures that specialized, best-performing AI models are dynamically deployed for both ID and OOD operational regimes, thereby preventing high-confidence erroneous predictions and maximizing metrology prediction accuracy and stability across varying semiconductor sample distributions.
As to dependent Claim 19,
it is a device claim of Claim 8 which contains similar limitations of Claim 8 and thus rejected under the same rationale.
As to dependent Claim 20,
it is a device claim of Claim 9 which contains similar limitations of Claim 9 and thus rejected under the same rationale.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over David et al. (David), US Patent, US 10,430,719 B2, Published in Oct 1, 2019, in view Inkawhich et al. (Inkawhich), Non-Patent Literature(NPL), “Improving Out-of-Distribution Detection by Learning From the Deployment Environment”, published on Jan 27 2022, pages: 17, in further view of Donghua et al. (Donghua), WIPO Application, WO-2017/161646-A1, English Translated Version.
As to independent Claim 10,
David teaches an operation method of a computer configured to operate an artificial intelligence (AI) algorithm, the method comprising:
receiving spectrum data indicating information of an actually measured spectrum of each of a plurality of semiconductors (
David, Pg21, Col6, Lines5-10, "These metrology measurements can also be thickness data determined by each metrology system, or the refractive index and absorption coefficient. In another example, metrology data can be collected during etch processes… Optical emissions spectra or spectral data from photoluminescence can be utilized as input data."
Pg23, Col10, Lines22-28, "These input parameters can include other metrology measurements from earlier process steps, including optical reflectometry or ellipsometry (normal incident, polarized or unpolarized light, oblique angles of incidence, and varying azimuth angles). These metrology measurements can be inputs to the algorithm as an intensity at a given wavelength."
Pg30, Col23, Lines49-51, "The input dataset can contain a set of die manufactured throughout the manufacturing process with associated input data for each die."
Pg21, Col5, Lines10-13, "… key performance indicators (KPIs) of the process steps, such as the critical dimensions of lithographic and etch steps 202, the dielectric file thickness 204…"
Pg26, Col15, Lines59-67, "CD measurements such as film thickness, trench depth, metal gate recess, high k recess, side wall angle… are used as inputs to the training model"
Pg23, Col9, Lines54-57, "In step 604, the parameters that are useful in evaluating the target are identified, and in step 606, input data relevant to the parameters is collected. Every set of input data is associated with a specific output or target."
Wherein David explicitly teaches receiving spectrum data representing optical/spectral measurements (e.g., optical emissions spectra, reflectometry intensities across wavelengths) used to predict a structure of a semiconductor representing structural/target features (e.g., film thickness, critical dimensions, gate structures) for training AI/machine learning algorithms);
predicting, from the spectrum data and using the AI algorithm, a structure of a semiconductor, of the plurality of semiconductors (
David, Pg24, Col12, Lines3-13, "In step 612, the data is then fed into the algorithm for training. The algorithm could be one of many different types of algorithms. Examples of machine learning algorithms include Decision Trees... Artificial Neural Networks... and Ensemble... The best algorithm may not be a single algorithm, but can be an ensemble of algorithms."
Pg25, Col13, Lines19-24, "As new input data and corresponding target data is generated, the algorithm can be retrained so as to produce a better model that will give better scores. A set of algorithms can be trained simultaneously with the same input and target dataset. The algorithm that gives the best output can be the algorithm that is ultimately deployed."
wherein David explicitly teaches predicting semiconductor structural/target features, e.g., film thickness, critical dimensions, or channel parameters, from input spectral data using trained AI algorithms, and deploying an optimal AI algorithm from among a plurality of trained algorithms)
predicting the structure of the semiconductor using an optimal AI algorithm, among a plurality of AI algorithms that have been trained, as the AI algorithm to predict the structure of the semiconductor (
David, Pg24, Col12, Lines3-13, "In step 612, the data is then fed into the algorithm for training. The algorithm could be one of many different types of algorithms. Examples of machine learning algorithms include Decision Trees... Artificial Neural Networks... and Ensemble... The best algorithm may not be a single algorithm, but can be an ensemble of algorithms."
Pg25, Col13, Lines19-24, "As new input data and corresponding target data is generated, the algorithm can be retrained so as to produce a better model that will give better scores. A set of algorithms can be trained simultaneously with the same input and target dataset. The algorithm that gives the best output can be the algorithm that is ultimately deployed."
wherein David explicitly teaches training a plurality of AI algorithms simultaneously using the same dataset, selecting/providing an optimal AI algorithm that gives the best output from among the plurality of trained algorithms, and deploying the optimal AI algorithm to predict the semiconductor structure.)
However, David does not explicitly teach (Highlighted Part)
generating a plurality of out of distribution (OOD) indexes by determining an OOD index for the spectrum of each of the plurality of semiconductors
predicting, from the spectrum data and using the AI algorithm, a structure of a semiconductor, of the plurality of semiconductors, when an OOD index of the semiconductor is smaller than a reference value for the AI algorithm;
predicting the structure of the semiconductor using an optimal AI algorithm, among a plurality of AI algorithms that have been trained, as the AI algorithm to predict the structure of the semiconductor, when the OOD index of the semiconductor is greater than or equal to the reference value
In the same field of endeavor, Inkawhich teaches
generating a plurality of out of distribution (OOD) indexes by determining an OOD index for input data (e.g., spectrum of each semiconductor), and evaluating whether the OOD index is relative to a reference value/threshold for triggering algorithmic decisions (
Inkawhich, Pg2072, Section III, Lines11-16, "The OOD detector then observes the signal from the classifier and produces a real-valued ID/OOD score. If the ID/OOD score is greater than some predefined threshold, the input is predicted to be ID and the classifier releases its prediction. Else, the sample is deemed to be OOD..."
Pg2073, RC, Section C, P2, Lines3-10, Eq7, "For input x and temperature T, the ODIN detector probes the DNN's logit vector
PNG
media_image3.png
29
82
media_image3.png
Greyscale
and produces a score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
which is the maximum value from a temperature-scaled softmax vector, as described by Eq7.
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD (and the system will abstain from releasing the classification prediction)."
PNG
media_image7.png
73
481
media_image7.png
Greyscale
wherein Inkawhich explicitly teaches determining an out-of-distribution (OOD) index/score for input data, and comparing the OOD index to a predefined threshold (
PNG
media_image5.png
24
55
media_image5.png
Greyscale
) / reference value to classify the input as ID vs OOD and conditionally perform prediction.)
David and Inkawhich are analogous to the claimed invention as they are from the same field of endeavor of machine learning algorithms and data processing for semiconductor manufacturing and feature prediction. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date, to combine the teachings of David with the OOD index determination and threshold comparison methodology of Inkawhich.
The motivation to combine is as recited by Inkawhich (
Inkawhich, Pg2070, Right Column(RC), Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings [5]–[7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8]–[10].")
such that a PHOSITA would be motivated to integrate Inkawhich's OOD index determination and threshold comparison methodology into David's semiconductor machine learning pipeline in order to detect anomalous or out-of-distribution input spectral datasets that deviate from the model's training distribution, thereby preventing unreliable or erroneous high-confidence structure predictions on unseen semiconductor spectrum data.
However, both David and Inkawhich do not explicitly teach
wherein the optimal AI algorithm is different from the AI algorithm used to predict the structure when the OOD index of the semiconductor is smaller than the reference value
In the same field of endeavor, Donghua teaches this (
Donghua, Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
Pg5, Lines4-6, "In forecasting, the most suitable model can be dynamically selected to abandon the model with poor prediction effect... to ensure the stability of the effect, on the other hand the error control in a reasonable range"
Pg11, P1, Lines1-2, "On the basis of the six weighting algorithms, the optimal weight algorithm selection algorithm selects the best weight algorithm"
Pg12, P4, Lines1-7, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model. Then, the training data will be put into the joint algorithm model to train and forecast, and the prediction data and error data will saved. Finally, the prediction effect of the joint algorithm and the general model is compared with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively. Respectively, to the three values of a weigh, take 0.3,0.3,0.4. And finally get the comprehensive error value"
wherein Donghua explicitly teaches dynamically selecting and deploying an optimal AI algorithm from among a plurality of trained candidate algorithms in a model library when data conditions or performance requirements change, wherein the dynamically selected optimal AI algorithm is different from a previously or default used AI model.)
David, Inkawhich, and Donghua are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing, out-of-distribution detection, and model selection for predictive systems. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, and Donghua. Specifically, it would have been obvious to combine David's semiconductor metrology prediction framework with Inkawhich's OOD index determination and threshold comparison methodology, and further with Donghua's dynamic optimal model selection framework, so as to predict semiconductor structure using a default/first AI algorithm when the OOD index is smaller than a reference value (In-Distribution), and to dynamically switch to and predict using a different optimal AI algorithm selected from a plurality of trained algorithms when the OOD index is greater than or equal to the reference value (Out-of-Distribution).
The motivation to combine is as recited by Donghua (
Donghua, Pg5, Lines4-6, "In forecasting, the most suitable model can be dynamically selected to abandon the model with poor prediction effect... to ensure the stability of the effect, on the other hand the error control in a reasonable range")
such that a PHOSITA would be motivated to integrate Donghua's dynamic model selection methodology into the combined David/Inkawhich framework, so that when input spectrum data triggers an OOD condition (greater than or equal to reference value), the system automatically transitions from the standard AI algorithm to a distinct optimal AI algorithm specifically selected from a library of trained algorithms, thereby ensuring accurate, stable, and reliable structural prediction even when encountering out-of-distribution semiconductor spectra.
Claims 11, 15 are rejected under 35 U.S.C. 103 as being unpatentable over David, Inkawhich and Donghua as discussed above in Claim 10, in further view of Li et al. (Li), NPL, “Improving Image Clustering through Sample Ranking and Its Application to remote-sensing images”, published in 26 Sep 2022, Pages: 39.
As to dependent Claim 11,
The combination of David, Inkawhich, and Donghua teaches all the limitations of Claim 10 as set forth above.
David further teaches the method of claim 10, further comprising:
loading a first data set including the spectrum data and a second data set including structure measurement data representing information of actually measured structures of the plurality of semiconductors (
David, Pg21, Col6, Lines5–10, "These metrology measurements can also be thickness data determined by each metrology system, or the refractive index and absorption coefficient. In another example, metrology data can be collected during etch processes… Optical emissions spectra or spectral data from photoluminescence can be utilized as input data."
Pg23, Col10, Lines22–28, "These input parameters can include other metrology measurements from earlier process steps, including optical reflectometry or ellipsometry (normal incident, polarized or unpolarized light, oblique angles of incidence, and varying azimuth angles). These metrology measurements can be inputs to the algorithm as an intensity at a given wavelength."
Pg26, Col15, Lines48–60, "An overlay process can be performed on one or more training wafers, and the training wafers are then analyzed for actual overlay errors... All available wafer geometry parameters, such as thickness... CD measurements such as film thickness, trench depth, metal gate recess..."
Pg30, Col23, Lines49–51, "The input dataset can contain a set of die manufactured throughout the manufacturing process with associated input data for each die."
wherein David explicitly teaches loading a first dataset including spectrum data (e.g., optical emissions spectra, reflectometry, or ellipsometry intensity measurements) and a second dataset including structure measurement data representing information of actually measured structures (e.g., film thickness, critical dimensions, trench depth, gate height) across a plurality of semiconductors, such as a set of dies or wafers);
wherein the optimal AI algorithm was trained on the at least one learning data set (
David, Pg24, Col12, Lines3-13, "In step 612, the data is then fed into the algorithm for training. The algorithm could be one of many different types of algorithms. Examples of machine learning algorithms include Decision Trees, such as CART (Classification and Regression Trees), C5.0, C4.5, and CHAID; Support Vector Regression; Artificial Neural Networks, including Perceptron, Back Propagation, and Deep Learning (BigData enabled); and Ensemble, including Boosting/Bagging, Random Forests, and GBM (Gradient Boosting Machine). The best algorithm may not be a single algorithm, but can be an ensemble of algorithms."
Pg25, Col13, Lines19–24, "As new input data and corresponding target data is generated, the algorithm can be retrained so as to produce a better model that will give better scores. A set of algorithms can be trained simultaneously with the same input and target dataset. The algorithm that gives the best output can be the algorithm that is ultimately deployed."
David, Pg26, Col16, Lines60-63, "The training dataset can be partitioned into training, testing, and validation portions to ensure a robust model is built that is not over-fit or over-biased. A typical partition can be 60% training, 30% testing, and 10% validation"
wherein David explicitly teaches partitioning a dataset into training/learning portions and training a plurality of machine learning algorithms on the learning dataset to provide and deploy an optimal AI algorithm that yields the best output.)
However, David does not explicitly teach (Highlighted Part)
loading a first data set including the spectrum data and a second data set including structure measurement data representing information of actually measured structures of the plurality of semiconductors respectively having the plurality of OOD indexes
calculating the plurality of OOD indexes based on the first data set
performing a data split on the first data set and the second data set, by cluster sampling the first data set and the second data set into at least one learning data set with respect to the OOD indexes according to the plurality of semiconductors
In the same field of endeavor, Inkawhich further teaches them (
Inkawhich, Pg2072, Section III, Lines11-16, "The OOD detector then observes the signal from the classifier and produces a real-valued ID/OOD score. If the ID/OOD score is greater than some predefined threshold, the input is predicted to be ID and the classifier releases its prediction. Else, the sample is deemed to be OOD and the system abstains from releasing any prediction"
Pg2073, RC, Section C, P2, Lines3-10, Eq7, "For input x and temperature T, the ODIN detector probes the DNN's logit vector
PNG
media_image3.png
29
82
media_image3.png
Greyscale
and produces a score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
which is the maximum value from a temperature-scaled softmax vector, as described by Eq7.
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD (and the system will abstain from releasing the classification prediction)."
PNG
media_image7.png
73
481
media_image7.png
Greyscale
wherein Inkawhich explicitly teaches calculating a plurality of OOD indexes/scores (e.g.,
PNG
media_image4.png
26
92
media_image4.png
Greyscale
or ID/OOD score) based on the first input dataset, particularly the spectrum data of wafers once combined with David’s framework.)
David and Inkawhich are analogous to the claimed invention as they are from the same field of endeavor of machine learning algorithms and data processing for semiconductor manufacturing and feature prediction. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David with the OOD index determination and distribution evaluation of Inkawhich.
The motivation to combine is as recited by Inkawhich (
Inkawhich, Pg2070, Right Column, Lines 4-10, "While DNNs have shown to be exceptionally accurate in many settings... they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes.")
such that a PHOSITA would have been motivated to integrate Inkawhich's OOD index determination into David's semiconductor machine learning pipeline in order to detect anomalous or out-of-distribution input spectral datasets that deviate from the model's training distribution, thereby preventing unreliable or erroneous high-confidence structure predictions on unseen semiconductor spectrum data.
However, David, Inkawhich and Donghua do not explicitly teach (Highlighted Part)
performing a data split on the first data set and the second data set, by cluster sampling the first data set and the second data set into at least one learning data set with respect to the OOD indexes according to the plurality of semiconductors
In the same field of endeavor, Li teaches this (
Li, Pg4, Section 3.1, Line1, "Consider K clusters of images where the clusters are formed by deep learning models..."
Pg5, P2, Lines5-7, "We then use majority voting to obtain m groups of samples as Algorithm 1. It is easy to understand that samples in g_i are less likely to be noise than samples in g_{i+1}."
Pg7, Algorithm 1, "Algorithm 1: Sample ranking and robust majority voting... Output: m groups of images... return g_1, g_2, … , g_m"
Pg8, P2, Lines2-3, "For the m groups of data, we should give different weights to their loss since the reliability of the data is different. Therefore, we modify the loss function as Eq9"
PNG
media_image8.png
50
161
media_image8.png
Greyscale
Pg9, EQ10, Pg10, EQ11
PNG
media_image9.png
21
187
media_image9.png
Greyscale
PNG
media_image10.png
53
187
media_image10.png
Greyscale
wherein Li explicitly teaches ranking samples within clusters based on confidence/likelihood metrics (M) computed from distances in feature space, and splitting/partitioning the samples into m learning dataset groups (g_1, g_2, … , g_m) to form weighted loss functions for training AI/neural network models.)
David, Inkawhich, Donghua and Li are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing systems, model selection, and dataset partitioning for predictive models. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine the teachings of David, Inkawhich, and Donghua with Li, specifically by incorporating Li's cluster sampling and dataset partitioning methodology into the OOD-aware metrology prediction framework of David, Inkawhich, and Donghua.
The motivation to combine is as recited by Li (
Li, Pg2, Introduction, P3, Lines9-11, "keeping inappropriate pseudo labels would mislead model training... make use of most samples in training by developing a method to rank the samples in each cluster based on their confidence of belonging to the cluster..."
Pg8, P2, Lines2-3, "For the m groups of data, we should give different weights to their loss since the reliability of the data is different.")
such that a PHOSITA would have been motivated to integrate Li's cluster sampling and dataset partitioning methodology into the combined pipeline of David, Inkawhich, and Donghua in order to systematically partition semiconductor spectral and structural datasets into optimized learning datasets based on their OOD/confidence characteristics within clusters. This prevents misleading model training from noise or extreme out-of-distribution samples, optimizes training dataset quality for each target operational regime, and ultimately improves the training stability, generalization, and prediction accuracy of the trained optimal AI models.
As to dependent Claim 15,
The combination of David, Inkawhich, Donghua, and Li teaches all the limitations of Claim 11 as set forth above.
However, the combination of David, Inkawhich, Donghua, and Li does not explicitly teach
the predicting of the structure of the semiconductor including extracting a first evaluation indicator, from each of the plurality of AI algorithms, by applying test data classified as a test group, of the at least one learning data set in the first data set and the second data set, to each of the plurality of AI algorithms
extracting a second evaluation indicator from each of the plurality of AI algorithms, by applying valid data classified into a valid group, of the at least one learning data set in the first data set and the second data set, to each of the plurality of AI algorithms
determining a ratio of the second evaluation indicator to the first evaluation indicator for each of the plurality of AI algorithms
selecting an AI algorithm having a smallest product of the first evaluation indicator and the ratio from among the plurality of AI algorithms as the optimal AI algorithm
In the same field of endeavor, Donghua further teaches these limitations.
Specifically, as to the predicting of the structure of the semiconductor including extracting a first evaluation indicator, from each of the plurality of AI algorithms, by applying test data classified as a test group, of the at least one learning data set in the first data set and the second data set, to each of the plurality of AI algorithms (
Donghua, Pg10, Algorithm 3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
Pg12, P4, Lines1-6, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model... with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively"
wherein Donghua explicitly teaches applying test/prediction data to candidate AI models to calculate prediction error metrics, specifically root mean square error (RMSE), which corresponds directly to extracting the claimed first evaluation indicator from test data.)
As to extracting a second evaluation indicator from each of the plurality of AI algorithms, by applying valid data classified into a valid group, of the at least one learning data set in the first data set and the second data set, to each of the plurality of AI algorithms (
Donghua, Pg10, Algorithm 3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
Pg12, P4, Lines1-6, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model... with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively"
wherein Donghua explicitly teaches applying validation data to candidate AI models to extract validation error metrics (RMSE), which corresponds directly to extracting the claimed second evaluation indicator from valid data.)
As to determining a ratio of the second evaluation indicator to the first evaluation indicator for each of the plurality of AI algorithms (
Donghua, Pg6, Algorithm 2, "Algorithm 2: Excludes the relatively poor model of 20% predicted results and give the same weight to the rest of the model"
Pg12, P4, Lines4-7, "the prediction error of the training set and the prediction error of the prediction set are calculated respectively. Respectively, to the three values of weight, take 0.3, 0.3, 0.4. And finally get the comprehensive error value"
wherein Donghua explicitly teaches determining selection and weighting ratios based on model error metrics—such as excluding the worst 20% or assigning specific weighting ratios—which corresponds directly to determining a ratio of the evaluation indicators.)
As to selecting an AI algorithm having a smallest product of the first evaluation indicator and the ratio from among the plurality of AI algorithms as the optimal AI algorithm (
Donghua, Pg6, Algorithm 2, "Algorithm 2: Excludes the relatively poor model of 20% predicted results and give the same weight to the rest of the model"
Donghua, Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
Pg12, P4, Lines4-7, "the prediction error of the training set and the prediction error of the prediction set are calculated respectively. Respectively, to the three values of weight, take 0.3, 0.3, 0.4. And finally get the comprehensive error value"
wherein Donghua explicitly teaches combining error metrics with weighting ratios to calculate a comprehensive error value for each algorithm configuration and selecting the algorithm configuration yielding the minimum comprehensive error, which corresponds directly to selecting an AI algorithm having a smallest product of the first evaluation indicator and the ratio.)
David, Inkawhich, Donghua, and Li are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing, model evaluation, and optimal algorithm selection for predictive systems. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, Donghua, and Li, specifically by applying Donghua's RMSE evaluation indicator extraction, ratio determination, and minimum-error algorithm selection framework to the predicting of the structure of the semiconductor using the OOD-partitioned learning datasets (comprising test and validation groups) of David, Inkawhich, and Li.
The motivation to combine is as recited by Donghua (
Donghua, Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
Pg10, Algorithm 3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model")
such that a PHOSITA would have been motivated to integrate Donghua's evaluation indicator extraction, ratio calculation, and minimum-error selection methodology into the combined framework of David, Inkawhich, Donghua, and Li in order to evaluate candidate AI algorithms across validation and test sets using weighted error ratios during the semiconductor structure prediction process, thereby identifying and selecting the optimal AI algorithm that minimizes prediction error and maximizes stability and accuracy in semiconductor structure metrology.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over David, Inkawhich, Donghua, and Li as discussed above in Claim 11, in view of Guang-qi et al. (Guang), Chinese Patent CN-102039277A, English Translated Version, and further in view of Zhang et al (Zhang), NPL, “Evaluation of Similarity measure methods for hyperspectral remote sensing data”, published on 2012, Pages: 4.
As to dependent Claim 12,
The combination of David, Inkawhich, Donghua, and Li teaches all the limitations of Claim 11 as set forth above.
However, the combination of David, Inkawhich, Donghua, and Li does not explicitly teach
extracting a first principal component and a second principal component, with respect to the first data set for each of the plurality of semiconductors, by performing principal component analysis (PCA) to reduce a dimension of the first data set
In the same field of endeavor, Guang teaches this (
Guang, Pg1, Abstract, Lines3-5, "WAT parameters are screened from the WAT data of the known wafers; the screened WAT parameters are subject to orthogonal analysis to obtain principal components which are selected"
Pg8, P2, Lines8-11, "Using this dimensionality reduction idea, a principal component analysis was generated, that is, through dimensionality reduction, several synthesis factors (principal components) were found to represent many variables"
wherein Guang explicitly teaches extracting principal components (first and second principal components) with respect to an input dataset by performing principal component analysis (PCA) to reduce the dimension of the dataset.)
David, Inkawhich, Donghua, Li, and Guang are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing systems, feature extraction, and semiconductor metrology. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date, to combine the teachings of David, Inkawhich, Donghua, and Li with Guang, specifically by incorporating Guang's principal component analysis (PCA) dimensionality reduction technique into the spectral data processing pipeline prior to calculating Inkawhich's OOD indices and performing Li's cluster sampling.
The motivation to combine is as recited by Guang (
Guang, Pg8, P2, Lines 4-9, "In most practical problems, there is a certain correlation between variables, so that the information reflected by the observed data overlaps to a certain extent. People naturally want to replace the original variables with fewer variables, and these fewer variables reflect the information of the original variables as much as possible. Using this dimensionality reduction idea, a principal component analysis was generated...")
such that a PHOSITA would be motivated to integrate Guang's PCA dimensionality reduction methodology into the combined pipeline of David, Inkawhich, Donghua, and Li in order to compress high-dimensional semiconductor spectrum data into principal components prior to calculating OOD indices, thereby eliminating redundant feature correlations, reducing noise, and lowering computational complexity without losing critical spectral information.
However, David, Inkawhich, Donghua, Li, and Guang do not explicitly teach
determining a value based on a Euclidean distance between vectors and a cosine distance between the vectors and an origin, for each of the plurality of semiconductors
extracting a normalized value as the OOD index, by normalizing an average value of a product of the Euclidean distance and the cosine distance for each of the plurality of semiconductors
In the same field of endeavor, Zhang teaches these limitations (
Zhang, Pg4138, Section 2.1, Lines1-6, "According to the shape and amplitude features of the hyperspectral data, we choose five widely used similarity measure methods for evaluation... The first type is sensitive to spectral amplitude, such as the Euclidean distance (ED) [6]. The second type is sensitive to spectral shape, such as the spectral angel cosine (SAC) [7]..."
Pg4138, Abstract, Lines6-13, "The results indicate that the spectral angle cosine-Euclidean distance method demonstrates the most superior ability to distinguish various land cover types among five methods because it fully utilizes both the spectral amplitude and shape feature in the hyperspectral data. A combination of the spectral amplitude-sensitive method and the shape-sensitive method will effectively improve the identification accuracy of different land cover types"
Pg4141, Conclusion, P2, Lines 4-7, "However, this method only considers the product of the spectral angle cosine and the Euclidean distance. In certain case, this integration is more sensitive to the spectral profile shape of the hyperspectral data..."
wherein Zhang explicitly teaches determining values based on Euclidean distance and cosine distance between feature vectors and an origin, and normalizing the product of Euclidean distance and cosine distance to form an index/score for discriminating spectral data.)
David, Inkawhich, Donghua, Li, Guang, and Zhang are analogous to the claimed invention as they are from the same field of endeavor of spectral data analysis and feature index calculation. Therefore, it would have been obvious to a PHOSITA, before the effective filing date of the claimed invention, to combine David, Inkawhich, Donghua, Li, and Guang with the spectral angle cosine-Euclidean distance (SAC-ED) calculation of Zhang.
The motivation to combine is as recited by Zhang (
Zhang, Pg4138, Abstract, Lines8-13, "A combination of the spectral amplitude-sensitive method and the shape-sensitive method will effectively improve the identification accuracy of different land cover types"
Pg4141, Conclusion, P2, Lines4-7, "the spectral angle cosine-Euclidean distance method has taken the amplitude and shape characteristics of the spectral profile into consideration, and achieves a higher accuracy to distinguish different land covers.")
such that a PHOSITA would be motivated to integrate Zhang's spectral angle cosine-Euclidean distance (SAC-ED) calculation into the combined framework of David, Inkawhich, Donghua, Li, and Guang (applying it within the dimensionally reduced PCA feature space) in order to comprehensively evaluate both the magnitude (amplitude) and directional (shape) characteristics of the reduced semiconductor spectrum vectors, thereby maximizing the discrimination power and accuracy of the extracted OOD index.
Claims 13, 14 are rejected under 35 U.S.C. 103 as being unpatentable over David, Inkawhich, Donghua, and Li as discussed above in Claim 11, in further view of Zhi-huang et al. (Zhihuang), Chinese patent, CN-111178399A, English Translated Version.
As to dependent Claim 13,
The combination of David, Inkawhich, Donghua, and Li teaches all the limitations of Claim 11 as set forth above.
The combination of David and Inkawhich further teaches
the at least one learning data set includes a first learning data set and a second learning data set and the performing of the data split includes sampling the first data set and the second data set into the training group, the valid group, and the test group of learning data set, according to the OOD indexes of the plurality of semiconductors (
David, Pg26, Col16, Lines60-63, "The training dataset can be partitioned into training, testing, and validation portions to ensure a robust model is built that is not over-fit or over-biased. A typical partition can be 60% training, 30% testing, and 10% validation"
Inkawhich, Pg2070, RC, Lines4-10, "While DNNs have shown to be exceptionally accurate in many settings [5]–[7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8]–[10]."
Inkawhich, Pg2073, LC, P2(Paragraph2), 2), Lines2-5, "In addition to the ID-only empirical training distribution
PNG
media_image1.png
30
42
media_image1.png
Greyscale
, OE introduces another empirical distribution
PNG
media_image2.png
40
60
media_image2.png
Greyscale
which contains samples from the wider OOD set."
Inkawhich, Pg2073, RC, Section C, P2, Lines3-10, Eq7, "For input x and temperature T, the ODIN detector probes the DNN's logit vector
PNG
media_image3.png
29
82
media_image3.png
Greyscale
and produces a score
PNG
media_image4.png
26
92
media_image4.png
Greyscale
which is the maximum value from a temperature-scaled softmax vector, as described by Eq7.
PNG
media_image4.png
26
92
media_image4.png
Greyscale
is then compared to a predefined threshold
PNG
media_image5.png
24
55
media_image5.png
Greyscale
. If
PNG
media_image6.png
27
175
media_image6.png
Greyscale
, x is deemed to be ID. Else, x is regarded as OOD (and the system will abstain from releasing the classification prediction)."
PNG
media_image7.png
73
481
media_image7.png
Greyscale
wherein David explicitly teaches partitioning training datasets (comprising paired first spectral datasets and second structural datasets) into a training group, a validation group, and a test group at a preset ratio, e.g., 60% training, 30% testing, and 10% validation. Thus, combining David’s partitioning practice with Inkawhich’s OOD standards can partition the first data set and the second data set at a preset ratio according to the OOD indexes of the plurality of semiconductors.)
However, the combination of David, Inkawhich, Donghua, and Li does not explicitly teach
sampling the first data set and the second data set into a training group, a valid group, and a test group of the first learning data set in descending order of decreasing OOD indexes according to the plurality of semiconductors
sampling the first data set and the second data set into a training group, a valid group, and a test group of the second learning data set in ascending order of increasing OOD indexes according to the plurality of semiconductors.
In the same field of endeavor, Zhihuang teaches this (
Zhihuang, Pg3, Claim 4, Lines4-11, "Arrange the features in descending order according to the relevance of the features to obtain an ordered feature sequence; Select the first M1 features from the ordered feature sequence, where M1 is a positive integer greater than or equal to 1; Clustering the object based on the first M1 features; If after clustering the objects based on the first M1 features, the ratio of the number of objects with a small proportion to the number of objects with a large proportion is 1:N, then the clustering result of the object is used as the clustering result"
Pg21, P1, Lines1-5, "It should be noted that, here is an example of sorting in descending order according to the relevance of the features, but in other embodiments, it is also possible to sort them in ascending order according to the relevance of the features, but the features are selected for clustering in the following steps When, you can select the last M1 features or the last M2 features"
wherein arranging features/indexes in both descending and ascending orders and selecting feature subsets for clustering corresponds to sampling data in descending order of decreasing indexes and in ascending order of increasing indexes to form first and second learning data sets.)
David, Inkawhich, Donghua, Li, and Zhihuang are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing and partitioning for model training. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, Donghua, and Li with Zhihuang. To the extent Zhihuang performs sorting on feature relevance metrics rather than individual semiconductor samples, it would have been obvious to a PHOSITA to apply Zhihuang's ascending and descending sorting/partitioning methodology directly to individual semiconductor samples according to their calculated OOD indexes, specifically by sampling the dataset into a first learning data set in descending order of decreasing OOD indexes and a second learning data set in ascending order of increasing OOD indexes.
The motivation to combine is as recited by Zhihuang (
Zhihuang, Pg7, P4, Lines11-13, "By adding minority samples, the problem of imbalance of data categories can be solved to ensure that the information of minority samples is not covered")
such that a PHOSITA would have been motivated to integrate Zhihuang's ascending and descending order sorting and feature-based partitioning methodology into the combined framework of David, Inkawhich, Donghua, and Li in order to sample semiconductor spectral and structural datasets into a first learning data set in descending order of decreasing OOD indexes and a second learning data set in ascending order of increasing OOD indexes, thereby systematically structuring the dataset based on varying OOD complexity, preventing extreme OOD data noise from interfering with initial model optimization, achieving balanced and controlled dataset partitioning, and ultimately improving the training stability, generalization, and prediction performance of the trained AI models.
As to dependent Claim 14,
The combination of David, Inkawhich, Donghua, Li and Zhihuang teaches all the limitations of Claim 13 as set forth above.
David, Inkawhich, Li and Zhihuang do not explicitly teach
the predicting of the structure of the semiconductor includes extracting a first evaluation indicator for each of first valid data and first test data from each of the plurality of AI algorithms, by applying the first valid data classified into the valid group of the first learning data in the first data set and the second data set and the first test data classified into the test group of the first learning data set in the first data set and the second data set to each of the plurality of AI algorithms
extracting a second evaluation indicator for each of second valid data and second test data from each of the plurality of AI algorithms, by applying the second valid data classified into the valid group of the second learning data set in the first data set and the second data set and the second test data classified into the test group of the second learning data set in the first data set and the second data set to each of the plurality of AI algorithms
selecting the optimal AI algorithm from among the plurality of AI algorithms, based on a valid root mean square error (RMSE) for the first valid data, a test RMSE for the first test data, a valid RMSE for the second valid data, and a test RMSE for the second test data.
In the same field of endeavor, Donghua further teaches them.
Specifically, as to extracting a first evaluation indicator for each of first valid data and first test data from each of the plurality of AI algorithms by applying the first valid data and the first test data of the first learning data set (
Donghua, Pg10, Algorithm3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
Pg12, P4, Lines1-6, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model... with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively"
wherein Donghua explicitly teaches evaluating candidate AI models on partitioned validation and prediction/test datasets to calculate error metrics, specifically root mean square error (RMSE) values for the validation set and prediction/test set. In the combined framework of David, Inkawhich, Li, Zhihuang, and Donghua, applying the first valid data and first test data from the first learning dataset (sampled in descending OOD order) to each candidate AI algorithm to extract validation RMSE and test RMSE metrics corresponds directly to extracting the claimed first evaluation indicator for each of the first valid data and first test data.)
As to extracting a second evaluation indicator for each of second valid data and second test data from each of the plurality of AI algorithms by applying the second valid data and the second test data of the second learning data set (
Donghua, Pg10, Algorithm3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model"
Pg12, P4, Lines1-6, "Firstly, the training data is put into the general model to train and forecast the data and error data of each model... with the prediction error of the training model, the prediction error of the training set and the prediction error of the prediction set are calculated respectively"
wherein Donghua explicitly teaches applying validation data and test/prediction data to candidate AI models to calculate respective validation and test RMSE metrics. Applying this teaching to the second learning dataset (sampled in ascending OOD order) of David, Inkawhich, Li, and Zhihuang, applying the second valid data and second test data to each candidate AI algorithm to extract validation RMSE and test RMSE metrics corresponds directly to extracting the claimed second evaluation indicator for each of the second valid data and second test data.)
As to selecting the optimal AI algorithm from among the plurality of AI algorithms based on a valid RMSE for the first valid data, a test RMSE for the first test data, a valid RMSE for the second valid data, and a test RMSE for the second test data (
Donghua, Pg11, P1, Lines 1-2, "On the basis of the six weighting algorithms, the optimal weight algorithm selection algorithm selects the best weight algorithm"
Pg12, P4, Lines4-7, "the prediction error of the training set and the prediction error of the prediction set are calculated respectively... And finally get the comprehensive error value"
Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
wherein Donghua explicitly teaches evaluating model prediction errors across validation and test sets to compute overall/comprehensive error metrics (specifically RMSE) and selecting the optimal algorithm that yields the minimum error across the evaluated datasets. In the combined framework, selecting the optimal AI algorithm based on the evaluated valid RMSE and test RMSE of the first learning dataset alongside the valid RMSE and test RMSE of the second learning dataset corresponds directly to selecting the optimal AI algorithm based on the valid RMSE for the first valid data, test RMSE for the first test data, valid RMSE for the second valid data, and test RMSE for the second test data.)
David, Inkawhich, Li, Zhihuang, and Donghua are analogous to the claimed invention as they are from the same field of endeavor of machine learning data processing, model evaluation, and optimal algorithm selection for predictive systems. Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA), before the effective filing date of the claimed invention, to combine the teachings of David, Inkawhich, Li, and Zhihuang with Donghua, specifically by applying Donghua's RMSE evaluation indicator extraction and model selection framework to the predicting of the structure of the semiconductor using the first (descending OOD) and second (ascending OOD) partitioned learning datasets (comprising respective valid and test groups) of David, Inkawhich, Li, and Zhihuang.
The motivation to combine is as recited by Donghua (
Donghua, Pg3, P4-5, "The data set predicted by the weight library is used to compare with the verification set to get the error. By the minimum error, the optimal weight algorithm is obtained"
Pg10, Algorithm3, "(RMSE) is calculated, and then a function of inverse trend is designed according to the size of the error root mean square, and the weight is given to each model")
such that a PHOSITA would have been motivated to integrate Donghua's RMSE-based evaluation and algorithm selection methodology into the combined framework of David, Inkawhich, Li, and Zhihuang in order to evaluate each trained AI algorithm across the valid and test groups of both the first (descending OOD) and second (ascending OOD) learning datasets, thereby obtaining precise quantitative error indicators (valid RMSE and test RMSE) across varying OOD complexity distributions and accurately selecting the overall optimal AI algorithm to ensure maximum prediction accuracy, robustness, and stability in semiconductor structure metrology.
Response to Arguments
Claim Objections
In review of Applicant’s amendments, filed August 19, 2025, the objections to the claims made in the previous office action have been fully considered are persuasive. Therefore, the claim objections have been withdrawn.
35 USC § 101
In review of Applicant’s arguments and amendments, filed August 03, 2026, regarding the abstract idea rejections from the previous office action made under 35 U.S.C. 101 have been fully considered but are not persuasive. Therefore, the rejections under 35 U.S.C. 101 have not been withdrawn.
Regarding Claims 1–6 and 8–20
Applicant argues (Remark, Pg14-15) that independent claims 1 and 16 do not recite a judicial exception because the limitation "training a plurality of AI algorithms..." does not explicitly recite mathematical formulas or equations, citing 2019 PEG Example 39 and the Kim Memo (Aug 4, 2025). Applicant further contends under Enfish and Finjan that the claims are directed to a technical improvement in controlling AI model overfitting as disclosed in paragraphs [0099], [0103], and [0104] of the specification. The Examiner respectfully disagrees.
First, regarding Example 39 and the Kim Memo, while the generic concept of "training an AI algorithm" may not explicitly list mathematical symbols, independent claims 1, 10, and 16 as a whole explicitly recite judicial exceptions including mathematical calculations (determining an OOD index) and mental processes (data splitting via cluster sampling, predicting structure based on index comparison, and selecting an optimal algorithm). Reciting high-level functional training steps without specific algorithmic implementations does not negate the presence of these judicial exceptions nor does it integrate them into a practical application under MPEP 2106.04(d).
Second, regarding the alleged technical improvement under Enfish and Finjan, it is well-established that a technical advantage described solely in the specification cannot render a claim eligible unless the specific technical solution that produces the improvement is recited as a limitation in the claim itself. Independent claims 1, 10, and 16 recite broad, functional results (e.g., determining an OOD index, cluster sampling, providing an optimal AI) at a high level of generality without incorporating the specific mathematical or structural constraints (such as the normalized PCA distance products of Claim 2 or the dual OOD RMSE evaluation logic of Claim 7) that achieve the alleged reduction in overfitting.
Third, the additional elements in independent claims 1, 10, and 16 (loading data sets, processing device, memory, AI algorithms) represent generic computer components and routine data-gathering activities recited at a high level of generality, which amount to a mere instruction to "apply" the abstract concepts using conventional computer tools (MPEP 2106.05(d), (f), (g)).
Accordingly, the rejection of Claims 1–6 and 8–20 under 35 U.S.C. 101 is MAINTAINED.
Regarding Claim 7
In view of Applicant’s arguments and the prior indication in the record, the Examiner agrees that Claim 7 recites specific model selection logic based on valid and test RMSEs extracted across both ascending and descending OOD sampling splits. These specific limitations integrate the judicial exception into a practical application under MPEP 2106.04(d) by providing a specific technological mechanism to control model overfitting. Accordingly, the rejection of Claim 7 under 35 U.S.C. 101 is WITHDRAWN.
35 USC § 103
Applicant’s arguments and amendments, filed July 16, 2026, regarding the rejections from the previous office action made under 35 U.S.C. 103 have been fully considered and see below for the details.
In the Remarks, Applicant argues in substance that
Claims 1 and 16
Applicant’s arguments with respect to Claims 1 and 16 have been considered but are moot in view of the new ground of rejection under 35 U.S.C. 103 over David in view of Inkawhich and Li, necessitated by Applicant’s amendment.
Applicant argues that the combination of Ivanov and David fails to disclose “predicting the structure of the semiconductor using an optimal AI algorithm… when the OOD index… is greater than or equal to the reference value” because Ivanov’s “conditioning parameters” represent input geometric variables for shape parameterization rather than an index indicating whether an input lies within or outside a DOE-trained domain (Remark, Pg20, Paragraph1-2). But the argument has been considered and has been rendered moot as mentioned above.
Specifically, David explicitly teaches the foundational semiconductor machine learning pipeline, including loading a first dataset representing semiconductor spectral measurements and a second dataset representing semiconductor structural features, training a plurality of AI algorithms, and providing/deploying an optimal AI algorithm from among the trained algorithms. Inkawhich directly overcomes the deficiency of David regarding OOD evaluation by explicitly teaching determining an out-of-distribution (OOD) index/score for input spectral datasets to identify data having a distribution different from the training dataset distribution. Li further overcomes the deficiency of David and Inkawhich regarding cluster-based dataset partitioning by explicitly teaching performing a data split on datasets by cluster sampling into at least one learning dataset with respect to sample confidence/OOD metrics for model training.
Accordingly, the combined teachings of David, Inkawhich, and Li teach or suggest every limitation of Claims 1 and 16 as amended, and the rejection of Claims 1 and 16 under 35 U.S.C. 103 is maintained. Please refer to the rejection above for further details.
Claim 10
Applicant’s arguments with respect to Claim 10 has been considered but are moot in view of the new ground of rejection under 35 U.S.C. 103 over David in view of Inkawhich and Donghua, necessitated by Applicant’s amendment.
Applicant argues that the prior combination of Ivanov and Inkawhich fails to disclose "predicting the structure of the semiconductor using an optimal AI algorithm... when the OOD index... is greater than or equal to the reference value" because Inkawhich abstains from releasing predictions for OOD samples and merely adjusts/fine-tunes a single model rather than selecting a different optimal AI algorithm (Remark, Pg24, Paragraph1). But the argument has been considered and has been rendered moot as mentioned above.
Specifically, David explicitly teaches the foundational semiconductor metrology prediction pipeline, including receiving spectrum data of semiconductors, predicting semiconductor structures using trained AI algorithms, and deploying an optimal AI algorithm from among a plurality of trained algorithms. Inkawhich overcomes the deficiency of David regarding OOD evaluation by explicitly teaching generating out-of-distribution (OOD) indexes for input spectral data and comparing the OOD index against a predefined reference value/threshold to classify data and conditionally trigger algorithmic predictions. Donghua directly addresses the deficiency noted by Applicant regarding model switching by explicitly teaching dynamically selecting and deploying a distinct/different optimal AI algorithm from a library of trained candidate algorithms when data conditions change, thereby providing a different optimal AI algorithm when encountering OOD inputs.
Accordingly, the combined teachings of David, Inkawhich, and Donghua teach or suggest every limitation of Claim 10 as amended, and the rejection of Claim 10 under 35 U.S.C. 103 is maintained. Please refer to the rejection above for further details.
Claims 2-9, 11-15, 17-20
Applicant did not present separate patentability arguments regarding these claims, but rather traversed their rejection based solely on their dependency from their respective parent claims. Thus, these Claims are rejected accordingly due to the change of the ground rejection for their parent Claims 1, 10, and 16, respectively. Please refer to the rejection above for further details.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DONG YOON JUNG whose telephone number is (571)270-0198. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DONG YOON JUNG/Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights