Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is responsive to the amendment and remarks filed 06/16/2026. Claims 1-5, 7-10, and 15-19 have been amended; claims 13 and 20 have been canceled; and new claim 21 has been added. Claims 1-12, 14-19, and 21 are pending and have been examined. Claims 1-12, 14-19, and 21 are rejected.
Applicant’s arguments filed on 06/16/2026 with respect to the 35 U.S.C. 101 rejections have been fully considered and are persuasive. Thus, the 35 U.S.C. 101 rejections are withdrawn.
Applicant’s arguments filed on 06/16/2026 with respect to the 35 U.S.C. 102 rejections have been fully considered and are persuasive. Thus, the 35 U.S.C. 102 rejections are withdrawn.
Applicant’s arguments filed on 06/16/2026 with respect to the 35 U.S.C. 103 rejections have been fully considered and are either unpersuasive or moot because of new grounds of rejection. See the response to arguments below.
Response to Arguments
Claim 5
Applicant’s observation (Remarks, p. 10) that dependent claim 5 was not rejected under 35 U.S.C. 102 or 103 in the 03/16/2026 Office action is correct; the omission was inadvertent. A rejection of claim 5 is set forth below for the first time. Because that rejection is not necessitated by Applicant’s amendment, this action is made non-final.
Claims 17-20
Applicant argues (Remarks, pp. 10-11) that the rejections of claims 17-20 were improper because those claims were rejected “under the same rationale” as claims 3, 4, 7, and 13 while the statement of rejection for claims 14-20 identified only the combination of Aas and Pan.In this action, each claim is rejected under an expressly stated combination of references, and the combination applied to each of device claims 16-19 is the same as, and includes every reference applied to, the corresponding method claim (claim 16 with claim 2; claim 17 with claim 3; claim 18 with claim 4; claim 19 with claim 7), as set forth below.
Rejections under 35 U.S.C. 102: “Aas et al. fails to disclose or suggest ‘modifying’ any reference data” (Remarks, pp. 12-16)
In view of the amendments to independent claims 1 and 15, the rejection of claims 1, 2, 6, and 12 under 35 U.S.C. 102(a)(1) as anticipated by Aas is withdrawn, and Applicant’s arguments directed to that rejection are moot to that extent. However, because Aas continues to be relied upon in the new grounds of rejection below for the generation of sample data by modifying reference data, Applicant’s arguments concerning Aas are addressed below.
First, the examiner clarifies the reliance on Aas. Applicant’s arguments are premised on the position that the Office equated the weights wS(xi) of Aas with the claimed “first modifying” (see, e.g., Remarks, p. 14 (“the Examiner’s cited portions of Aas et al. describe an entirely different mechanism of probabilistic weighting for density estimation”); p. 15 (“the Office relied upon weighting operation [of] Aas et al.”)). To be clear, the weighting of Aas is not relied upon as the claimed “first modifying.” The claimed “first modifying” reads on Aas’s construction of the evaluation samples themselves. (Page 6, equation (9), “vKerSHAP(S) = (1/K) Σk f(x̄Sk, x*S), where x̄Sk, k = 1, …, K are samples from the training data.” Page 8, “(3) Sort the weights wS(x*, xi) in increasing order, and let x[k] be the training instance corresponding to the kth largest weight. (4) Approximate the integral in (8) with a weighted version of (9)”; Page 8, “Our approach is, however, more sampling effective as it uses each training observation only once, and uses their weights in the integral computation, rather than as input for the sampling only.”) That is, each piece of sample data evaluated by the model is an actual training (reference) data record x^[k] in which the feature values of the conditioned subset S are changed from the record’s own values (first feature values) to the different values x*S (second feature values) of the same features. The weights of Aas merely determine which training records are used and their weights in the integral. The substitution of feature values performed on the training records is what generates the sample data. Accordingly, Applicant’s assertions that “No feature value inside any reference data record is ever changed” and that the “training/reference data [of] Aas et al. remains untouched” (Remarks, p. 14) are not persuasive as applied to equation (9) and the empirical conditional approach of Aas: the sample vectors evaluated by the model are reference records whose subset-S feature values have been replaced with different values of the same features.
Second, this interpretation is the broadest reasonable interpretation of “modifying” consistent with the instant specification, which describes the claimed modifying as: (instant application, para. 55, “first sample data obtained by modifying a process feature of the reference data to have the same feature value as a corresponding process feature of the input data 101”; see also para. 69.) Nothing in claim 1 requires that the modification be permanent, that it be performed “in place” on a stored data set, or that it alter every record of the reference data; the claim recites “generating sample data by first modifying at least a portion of reference data,” which is met when sample data is generated from reference-data records at least a portion of which (the subset-S feature values) has been changed to different values of the same features.
Third, Applicant’s four asserted distinctions (Remarks, pp. 13-14) (1) what is changed, (2) how it is changed, (3) when the change occurs, and (4) the resulting data – each deal with the weighting characterization addressed above and/or the parametric (Gaussian and Gaussian-copula) sampling variants of Aas, and are unpersuasive as to equation (9) and the empirical conditional approach relied upon: (1) what is changed is a feature value of a reference record (x[k]S is replaced by x*S), not merely a selection probability; (2) the change is a deterministic substitution for the conditioned subset where every generated sample receives exactly the values x*S; (3) the change occurs upstream, when the sample vectors are constructed, before any model evaluation; and (4) the resulting sample data are modified versions of actual reference records (Aas, p. 8, “uses each training observation only once”), not merely new parametric draws.
Fourth, Applicant argues that Aas “could not operate if the Office attempted to force the claimed wherein clause onto the approach of Aas,” and that one skilled in the art “would have not reasonable reason to modify Aas” (Remarks, pp. 15-16). These arguments are not persuasive because the rejection does not propose any alteration of the operation of Aas. As set forth above and in the rejections below, Aas supplies the substitution/construction mechanism; Senoner supplies the physical manufacturing constraint on the substituted value. Moreover, the conditional sampling of Aas is precisely what ensures that the generated sample data “respects … dependencies” as recited in amended claim 1; the stated purpose of Aas (attribution accuracy under feature dependence) is preserved, not destroyed, in the combination.
“Aas et al. attribution operation is different from claimed attribution identification” (Remarks, pp. 17-18)
Applicant argues that Aas does not identify its attribution based on "two different machine learning model results," because Aas averages model outputs into v(S) values and then puts those values into the Shapley formula (Remarks, p. 17). This is not persuasive. As mapped in the new rejection of amended claim 1, the "manufacturing process result" is the model's prediction for the input data, f(x*), and the "sample manufacturing process result" is the same model's prediction for each piece of generated sample data, f(x̄S[k], x*S) (Aas, p. 6, eq. (9); p. 8, step (4)). Both go into Aas's attribution. The v(S) values are simply the weighted average of the predictions on the sample data (p. 8, step (4)), and f(x) is the contribution value for the full set of features, v(M) (p. 4, eq. (2)), which is added to the list of contribution values and fixed by the constraint that the Shapley values sum to v(M) (p. 5). The Shapley values are then solved from that list (p. 5, eq. (7)), so they are identified based on both results. Claim 1 only requires that the attribution be identified "based on" the two results; it does not exclude averaging them or using the Shapley formula, and Applicant's own specification states that the attribution "may be identified by a Shapley value" (para 73) based on the sample process result and the output process result.
Arguments directed to the references individually (Remarks, pp. 16-18)
Applicant argues that Senoner “never generates synthetic sample data by any modifying step,” that Pan “only discloses pure hardware,” that Covert “adds no modification of any data,” and that Lundberg “does not disclose any interpretable modifying operation.” These arguments are not persuasive because they attack the references individually, whereas the rejections are based on combinations of references. In the new 103 rejection below, Senoner is not relied upon for generating sample data by a modifying step (Aas is), and Aas is not relied upon for the physical-manufacturing context, the component-allocation/routing features, or the adjusting step (Senoner is). It is additionally noted that Applicant’s own characterization of Senoner where they stated that Senoner “applies standard (non-conditional) SHAP to real historical production batches and then selects improvement actions such as altering machine prioritization” (Remarks, p. 16) is consistent with the elements for which Senoner is relied upon below and Senoner’s use of SHAP on production parameters that Senoner itself describes as interdependent (Senoner, pp. 5708, 5711, 5713) IS the deficiency that Aas expressly addresses (Aas, p. 6), which supports the motivation to combine set forth in the rejections.
Newly added limitations (Remarks, p. 18)
Applicant argues that none of the previously applied references disclose or suggest the newly added limitations, including the “physical dependency” constraint of the wherein clause, the input data whose “process features define component allocation and/or routing constraints of the physical manufacturing process,” and “adjusting at least one component allocation or routing constraint of the physical manufacturing process based on the identified attribution.” These limitations are addressed in the new grounds of rejection below, primarily by Senoner (see, e.g., Senoner, pp. 5705, 5711, 5713-5715, Tables 3-5, Figure 8), which is now applied as the primary reference. To the extent the arguments are directed to the combinations as previously applied, they are moot in view of the new grounds of rejection, which were necessitated by Applicant’s amendment.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Examiner’s Note: Some rejections will include an Examiner’s Note (labeled ‘EN’) to provide additional context or rationale explaining the basis for the rejection.
Claims 1, 2, 4-6, 12, 15, 16, 18, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Senoner et al., “Using Explainable Artificial Intelligence to Improve Process Quality: Evidence from Semiconductor Manufacturing,” (hereinafter “Senoner”) in view of Aas et al., “Explaining Individual Predictions When Features Are Dependent: More Accurate Approximations to Shapley Values,” (hereinafter “Aas”)
Claim 1
Senoner teaches:
“A processor-implemented method, the method comprising:” (Page 5712, “The metamodel is estimated based on all production parameters and the normalized yield using gradient boosting with decision trees (Ke et al. 2017)”; Page 5712, “On conventional office hardware (Intel Core i7-8550U processor with 1.8 GHz), the computation of feature attributions for the entire training set takes around one second.” – EN: Senoner’s decision model, metamodel training, attribution computation, and action selection is performed by a processor.)
“generating a manufacturing process result using a first machine learning model provided input data,” (Page 5708, “The basis for the decision model is a metamodel f: RN → R that is estimated based on past observations of production parameters and process quality outcomes. This can be an arbitrary predictive model f that can emulate high-dimensional and nonlinear relationships (e.g., tree ensembles, deep neural networks).” Page 5708, “the SHAP value method explains model f locally at each observation i. The explanation is formally given by additive feature attributions that sum to the output of the metamodel; that is, f(x(i)) = φ0 + Σj φj(i)”. – EN: the gradient-boosting metamodel f corresponds to the claimed “first machine learning model”; the model output f(x(i)), the estimated process quality (normalized yield) of a semiconductor production batch corresponds to the claimed “manufacturing process result”; and the observed production parameters x(i) provided to the metamodel correspond to the claimed “input data.”)
“where the input data comprises first feature values corresponding to a plurality of process features of a physical manufacturing process,” (Page 5706, “let x(i) ∈ RN denote the observed production parameters of the ith observation with production parameters j = 1, …, N.” Pages 5710-5711, “The transistor chip production at Hitachi ABB consists of 200 processes that are carried out in a low-vibration and temperature-constant clean room.” Page 5711, “The fabrication conditions of each production batch are described by N = 3,614 production parameters from K = 200 different processes.” – EN: the production parameters are process features of a physical semiconductor manufacturing process, and the observed parameter values of a given batch are the claimed “first feature values.”)
"and the plurality of process features define component allocation and/or routing constraints of the physical manufacturing process;" (Page 5706, "the process specification P_k … define[s] which specific production parameters belong to a certain process k." Page 5711, "the average pressure measured in a machine". Page 5713, "the measured production parameters can depend on the production equipment used, and therefore, possible improvement actions involve a change in the material routing." Page 5714, Table 3 (parameters x278–x340 belong to process 25, machines QI613–QI615; x2197–x2369 to process 166, machines QP211–QP233); "strong evidence of machine-related performance differences." Page 5705, "Production parameters can … refer to … material routings". – EN: the claim doesn’t require the features to be machine identifiers. The instant specification lists "a recipe feature" among its process features (para. 51). Under BRI, Senoner's parameters where each of them are assigned to one stage of the 200-process sequence (P_k) and dependent on the machine that carries out that stage (Table 3) define component allocation (which machine performs each stage) and routing constraints (the path through the stages).
“where the reference data comprises reference feature values corresponding to the plurality of process features, the reference feature values having been used to generate a reference manufacturing process result;” (Page 5708, “The initial input for the decision model is given by historical manufacturing data {(x(i), y(i))}, i = 1, …, M.” Page 5711, “Hitachi ABB provided us with historical data on M = 1,197 production batches (approximately 1.8 million transistor chips) produced prior to April 2019.”; “each production batch i is associated with a quality variable (from the electrical testing) given by the yield μ(i)”. – EN: the historical production batches are reference data; their production-parameter values are reference feature values that were actually used, in the physical fabrication of those batches, to generate each batch’s measured yield, a “reference manufacturing process result.” In the combination below, this historical data is the training data of Aas.)
“identifying an attribution of the plurality of process features of the physical manufacturing process …” (Page 5708, “we use SHAP values (for details, see Appendix A) to provide explanations of how the estimated process quality changes when the effect of a production parameter is omitted. For this, the SHAP value method explains model f locally at each observation i.”; “φj(i) ∈ R corresponds to the SHAP value of the observed production parameter xj(i)”. – EN: Senoner identifies per-feature attributions (SHAP values) of the production parameters of the physical manufacturing process. The particular mechanism recited in the claim which states generating the attribution from the manufacturing process result and sample manufacturing process results produced from generated sample data is taught by Aas, as set forth below.)
“adjusting at least one component allocation or routing constraint of the physical manufacturing process based on the identified attribution,” (Page 5713, “possible improvement actions involve a change in the material routing. This is achieved by altering the corresponding prioritization of machines so that one machine is preferred over others from the same process.” Page 5714, “For each action, the decision model returns the mean feature attribution, which quantifies the estimated average influence on the normalized yield (Table 4) … The decision model estimates that action a3-25 and action a2-166 are associated with a positive influence on the normalized yield and are thus selected as improvement actions.”; “As improvement actions, the decision model suggests that selecting machine QI615 (a3-25) in process 25 and machine QP212 (a2-166) in process 166 improves the normalized yield. Figure 8 shows the suggested material routing through the two prioritized processes.” Page 5715, “Hitachi ABB produced a completely new production batch of transistor chips (i.e., not included in the original data set), where the routing of materials through the fabrication processes was altered.” – EN: Senoner’s equations (5)-(6) select, based on the identified feature attributions, the machine-prioritization action having the largest mean attribution, and the machine allocation / material routing of the physical fabrication process is adjusted accordingly (as implemented in the field experiment and post-experimental rollout). This is “adjusting at least one component allocation or routing constraint of the physical manufacturing process based on the identified attribution.”)
As to the wherein clause, Senoner teaches: "… [feature values of the process features] … constrained by the physical dependency to represent an existing physical component or an existing operation stage path of the physical manufacturing process" (Page 5705, "Production parameters can … refer to … material routings"; Page 5713, "the measured production parameters can depend on the production equipment used, and therefore, possible improvement actions involve a change in the material routing"; Page 5710, "the possible candidate actions are simply the set of machines that can be used to carry out that process"; Page 5714, Table 3 (process 25: machines QI613–QI615; process 166: machines QP211–QP233); Figure 8, "Suggested Material Routing Through Prioritized Processes"; Table 5, "different process routings". – EN: Senoner teaches production parameters that refer to material routings, i.e., which machine carries out a process. The physical dependency (a process's measured parameters depend on the machine used, p. 5713) ties such a parameter's value to the machine that actually carried out the process, and that machine is one of the machines that exist for the process (Table 3; p. 5710). The value therefore represents an existing physical component (the machine) and an existing operation stage path (Senoner's "material routing," Fig. 8). Either alternative of the "or" suffices. )
Senoner further teaches that the production parameters of the physical manufacturing process are dependent on one another through the physical production system (Page 5708, “Because processes are potentially interdependent, the decision model must account for nonlinear interdependencies. For example, such nonlinearities can appear if a production parameter xj′ from a process k′ interacts with a production parameter xj″ from another process k″.” Page 5711, “Additionally, there may be critical combinations of production parameters that can trigger undesired interaction effects. For example, a machine from a given process may only induce quality issues if it is used in combination with another machine from a different process.” Page 5713, “the measured production parameters can depend on the production equipment used”. – EN: dependencies among production parameters that arise from the physical production system like the equipment used, machine combinations, and material routing are “physical dependenc[ies] between two or more of the plurality of process features.”)
Senoner does not explicitly disclose:
“generating sample data by first modifying at least a portion of reference data based on physical dependency between two or more of the plurality of process features”;
“[attribution] … based on the manufacturing process result and a sample manufacturing process result generated using the first machine learning model, or a second machine learning model related to the first machine learning model, provided the generated sample data that respects physical dependencies between process features of the physical manufacturing process”;
“wherein the first modifying of the at least a portion of the reference data comprises: with a first feature value and a second feature value being different feature values of a first process feature of the reference data, modifying the first feature value of the first process feature to be the second feature value of the first process feature, where the second feature value is constrained by the … dependency …" (EN: the physical nature of the dependency, and the existing physical component or operation stage path that the value represents, being taught by Senoner as set forth above.)
(EN: the “physical” and “manufacturing” aspects of these limitations being taught by Senoner as set forth above).
However, Aas teaches:
“generating sample data by first modifying at least a portion of reference data based on … dependency between two or more of the plurality of process features”: (Page 4, “Consider a classical machine learning scenario where a training set {yi, xi}, i = 1, …, ntrain of size ntrain has been used to train a predictive model f(x) attempting to resemble a response value y as closely as possible. Assume now that we want to explain the prediction from the model f(x*), for a specific feature vector x = x*.” Page 6, equation (9), “vKerSHAP(S) = (1/K) Σk f(x̄Sk, x*S), where x̄Sk, k = 1, …, K are samples from the training data.” Page 8, “(3) Sort the weights wS(x*, xi) in increasing order, and let x[k] be the training instance corresponding to the kth largest weight. (4) Approximate the integral in (8) with a weighted version of (9)”; Page 8, “Our approach is, however, more sampling effective as it uses each training observation only once”; see also Page 8, “By viewing the estimation of E[f(x)|xS = x*S] as a regression problem with response f(x̄Si, x*S) and covariates xSi, i = 1, …, ntrain, it turns out that our empirical conditional distribution approach (with K = ntrain) is equivalent to the Nadaraya-Watson estimator” . – EN: the training data used to train the model is reference data. Each evaluation sample (x̄S[k], x*S) is an actual training (reference) record x[k] in which the feature values of the conditioned subset S are changed from the record’s own values (first feature values) to the different values x*S (second feature values) of the same features i.e., sample data generated by modifying at least a portion of the reference data. This reading is consistent with the instant specification’s own description of the claimed modifying: “first sample data obtained by modifying a process feature of the reference data to have the same feature value as a corresponding process feature of the input data” (instant publication, para. 55).)
As to the first modifying being performed “based on … dependency between two or more of the plurality of process features”: (Page 6, “If the features in a given model are highly dependent, the Kernel SHAP method may give a completely wrong answer … It would therefore be desirable to incorporate dependence into the Kernel SHAP method by relaxing the independence assumption. This can be done by estimating/approximating p(x̄S | xS = x*S) directly and generate samples from this distribution, instead of generating them independently from xS as in Section 2.3.2.” Page 7, “we have developed an empirical conditional approach … to sample approximately from p(x̄S | xS). The method … is motivated by the idea that samples (x̄S, xS) with xS close to x*S are informative about the conditional distribution p(x̄S | x*S)”. – EN: the modifying and sample generation are performed based on the dependency between the features, the values of the remaining (dependent) features of each sample are determined conditionally on the substituted feature values rather than independently. In the combination, those dependencies are the physical dependencies of Senoner’s fabrication process (equipment, machine combinations, routing) taught by Senoner as set forth above.)
Aas further teaches “[attribution] … based on the … process result and a sample … process result generated using the first machine learning model, or a second machine learning model related to the first machine learning model, provided the generated sample data that respects … dependencies between process features”
As to “identifying an attribution … based on the … process result and a sample … process result” (Page 5, “imposing the constraints that φ0 = v(∅) and Σj=0…M φj = v(M) into the problem”; Page 5, “S = ∅ and S = M are excluded from the sampling procedure … their corresponding Z-rows are appended to ZD, their v(S)-elements are appended to vD”; Page 5, equation (7), φ = RD vD; Page 4, equation (2) (for S = M, v(M) = E[f(x)|x = x*] = f(x*)); Page 8, step (4). – EN: the attribution vector φ is computed from the vector v of contribution values (eq. (7)), which contains both v(M) = f(x*), the model’s result for the input data, and the v(S) values estimated from the model’s outputs on the generated sample data (step (4)). The attribution is therefore identified based on the process result and the sample process results, both produced by the same model f. In the combination, f(x*) is Senoner’s estimated yield for the batch being explained.)
As to “a sample … process result generated using the first machine learning model … provided the generated sample data”: (Page 8, step (4) (the same model f is evaluated on each piece of the generated sample data, producing the outputs f(x̄S[k], x*S), whose weighted average forms vcondKerSHAP(S)); Page 4, equation (2), “v(S) = E[f(x)|xS = x*S]”; Page 4, “f(x*) = φ0 + Σj φj* … the Shapley values explain the difference between the prediction y* = f(x*) and the global average prediction”. – EN: the outputs of the model f on the generated sample data are the claimed sample process results. In the combination, estimated yields under the sampled fabrication conditions. The model f evaluated on the sample data is the same model that generated the process result, i.e., the “first machine learning model,” meeting the first of the claim’s alternatives (“using the first machine learning model, or a second machine learning model”). The Shapley-value attributions are thereby identified based on the process result for the input data together with these sample process results, as recited in the identifying step.)
As to the generated sample data “that respects … dependencies between process features”: (Page 6, “estimating/approximating p(x̄S | xS = x*S) directly and generate samples from this distribution, instead of generating them independently from xS”; Page 7, “we have developed an empirical conditional approach … to sample approximately from p(x̄S | xS)”. – EN: because the values of the dependent features in each sample are drawn conditionally on the substituted values rather than independently, the generated sample data provided to the model respects the dependencies between the process features. In the combination, those are the physical dependencies of Senoner’s manufacturing process taught by Senoner as set forth above, such that the sample data respects physical dependencies between process features of the physical manufacturing process.)
“wherein the first modifying of the at least a portion of the reference data comprises: with a first feature value and a second feature value being different feature values of a first process feature of the reference data, modifying the first feature value of the first process feature to be the second feature value of the first process feature,” (Page 6, equation (9); Page 8, steps (3)-(4), as quoted above. – EN: for each training (reference) record x[k] and each conditioned feature j in S, the record’s own value x[k]j (first feature value) is replaced by the different value x*j (second feature value) of the same feature j to form the sample vector.)
“where the second feature value is constrained by the … dependency …” (Page 8, steps (3)–(4) (each evaluation vector (x̄S^[k], xS) is training record x^[k] with its subset-S values replaced by xS); Page 7, "samples (x̄S, xS) with xS close to xS are informative about the conditional distribution p(x̄S | xS)"; Page 8, "uses each training observation only once." – EN: the second value xS is an observed value of the same feature, not an arbitrary one, and it is substituted only into reference records selected for consistency with it under the dependency among the features (the weights of steps (2)–(3)). The substitution is therefore constrained by the dependency.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the machine-learning-based process-quality decision model of Senoner which trains a metamodel on historical semiconductor production data, computes a Shapley (SHAP) feature attribution for each production parameter, and uses those attributions to prioritize processes and select machine-prioritization and material-routing improvement actions with the conditional, dependence-aware sample generation of Aas, which generates the evaluation samples used to estimate each Shapley attribution from the training data conditioned on the dependence among the features. The motivation for doing so would be to obtain more accurate feature attributions when the features are dependent on one another. Aas discloses this benefit: "Like several other existing methods, this approach assumes that the features are independent, which may give very wrong explanations… In this paper, we extend the Kernel SHAP method to handle dependent features… where our method gives more accurate approximations to the true Shapley values." (Aas, Abstract; see also p. 6: "If the features in a given model are highly dependent, the Kernel SHAP method may give a completely wrong answer.") As for Senoner, Senoner states that "the measured production parameters can depend on the production equipment used" (p. 5713), that "there may be critical combinations of production parameters that can trigger undesired interaction effects" (p. 5711), and that "the decision model must account for nonlinear interdependencies" (p. 5708).
Claim 2
Senoner in view of Aas teaches the method of claim 1 as set forth above. Aas further teaches:
“wherein the first modifying further comprises: in response to a determination that a second process feature is dependent on the first process feature”: (Page 6, “If the features in a given model are highly dependent, the Kernel SHAP method may give a completely wrong answer … This can be done by estimating/approximating p(x̄S | xS = x*S) directly and generate samples from this distribution, instead of generating them independently from xS as in Section 2.3.2.”; Page 8, “Typically, when the features are highly dependent, a small σ is needed such that the bias does not dominate. When the features are essentially independent, there is no bias and a larger σ is preferable.” – EN: the conditional selection of the dependent features’ values is invoked in response to the features being dependent, and Aas expressly varies the selection behavior in accordance with whether the features are dependent. In the combination, the determination that a second process feature (e.g., a measured production parameter) is dependent on the first process feature (e.g., a feature of the production equipment used) is supplied by Senoner (p. 5713, “the measured production parameters can depend on the production equipment used”).)
“selecting a third feature value of the second process feature from a candidate feature value that is dependent on the second feature value of the first process feature”: (Page 7, equation (13), “DS(x*, xi) = √((x*S − xiS)ᵀ ΣS−1(x*S − xiS)/|S|)”; Page 8, “(2) Compute weights wS(x*, xi) for all training instances … (3) Sort the weights wS(x*, xi) in increasing order, and let x[k] be the training instance corresponding to the kth largest weight”; Page 8, equation (14) (selecting the number K of retained samples); Page 8, “A small σ puts most of the weight to a few of the closest training observations”. – EN: the candidate feature values of the dependent (second) process feature are the values that feature takes in the training (reference) records, and which candidate value enters the sample data is selected as a function of the substituted (second) feature value of the first process feature, because the distances and weights governing the selection are computed from x*S. The dependent-feature value of each retained training instance is the claimed “third feature value,” selected from candidates in dependence on the second feature value.)
“wherein the sample data is based on the second feature value of the first process feature and the selected third feature value”: (Page 8, step (4) (each evaluated vector f(x̄S[k], x*S) contains both the substituted second feature value x*S and the selected dependent-feature value x̄S[k]). – EN: the sample data is based on both the second feature value and the selected third feature value.)
See the motivation to combine as set forth in claim 1.
Claim 4
Senoner in view of Aas teaches the method of claim 1 as set forth above. Senoner further teaches:
"wherein the first process feature is related to a first operation stage among pieces of the reference data;" (Page 5706, "We consider a manufacturing system with sequential processes (see Figure 1). Each process is specified by production parameters … let the process specification Pk ⊆ {1, …, N} define which specific production parameters belong to a certain process k." Page 5711, "N = 3,614 production parameters from K = 200 different processes." Page 5714, Table 3. – EN: each of the 200 sequential processes is an operation stage, and every production parameter belongs to exactly one of them. The first process feature of claim 1, the routing parameter of process 25 (machines QI613–QI615), belongs to process 25 and is therefore related to a first operation stage among the historical batches.)
"a path, which includes the first operation stage, also includes a second operation stage" and "a second process feature that is related to the second operation stage" (Page 5706, Figure 1 (Process 1 → … → Process K); Page 5711, "Hitachi ABB also provided us with process specifications Pk that define which production parameters belong to a certain process"; Page 5708, "such nonlinearities can appear if a production parameter xj′ from a process k′ interacts with a production parameter xj″ from another process k″"; Page 5714, Table 3; Figure 8; Table 5. – EN: the fabrication path that includes process 25 also includes process 166: every batch passes through all K sequential processes (Fig. 1), and Fig. 8 and Table 5 show the routing through both. The process specifications Pk, an input to the decision model (Fig. 3), establish which stages the path includes and which parameters belong to each before any attribution is computed; that is a determination whose result is that the path also includes the second operation stage. The routing parameter of process 166 (machines QP211–QP233) and parameters x2197–x2369 are second process features related to that stage.)
Senoner does not explicitly disclose:
"wherein the first modifying further comprises, based on a result of a determination of whether a path, which includes the first operation stage, also includes a second operation stage being that the path also includes the second operation stage, generating the sample data corresponding to the path by modifying a third feature value of a second process feature that is related to the second operation stage" (EN: the stages, the path, the determination that the path includes both stages, and the features related to each stage being taught by Senoner as set forth above; what Senoner lacks is generating the sample data by modifying the second-stage feature's value.)
However, Aas teaches:
"generating the sample data … by modifying a third feature value of a second process feature" (Page 3, equation (1) (the Shapley value sums over all subsets S of the remaining features); Page 5, "Z … binary matrix representing all possible combinations of inclusion/exclusion of the M features, where … entry j+1 of row l is 1 if feature j is included in combination l, and 0 otherwise"; Page 5, "sampling (with replacement) a subset D of M from a probability distribution following the Shapley weighting kernel"; Page 6, equation (9); Page 8, step (4). – EN: for each coalition S, every feature in S has its value in the reference record replaced by x*'s value. For coalitions that include the second process feature together with the first, the sample vector is generated by also modifying the record's own value of the second process feature (the third feature value) to x*'s value. In the combination, a coalition containing the routing parameters of both process 25 and process 166 yields sample data for the path through both stages with the explained batch's machines at each (Fig. 8), generated by modifying the record's process-166 routing value as well as its process-25 routing value. A feature is in S for some coalitions and in S̄ for others (eq. (1)), so the selection of claim 2 and the modification of claim 4 both occur in the method adopted for claim 1.)
The added limitations of claim 4 are met by Senoner's own stage and path structure and by the coalition-based substitution of the Kernel SHAP method adopted for claim 1, which evaluates coalitions including features of more than one process; no further modification of Senoner is proposed, and the rationale stated for claim 1 applies. Senoner itself explains why coalitions spanning processes are needed: "a production parameter xj′ from a process k′ interacts with a production parameter xj″ from another process k″" (p. 5708), and SHAP "accounts for interaction effects because feature attributions are computed over all possible subsets of production parameters" (p. 5709).
Claim 5
Senoner in view of Aas teaches the method of claim 1. Aas further teaches “wherein the generating of the sample data by the first modifying of the at least a portion of the reference data comprises generating the sample data using a sample generation machine learning model” (Pages 7-8, Section 3.3 (the empirical conditional approach relied upon for claim 1): the sample data is generated using a kernel model, “(2) Compute weights for all training instances … similarly to a Gaussian distribution kernel: wS(x*, xi) = exp(−DS(x*, xi)²/(2σ²)), where σ may be viewed as a smoothing parameter or bandwidth that needs to be specified,” whose bandwidth parameter is learned from the data by minimizing the AICc criterion, Pages 8-9, equation (15) and “to select σ we compute the AICc in (15) for various σ values and select the σ corresponding to the smallest AICc value”; Page 8, “our empirical conditional distribution approach (with K = ntrain) is equivalent to the Nadaraya-Watson estimator”. – EN: the kernel/Nadaraya-Watson model is a model whose parameter is trained on the training (reference) data and which is applied to the reference data to generate the sample data satisfying the dependency between the process features. Under the broadest reasonable interpretation, and consistent with the instant specification which describes the sample generation model as “a model trained to output sample data as the sample generation model is applied to the reference data” (Para 71) such a model learned from data constitutes a “sample generation machine learning model.”)
See the motivation to combine as set forth in claim 1.
Claim 6
Senoner in view of Aas teaches the method of claim 1. Aas further teaches:
“wherein the generating of the sample data comprises generating a respective sample data for each of the plurality of process features” (Page 3, equation (1), “φj(v) = φj = ΣS⊆M\{j} (|S|!(M − |S| − 1)!/M!)(v(S ∪ {j}) − v(S)), j = 1, …, M”. – EN: computing the Shapley value φj for each individual feature j = 1, …, M requires generating samples to estimate the contribution function v(S) for the subsets associated with that feature; sample data is thereby generated on a per-feature basis.)
See the motivation to combine as set forth in claim 1.
Claim 12
Senoner in view of Aas teaches the method of claim 1. Senoner further teaches
“further comprising sorting at least one of the plurality of process features based on the identified attribution” (Page 5713, “The top 10 quality drivers in the transistor chip production are listed in Table 2.”; Table 2, “Top 10 Quality Drivers Ranked by Mean Absolute Feature Attribution”. – EN: the process features are sorted (ranked) by their identified attributions. See also Aas, page 18, “One way to then visually present the explanation of a particular prediction could be to rank the absolute Shapley values and present them and their corresponding features in descending order, for example for the ten most important features.”)
Claim 15
Claim 15 recites “An electronic device, the electronic device comprising: a processor configured to” perform operations substantially the same as the method of claim 1 (generate a manufacturing process result; generate sample data by performance of a first modification of at least a portion of reference data; identify an attribution; adjust at least one component allocation or routing constraint; and the corresponding wherein clause). Senoner teaches “An electronic device, the electronic device comprising: a processor configured to” (Page 5712, “On conventional office hardware (Intel Core i7-8550U processor with 1.8 GHz), the computation of feature attributions for the entire training set takes around one second.”; see also Aas, page 2, “Our methodology has been implemented in an R-package currently available at: https://github.com/NorskRegnesentral/shapr”. – EN: the metamodel, attribution computation, and decision model are executed by a processor of a computing device configured by the corresponding software.) The remaining limitations of claim 15 are substantially the same as the limitations of claim 1; therefore, claim 15 is rejected under the same rationale and evidence as claim 1, as set forth above.
Claim 16
Claim 16 recites, in device form, limitations substantially the same as those of method claim 2. Senoner in view of Aas teaches these limitations for the reasons set forth for claim 2 above (see Aas, pages 7-8, eq. (13)/steps (2)-(4); Senoner, page 5713). Therefore, claim 16 is rejected under the same rationale as claim 2.
Claim 18
Claim 18 recites, in device form, limitations substantially the same as those of method claim 4. Senoner in view of Aas teaches these limitations for the reasons set forth for claim 4 above (see Senoner, pages 5706, 5713-5715, Table 3, Figure 8; Aas, pages 6-8). Therefore, claim 18 is rejected under the same rationale as claim 4.
Claim 21
Senoner in view of Aas teaches the method of claim 1 as set forth above. Senoner further teaches:
“wherein the second feature value comprises: a default value or a dummy value indicating that a component related to an operation stage corresponding to the first process feature does not exist; or a feature value set to identify an attribution of the first process feature to be zero.” (Page 5708, “In our setting, missingness assures that absent production parameters have no feature attribution.”; Page 5720, “Missingness states that absent features have no attribution; that is, if fx(S ∪ {j}) = fx(S) for all subsets S in the power set of F, then φj(f, x) = 0.” – EN: an absent production parameter is one whose value is absent from the observation, and Senoner teaches that this value results in zero attribution for that parameter. In the combination set forth for claim 1, the second feature value is the value xj* of the observation being explained that replaces the training record’s own value of parameter j. Where parameter j is absent from that observation, the substituted value is the absent value, which identifies the attribution of parameter j to be zero. This is consistent with the instant specification, which describes the claimed value as “a feature value (e.g., a feature value based on the process feature among the pieces of reference data) that is set to identify an attribution of the process feature to be 0” (instant publication, para. 87).)
See also Aas: (Page 4, equation (3), “φj = βj (xj* − E[xj])”; Page 24, “Dummy player:: A feature that does not change the prediction, no matter which other features it is combined with, has a Shapley value of 0.” – EN: the attribution of a feature is zero when the feature takes its expected value E[xj] over the training (reference) data, i.e., a feature value based on the reference data identifies the attribution of that feature to be zero.)
Claim 21 recites alternatives joined by “or.” The second alternative is relied upon.
The motivation to combine is the same as set forth for claim 1.
Claims 3, 14, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Senoner in view of Aas, further in view of Pan et al., US Patent 11,693,326 B1 (hereinafter “Pan”)
Claim 3 Senoner in view of Aas teaches the method of claim 1 as set forth above. Senoner further teaches:
"wherein the first process feature corresponds to first equipment among pieces of the reference data, and the second feature value of the first process feature indicates another equipment," (Page 5710, "the possible candidate actions are simply the set of machines that can be used to carry out that process"; Page 5713, "implantation process 25 by machines QI613, QI614, or QI615 and etching process 166 by machines QP211, QP212, QP232, or QP233"; Page 5714, Table 3; Page 5716, Table 5 (machine in process 25: QI613 for the control group, QI615 for the treatment groups). – EN: in the combination set forth for claim 1, the first process feature is the machine used at process 25, whose value in a historical batch (a piece of the reference data) is a first equipment, e.g., QI613, and whose substituted second value is the machine of the batch being explained, another equipment, e.g., QI615 (Aas, p. 8, step (4)).)
"...which is related to at least one of a (Page 5713, "The process engineers at Hitachi ABB suggested that the measured production parameters can depend on the production equipment used" Additional context on Page 5711, "Additionally, there may be critical combinations of production parameters that can trigger undesired interaction effects. For example, a machine from a given process may only induce quality issues if it is used in combination with another ma-chine from a different process.")
"to be another feature value indicating at least one of a corresponding (Page 5713, "possible improvement actions involve a change in the material routing. This is achieved by altering the corresponding prioritization of machines so that one machine is preferred over others from the same process." Page 5717, "In implantation process 25, the worst-performing machines, Q1613 and QI614, are prone to particles that induce chip failures during processing. In contrast, machine Q1615 does not engender the same amount of contamination." - EN: this denotes that when the machine (equipment) selection is modified (like when the first feature value related to equipment is changed from one machine to another), the dependent production parameters inherently change to values corresponding to the newly selected machine.)
Senoner does not explicitly disclose:"wherein the first modifying further comprises modifying a third feature value of a second process feature … to be another feature value …, and wherein the sample data is based on the second feature value of the first process feature and the modified third feature value"
However Aas further teaches:
"wherein the first modifying further comprises modifying a third feature value of a second process feature … to be another feature value …, and wherein the sample data is based on the second feature value of the first process feature and the modified third feature value" (Page 3, equation (1); Page 5, Z matrix and coalition sampling; Page 6, equation (9); Page 8, step (4), as set forth for claim 4. – EN: for a coalition S containing both the first and the second process feature, the reference record's own value of the second process feature (the third feature value) is modified to x*'s value of that feature, and the evaluation vector (x̄S[k], x*S) contains both the substituted second feature value and the modified third feature value.)
The motivation to combine is the same as set forth for claim 1.
Senoner in view of Aas does not explicitly disclose:"chamber" and "reticle"
However Pan teaches: "chamber" and "reticle" (Col 8, line 46-48, "The EUV system 400 depicted in FIG. 4 further includes a process chamber 442 wherein the reticle assembly 100, 200, 300 is positioned for processing of the wafer 460."
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the machine-learning-based process-quality decision model of Senoner and the conditional, dependence-aware sample generation of Aas with the per-lot tool, reticle, and process-chamber information of Pan, so that the reticle and process chamber used with each lithography machine are included among Senoner's production parameters. The motivation for doing so would be to account for the reticle and chamber as sources of pattern defects and overlay variation, so that these quality drivers are not missed and wafer uniformity is improved. Pan discloses this benefit: "Each exposure of the reticle during EUV operations causes fluctuations in reticle temperature, which can cause defects in the transferred pattern" (col. 1:19-22), and this variation "may result in significant changes to overlay performance, thus negatively impacting wafer uniformity and production" (col. 9:6-11). As for Senoner, Senoner's fabrication includes photolithography processes (pp. 5710–5711), and Senoner cautions that "When important production parameters are omitted, there is a risk that quality drivers may go unnoticed" (p. 5719)
Claim 14
Senoner in view of Aas teaches the method of claim 1 as set forth above, performed by software executing on computing hardware (Senoner, p. 5712; Aas, p. 2).
Senoner in view of Aas does not explicitly disclose
“a non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of
However, Pan teaches “a non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform…” (Col. 19, lines 13-24, “The methods illustrated throughout the specification, may be implemented in a computer program product that may be executed on a computer. The computer program product may comprise a non-transitory computer-readable recording medium on which a control program is recorded, such as a disk, hard drive, or the like. Common forms of non-transitory computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, or any other magnetic storage medium, CD-ROM, DVD, or any other optical medium, a RAM, a PROM, an EPROM, a FLASH-EPROM, or other memory chip or cartridge, or any other tangible medium from which a computer can read and use.”
“the method of claim 1.” (EN: See the rejection for claim 1)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to implement the method of Senoner in view of Aas as instructions stored on a non-transitory computer-readable medium executed by a processor, as taught by Pan, to provide a tangible implementation executable on general-purpose computing hardware. see Pan, Col. 18, lines 45-50, “The exemplary embodiment also relates to an apparatus for performing the operations discussed herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer.”
Claim 17
Senoner in view of Aas teaches the device of claim 15 as set forth above. Claim 17 recites, in device form, limitations substantially the same as those of method claim 3, with the first modification performed by the processor (Senoner, p. 5712, as set forth for claim 15). Senoner in view of Aas, further in view of Pan, teaches these limitations for the reasons set forth for claim 3 above, and the motivation to combine set forth for claim 3 applies. Therefore, claim 17 is rejected under the same rationale as claim 3.
Claims 7-9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Senoner in view of Aas, further in view of Covert et al., “Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regression” (hereinafter “Covert”)
Claim 7
Senoner in view of Aas teaches the method of claim 1 as set forth above, including identifying the attribution based on sample manufacturing process results generated by evaluating the model on the generated sample data.
Covert teaches: “calculating confidence of respective sample data corresponding to each of the plurality of process features based on the generated sample (Page 6, Section 4.3, “we suggest estimating the variance by selecting an intermediate value m such that m << n and calculating multiple independent estimates”; Page 7, Section 5, “Figure 3: Shapley value-based explanations with 95% uncertainty estimates.”; Page 8, Section 6, “Both explanations used a convergence threshold of t = 0.01 and display 95% confidence intervals, which are features not previously offered by KernelSHAP.” – EN: this denotes computing per-feature variance/standard deviation (confidence) based on model evaluations on sampled coalitions (sample process results).
identifying the attribution based on the calculated confidence.” (Page 6, Section 4.3, “For detection, we propose stopping at the current value n when the largest standard deviation is a sufficiently small portion … (e.g., t = 0.01) of the gap between the largest and smallest Shapley value estimates.” – EN: section 4.3 denotes outputting the final Shapley value attributions once the calculated confidence meets a specified convergence threshold.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the attribution computation related to manufacturing of Senoner in view of Aas with the confidence/uncertainty estimation and convergence detection of Covert. The motivation for doing so would be to give better insights on the feature attributions accuracies. See page 1, section 1 of Covert, "KernelSHAP does not provide un-certainty estimates. Furthermore, it provides no guidance on the number of samples required because its convergence properties are not well understood." Covert further demonstrates that the improved estimators "display 95% confidence intervals, which are features not previously offered by KernelSHAP."
Claim 8
Senoner in view of Aas teaches the method of claim 1. Covert further teaches:
wherein the identifying of the attribution of the plurality of process features comprises: when the sample (Page 4, Section 4.1, “we arrive at an alternative to the original KernelSHAP estimator, which we refer to as unbiased KernelSHAP” Page 6, “calculating multiple independent estimates 3, while accumulating samples for 3,,.” – EN: this denotes using alternative estimator models (like original vs unbiased KernelSHAP, or multiple independent batch estimators) to generate independent sample results for the same predictive model.)
when the sample ; (Page 5, Section 4.2, “Substituting this into unbiased KernelSHAP (Eq. yields a new estimator 3, that preserves the properties of being both consistent and unbiased” – EN: this denotes utilizing a third distinct estimator variant (like the paired sampling estimator) to generate additional distinct sample results.)
calculating confidence of respective sample data corresponding to each of the plurality of process features based on the generated sample sample (Page 6, section 4.3,
PNG
media_image1.png
173
520
media_image1.png
Greyscale
EN: this denotes empirically calculating the variance (confidence) by comparing the outputs of these multiple independent estimators.) identifying the attribution based on the calculated confidence. (Page 6, section 4.3, “For detection, we propose stopping at the current value n when the largest standard deviation is a sufficiently small portion ¢ (e.g., t = 0.01) of the gap between the largest and smallest Shapley value estimates.” – EN: section 4.3 denotes outputting the final Shapley value attributions once the calculated confidence meets a specified convergence threshold.)
EN: the manufacturing aspects of the claim is taught by Senoner in the combination of Senoner in view of Aas as presented above in claim 1.
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the ML based manufacturing method of Senoner and the feature dependent machine learning prediction model of Aas with the multiple independent estimator variance approximation of Covert. The motivation for doing so would be to obtain a reliable estimate of the covariance of the Shapley value estimator. See Page 6, section 4.3 where covert explains “the original KernelSHAP (Bn) difficult to characterize” with respect to it’s variance, so Covert propose “calculating multiple independent estimates” at an intermediate sample size to approximate the covariance.
Claim 9
Senoner in view of Aas teaches the method of claim 1. Covert further teaches:generating another sample (Page 5, section 4.2, “‘When sampling n subsets according to the distribution 2; ~p(Z), we suggest a paired sampling strategy where each sample z; is paired with its complement 1-zi… consider using the following modified estimator that combines z; with 1 - z;:” – EN: this denotes evaluating the model using both the originally modified sample coalition and its complement (a second, distinct modification) to generate two sets of results.)
wherein the identifying of the attribution of the plurality of process features comprises: (Page 6, section 4.2, “Theorem [1] shows that 3, is a more precise estimator than B2n “
PNG
media_image2.png
381
498
media_image2.png
Greyscale
– EN this denotes computing tighter confidence bounds (variance ellipsoids) by combining the results from both the original and complement samples)
calculating a confidence of the sample data based on the sample (Page 6 Section 4.2, “Figure 2 illustrates the result of Theorem 1 empirical 95% confidence ellipsoids for two SHAP values.” – EN: this denotes outputting the converged Shapley value as the final feature attribution once the paired-sampling confidence is established.)
"calculating an attribution of a process feature based on the calculated confidence." (Page 6, Section 4.3, "we propose stopping at the current value n when the largest standard deviation is a sufficiently small portion t (e.g., t = 0.01) of the gap between the largest and smallest Shapley value estimates"; Appendix G, Algorithm 1, "converged = (max(σn)/(max(βn) − min(βn)) < t) … return βn, σn". – EN: the Shapley attribution βn of each feature is recalculated from additional samples until its standard deviation σn (the calculated confidence) meets the threshold, and only then returned; the attribution is thus calculated based on the calculated confidence.)
EN: the manufacturing aspects of the claim is taught by Senoner in the combination of Senoner in view of Aas as presented above in claim 1.
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the ML based manufacturing method of Senoner and the feature dependent machine learning prediction model of Aas with the paired sampling estimation technique of Covert. The motivation for doing so would be to obtain a more precise confidence estimates for feature attributions by evaluating the model on both an original sample modification and a second, distinct modification of that same sample. Covert explains that by combining a sample with its complement (a second, different modification) the resulting estimator produces tighter confidence bounds, noting that the paired estimator “is a more precise estimator” and illustrating that the approach yields smaller “95% confidence ellipsoids” when comparing results from both modifications. (Page 6, Section 4.2 and Page 6, Figure 2).
Claim 19
Senoner in view of Aas teaches the device of claim 15 as set forth above. Claim 19 recites, in device form, limitations substantially the same as those of method claim 7. Senoner in view of Aas, further in view of Covert, teaches these limitations for the reasons set forth for claim 7 above, and the motivation to combine set forth for claim 7 applies; as to the recited processor, see Senoner, page 5712. Therefore, claim 19 is rejected under the same rationale as claim 7.
Claims 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Senoner in view of Aas, further in view of Lundberg et al., “Explainable AI for Trees: From Local Explanations to Global Understanding” (hereinafter “Lundberg”)
Claim 10
Senoner in view of Aas teaches the method of claim 1. Lundberg further teaches:
wherein the generating of the (Page 18, “When explaining an ensemble model made up of a sum of many decision trees, the Saabas values for the ensemble model are defined as the sum of the Saabas values for each tree.” Page 3, “TreeExplainer enables the exact computation of optimal local explanations for tree-based models” applied to “ (Page 1) Random forests, gradient boosted trees, and other tree-based models… used in … manufacturing … and many other areas to make predictions based on sets of input features”)
wherein the identifying of the attribution of the plurality of process features comprises: respectively calculating attributions for a process feature, of the plurality of process features, based on the generated process results; and (Page 22, “Algorithm 2 reduces the computational complexity of exact SHAP value computation from exponential to low order polynomial for trees and sums of trees (since the SHAP values of a sum of two functions is the sum of the original functions’ SHAP values).” Page 23 Algorithm 2 TreeSHAP. – EN: this denotes algorithm 2 which computes per-feature attribution values (φ) for each individual tree.)
identifying an attribution of the process feature based on the calculated attributions. (Page 18, “When explaining an ensemble model made up of a sum of many decision trees, the Saabas values for the ensemble model are defined as the sum of the Saabas values for each tree.”)
EN: the manufacturing aspects of the claim is taught by Senoner in the combination of Senoner in view of Aas as presented above in claim 1.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the attribution framework of Senoner in view of Aas with the ensemble-based attribution computation of Lundberg. The motivation would have been to improve the reliability and accuracy of the feature attributions by using multiple models. See Lundberg, page 7, “Note that, as expected, Saabas becomes a better approximation to the Shapley values (and so a better attribution method) as the number of trees increases (Methods 9)”
Claim 11
Senoner in view of Aas teaches the method of claim 1. Lundberg further teaches: for first input data and second input data, respectively of the input data, comprising a same feature value for a target process feature, (Page 31, “Many different individuals have a recorded blood pressure of 180 mmHg in the mortality dataset, but the impact that measurement has on their log-hazard ratio varies from 0.2 to 0.6 because of other factors that differ among these individuals.” – EN: this denotes identifying multiple distinct input data samples that share the same feature value for a target feature.)
identifying a representative attribution, as the identified attribution, of the target process feature based on a sum of a first attribution of the target process feature, which is calculated based on the first input data, and a second attribution of the target process feature, which is calculated based on the second input data. (Page 30, “By averaging the SHAP values across a dataset, we can get a single global measure of feature importance that retains the theoretical guarantees of SHAP values.” Page 8, “a standard bar-chart based on the average magnitude of the SHAP values” is produced by combining “local explanations from TreeExplainer across an entire dataset” – EN: this denotes individual SHAP attributions are computed per-sample for each feature and then averaged across the dataset to produce a single representative global attribution value for each feature, where the averaging operation inherently involves summing a first attribution calculated from first input data and a second attribution calculated from second input data.”)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the attribution framework of Senoner in view of Aas with the attribution aggregation of Lundberg. The motivation would have been to obtain a better measure of a feature’s importance that accounts for variability across samples sharing the same feature value. See page 8 of Lundberg, "Combining local explanations from TreeExplainer across an entire dataset enhances traditional global representations of feature importance by: 1) avoiding the inconsistency problems of current methods (Supplementary Figure 2), 2) increasing the power to detect true feature dependencies in a dataset (Supplementary Figure 7), and 3) enabling us to build SHAP summary plots that succinctly display the magnitude, prevalence, and direction of a feature's effect."
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAYMUR RAHMAN ALI whose telephone number is (571)272-0007. The examiner can normally be reached Mon-Fri. 9:30-6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571)270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NAYMUR RAHMAN ALI/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights