Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is in response to application 18/771,783 filed 07/12/2024. Claims 1-20 are pending and are examined. No claims are allowed.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 2, and 13 recite “a model interpretability method”. However, this fails to comply with the written description requirement because the specification does not discuss the steps to perform this method. All that is described is the intended results (matrices) of the method with no steps outlining how the data is processed by the “model interpretability method” to arrive at the said results. For the purposes of compact prosecution “a model interpretability method” is interpreted to mean inputting data into a model, or any form of organizing data into tabular or matrix form.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) with no practical application and without significantly more.
The claimed invention is directed to an abstract idea in that the instant application is directed to mathematical concepts (See MPEP 2106.04(a)(2)(I)). The independent claims (1, 2, and 13) recite a method and systems to perform mathematical calculations to determine mathematical relationships between variables within datasets. These claim elements are being interpreted as mathematical concepts (including relationships and calculations). Performing matrix operations and computing differences and averages to determine relationships between variables falls under the “mathematical concept” grouping set forth in the MPEP 2106.04(a)(2)(I).
The claimed invention is directed to an abstract idea in that the instant application is directed to a mental process (See MPEP 2106.04(a)(2)(III)). The independent claims (1, 2, and 13) recite a method and systems to collect data, analyze data, and display results. These claim elements are being interpreted as concepts performed in the human mind (including observation, evaluation, judgement, and opinion). Observing relationships between data to identify drivers can equivalently be performed by human observation and evaluation of data. The claims recite an abstract idea consistent with the “mental process” grouping set forth in the MPEP 2106.04(a)(2)(III).
The instant application fails to integrate the judicial exceptions into a practical application because the instant application merely recites an “apply it” (or an equivalent) with the judicial exceptions, or merely includes instructions to implement an abstract idea. The instant application is directed towards a method and systems to implement the identified abstract ideas of mathematical concepts and mental processes (i.e. performing mathematical calculations to determine relationships between variables, and receiving information, processing information, and displaying the result of analysis and the like) in a general computer environment. The independent claims recite the additional elements “one or more processors”, “one or more non-transitory, computer-readable media”, “one or more machine learning models”, and “a baseline model. These claim elements are recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a general computer environment. The machines merely act as a modality to implement the abstract ideas and are not indicative of integration into a practical application (i.e., the additional elements are simply used as a tool to perform the abstract idea), see MPEP 2106.05(f).
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed in Step 2A Prong Two analysis, the additional elements in the claims amount to no more than mere instructions to apply the exception using generic computer components. The same analysis applies here in 2B and does not provide an inventive concept.
In regards to the dependent claims
Claims 4 and 16-17 recite an “updated model”. However, using an updated model is not indicative of integration into a practical application. The model merely acts as a modality to implement the abstract idea (i.e. the additional element is simply used as a tool to perform the abstract idea), see MPEP 2106.05(f)
Claims 3, 5-12, 14-15, and 18-20 recite no new additional elements or new abstract ideas and do not impact analysis under 35 USC 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-10, 12-15, and 18-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Whatley (US 20230033680 A1).
Regarding Claims 1, 2, and 13, (substantially similar in scope and language), Whatley teaches:
A system for identifying one or more variables as principal drivers of change for a value of interest, the system comprising: one or more processors; and one or more non-transitory, computer-readable media comprising instructions that, when executed by the one or more processors, cause operations comprising: [see at least Whatley: (Para 0018) “In yet another aspect, example embodiments may involve an article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, when executed by one more processors of a system, cause the system to carry out various operations”]
receiving, from a user, a request for identifying one or more variables from a set of variables as principal drivers of change for a value of interest, wherein the request comprises (1) the set of variables, (2) a baseline dataset, and (3) an updated dataset, wherein the updated dataset exhibits a change in the value of interest as compared to the baseline dataset; [see at least Whatley: (Para 0016) “computing a fair distribution of first respective quantitative contributions of each of the plurality of operational features to the one or more predicted performance characteristics of the trained ML model, wherein the first subset includes at least those training data records sufficient to represent a baseline of observed performance characteristics; for each input data record of a second subset of the set of training data records”]
generating, using one or more machine learning models, a baseline model indicative of a relationship between the value of interest and each variable of the set of variables based on the baseline dataset; [see at least Whatley: (Para 0016) “using at least a portion of the set of training data records to train a machine learning (ML) model of network performance to predict expected performance characteristics given the plurality of operational features in the training data records as input and the one or more observed performance characteristics as ground truths,”]
processing the baseline model using the baseline dataset with a model interpretability method to obtain a first matrix and processing the baseline model using the updated dataset with the model interpretability method to obtain a second matrix, wherein each matrix comprises quantitative measures of a contribution of each variable in the set of variables to the value of interest for each sample; [see at least Whatley: (Table 1), (Para 0016) “for each input data record of a first subset of the set of training data records, computing a fair distribution of first respective quantitative contributions of each of the plurality of operational features to the one or more predicted performance characteristics… for each input data record of a second subset of the set of training data records, computing a fair distribution of second respective quantitative contributions of each of the plurality of operational features to the one or more predicted performance characteristics …”]
computing a population shift value representing a change in data distribution by computing an absolute difference between each of a plurality of row averages of the first matrix and a corresponding plurality of row averages of the second matrix; [see at least Whatley: (Para 0111) “In either case, it is taken to be representative of overall network performance, such that it provides a baseline or control sample of predicted network performance against which predicted performance of the sample of interest can be compared. When taking a random sample for the entire training set, the mean values of all feature contributions is expected to be zero (0), and, consequently, the sample is not explicitly required in cases where only the difference in mean feature contributions are compared”, (Para 0124) “To do this, feature impact may be evaluated by comparing SHAP values distributions from the sample of interest to an appropriate baseline, which can be randomly sampled baseline or a selectively sampled baseline constructed to be representative by selective sampling from a non-problematic data set (i.e., control sample). The difference can be quantified, for example, by computing an effect size (e.g, Hedge's g) for a difference that compares the baseline importance of a specific feature in the problematic sample case to its importance in the base case”]
and identifying, from the set of variables, the one or more variables as principal drivers of change for the value of interest based on absolute differences between row averages. [see at least Whatley: (Para 0113) “As such, the fair-distribution approach effectively yields an empirically derived quantitative contribution for each input feature-value pair in a record”, (Para 0124) “The difference can be quantified, for example, by computing an effect size (e.g, Hedge's g) for a difference that compares the baseline importance of a specific feature in the problematic sample case to its importance in the base case. This offers an objective way to gauge the impact of feature importance given the full context presented in the problematic sample”]
Regarding Claim 3, Whatley teaches the limitations set forth above, Whatley further teaches:
generating a command for displaying, to a user, the one or more variables; and transmitting the command to a remote device. [see at least Whatley: (Para 0058), “The User Interface 116 may display results, as well as provide a user to adjust and/or select specific types of analyses to conduct”, (Figures 2-3), (Para 0080) “As noted, server devices 306 may be configured to transmit data to and receive data from cluster data storage”, (Para 0074) “Accordingly, the computing devices may be referred to as “cloud-based” devices that may be housed at various remote locations”]
Regarding Claim 4, Whatley teaches the limitations set forth above, Whatley further teaches:
generating, using the one or more machine learning models, an updated model indicative of a relationship between the value of interest and each variable of the set of variables based on the updated dataset. [see at least Whatley: (Para 0102) “As shown in FIG. 4, model training involves a training loop in which the ML model is evaluated in an evaluation operation 402-a to generate outputs followed by an updating operation 404 to update/adjust model parameters to minimize error/loss functions, after which the model is reevaluated using the updated model parameters.”]
Regarding Claim 5, Whatley teaches the limitations set forth above, Whatley further teaches:
wherein computing the absolute difference between each of the plurality of row averages of the first matrix and the corresponding plurality of row averages of the second matrix comprises: averaging values of each row of the first matrix to obtain a first plurality of average values, wherein each average value of the first plurality of average values corresponds to an average contribution of a variable to the value of interest in samples of the baseline dataset according to the baseline model; and averaging values of each row of the second matrix to obtain a second plurality of average values, wherein each average value of the second plurality of average values corresponds to an average contribution of a variable to the value of interest in samples of the updated dataset according to the baseline model. [see at least Whatley: (Para 0016), (Para 0113) “Rather, the fair distribution criteria in an additive explanation strategy like SHAP values ensures that for each given predicted performance characteristic, the sum of the contributions of each of the input feature-value pairs adds up to the difference between the predicted performance characteristic for the record and the mean (i.e., expected) performance characteristic of all records in the model training set.”]
Regarding Claim 6, Whatley teaches the limitations set forth above, Whatley further teaches:
further comprising computing the absolute difference between each average value of the first plurality of average values and a corresponding average value of the second plurality of average values. [see at least Whatley: (Para 0124) “The difference can be quantified, for example, by computing an effect size (e.g, Hedge's g) for a difference that compares the baseline importance of a specific feature in the problematic sample case to its importance in the base case”]
Regarding Claim 7, Whatley teaches the limitations set forth above, Whatley further teaches:
further comprising determining the set of variables for analysis from a superset of variables based on data of the baseline dataset and the updated dataset. [see at least Whatley: (Para 0014-0015) “The strategy involves first representing telecom data into a non-linear model, such as a decision tree structure formed using a gradient boosting algorithm, to represent relationships and interactions between a set of inputs (e.g., call detail record, or CDR, dimensions and metrics) to a target variable outcome (e.g., average throughput, call status) that might serve as a key performance indicator (KPI)…”]
Regarding Claim 8 and 18, Whatley teaches the limitations set forth above, Whatley further teaches:
further comprising: generating a graphical representation of a difference between each of the plurality of row averages of the first matrix and the corresponding plurality of row averages of the second matrix; and [see at least Whatley: (Figure 6A-6B), (Figures 9-11)]
generating a command for displaying, to a user, the graphical representation. [see at least Whatley: (Para 0057) “The User Interface 116 may display results, as well as provide a user to adjust and/or select specific types of analyses to conduct.”]
Regarding Claim 9 and 19, Whatley teaches the limitations set forth above, Whatley further teaches:
wherein the first matrix and the second matrix are two-dimensional matrices comprising rows and columns, each row of the first matrix representing a sample from the baseline dataset and each column representing a variable from the set of variables. [see at least Whatley: (Table 1), (Para 0042) “Each row of the table may also correspond to a record of a database or dataset of performance data that may be obtained by one or more monitoring devices in or of a communication network.” (Para 0044) “Specifically, it may be seen that each feature corresponds to a column in the table, and that the features of each row correspond to feature vectors.”]
Regarding Claim 10 and 20, Whatley teaches the limitations set forth above, Whatley further teaches:
further comprising obtaining a user selection of the set of variables for analysis from a superset of variables. [see at least Whatley: (Para 0057) “The User Interface 116 may display results, as well as provide a user to adjust and/or select specific types of analyses to conduct”, (Para 0082) “Given a geographical region with low throughput, CDRs within the selected region can be analyzed”]
Regarding Claim 12, Whatley teaches the limitations set forth above, Whatley further teaches:
further comprising transmitting, to a remote server, a request for storing parameters of the baseline model. [The limitations describe storing data; see at least Whatley: (Para 0080) “As noted, server devices 306 may be configured to transmit data to and receive data from cluster data storage”, (Para 0114) “Further, the feature-contribution data generated from the control sample of data records can provide baseline feature contributions for one or more predicted performance characteristics output by the ML model.”]
Regarding Claim 14, Whatley teaches the limitations set forth above, Whatley further teaches:
wherein the instructions further cause operations comprising: identifying, from the set of variables, the one or more variables as principal drivers of change for a value of interest based on the absolute differences between the plurality of row averages; and [see at least Whatley: (Para 0113) “As such, the fair-distribution approach effectively yields an empirically derived quantitative contribution for each input feature-value pair in a record”, (Para 0124) “The difference can be quantified, for example, by computing an effect size (e.g, Hedge's g) for a difference that compares the baseline importance of a specific feature in the problematic sample case to its importance in the base case. This offers an objective way to gauge the impact of feature importance given the full context presented in the problematic sample”]
generating a command for displaying, to a user, the one or more variables. [see at least Whatley: (Para 0058), “The User Interface 116 may display results, as well as provide a user to adjust and/or select specific types of analyses to conduct”]
Regarding Claim 15, Whatley teaches the limitations set forth above, Whatley further teaches:
wherein the instructions further cause operations comprising determining the set of variables for analysis from a superset of variables based on data of the baseline dataset and the updated dataset. [see at least Whatley: (Para 0014-0015) “The strategy involves first representing telecom data into a non-linear model, such as a decision tree structure formed using a gradient boosting algorithm, to represent relationships and interactions between a set of inputs (e.g., call detail record, or CDR, dimensions and metrics) to a target variable outcome (e.g., average throughput, call status) that might serve as a key performance indicator (KPI)…”]
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Whatley (US 20230033680 A1) in view of Sengupta (WO 2016160734 A1).
Regarding Claim 11, Whatley teaches all the limitations of claim 10, Whatley further teaches:
further comprising: determining a value indicative of model performance of the baseline model; and [see at least Whatley: (Para 0092) “This is done by using “random data,” or “control data,” after applying appropriate filters to produce a control, from the training data to derive or generate baseline performance”]
responsive to determining that the value does not exceed a minimum threshold for model performance, [see at least Whatley: (Para 0102) “Once the prescribed or threshold level of prediction performance is attained, the model may be considered to be trained. It may then be used in the optional second, evaluation phase, or directly in the third, analysis phase”, (Para 0112) “More specifically, the problematic sample may be selected specifically on the basis of an observed performance characteristic that is considered problematic, suboptimal, or otherwise representative of degraded performance. For example, the observed dropped call rate in a particular region may be unacceptably high compared to an observed dropped call rate averaged over all regions. The data composing the problematic sample may thus be selected to include all records for the particular region that were dropped.”]
However, Whatley does not teach but Sengupta does teach:
generating a command for prompting a user to select a new set of features from a superset of features. [see at least Sengupta: (Figures 8J-8N)]
Further, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method of determining relationships between variables (Whatley) with the generation of a prompt for a user (Sengupta). One of ordinary skill would have recognized the benefits of prompting a user to select new features after the model is able to perform its task. Generating a prompt for a user to select features would have yielded predictable results in the tasks performed by the system taught by Whatley.
Regarding Claim 16, Whatley teaches all the limitations of claim 13, Whatley further teaches:
wherein the instructions further cause operations comprising: determining a value indicative of model performance of an updated model; and responsive to determining that the value does not exceed a minimum threshold for model performance, [see at least Whatley: (Para 0102) As shown in FIG. 4, model training involves a training loop in which the ML model is evaluated in an evaluation operation 402-a to generate outputs followed by an updating operation 404 to update/adjust model parameters to minimize error/loss functions, after which the model is reevaluated using the updated model parameters. The model is applied to the training data, and the loop begins with an initial model input (e.g., initial parameters). The loop is carried out until a prescribed level of agreement between the model predictions and the ground truths is achieved.]
However, Whatley does not teach but Sengupta does teach:
generating a command for prompting a user to select a new set of features from a superset of features. [see at least Sengupta: (Figures 8J-8N)]
Further, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method of determining relationships between variables (Whatley) with the generation of a prompt for a user (Sengupta). One of ordinary skill would have recognized the benefits of prompting a user to select new features after the model is able to perform its task. Generating a prompt for a user to select features would have yielded predictable results in the tasks performed by the system taught by Whatley.
Regarding Claim 17, the combination of Whatley and Sengupta teach all the limitations of claim 13, Whatley further teaches:
wherein the instructions further cause operations comprising transmitting, to a remote server, a request for storing parameters of the updated model. [The limitations describe storing data; see at least Whatley: (Para 0080) “As noted, server devices 306 may be configured to transmit data to and receive data from cluster data storage”, “Once the prescribed or threshold level of prediction performance is attained, the model may be considered to be trained. It may then be used in the optional second, evaluation phase, or directly in the third, analysis phase.”]
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Examiner Benjamin Truong, whose telephone number is 703-756-5883. The examiner can normally be reached on Monday-Friday from 9 am to 5 pm (EST)
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nathan Uber SPE can be reached on 571-270-3923. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300 Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.L.T./
Examiner, Art Unit 3626
/NATHAN C UBER/Supervisory Patent Examiner, Art Unit 3626