Notice of Pre-AIA or AIA Status
This Final communication is in response to Application No. 18/217,876 filed 07/03/2023. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 6/23/2026 have been entered. They provide amendments to claims 1-4, 10-13 and cancelled claims 9 and 18. Claims 1-8, 10-17, 19-20 are pending.
Response to Arguments
Applicant’s arguments with respect to 35 U.S.C 101 filed 6/23/2026 (pages 11-16 of applicant’s arguments) have been fully considered but they are not persuasive.
Applicant argues the amended independent claims are integrated into a practical application. Specifically, applicant states the claimed features provide an improvement to existing machine learning and artificial intelligence technology for identifying and providing consistent explanations over time. The examiner respectfully disagrees. According to MPEP 2106.05(a) the improvement cannot come from the judicial exception (mental process in this case) alone. The improvement must be shown by the additional elements. The additional elements of the independent claims are directed to receiving data, segmenting the data, training models based on the data and outputting the reconciled explanation. It is unclear how these additional elements show identifying and providing consistent explanations over time.
Applicant argues the amended independent claims are not directed to an abstract idea. The examiner respectfully disagrees. According to MPEP 2106.04(a), a claim recites an abstract idea if a limitation recites one of the following: mathematical concept, certain methods of organizing human activity, or mental processes. The limitations: “generating, …, at least one prediction for a target time that is indicated by a timestamp included in the at least one input;”, “generating, by the at least one processor, a set of common background data for each time window based on the at least one input;”, “determining, by the at least one processor, at least one respective mode explanation for each time window based on the corresponding set of common background data, the corresponding at least one trained model, and the corresponding at least one prediction, wherein each respective mode explanation of the at least one respective mode explanation includes at least one respective feature attribution that is generated for each time window with respect to a respective corresponding background data distribution of the generated set of common background data associated with the corresponding respective time window;”, and “determining, by the at least one processor, at least one reconciled explanation for a target prediction that corresponds to the target time based on the at least one input and the at least one respective mode explanation; and” given their broadest reasonable interpretations can all be performed in the human mind or with pen/paper. A person can make decisions determining and identifying patterns and relationships. A person can make computations using pen and paper. Thus, the claims do recite a judicial exception.
Thus the 101 rejection is maintained.
Applicant’s arguments with respect to 35 U.S.C 103 filed 6/23/2026 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
101 Subject Matter Eligibility Analysis
Step 1: Claims 1-20 are within the four statutory (a process, machine, manufacture or composition of
matter.) Claims 1-9 describe a process and 10-20 describes a machine.
With respect to claim 1:
Step 2A Prong 1: The claim recites an abstract idea enumerated in the 2019 PEG.
generating, …, at least one prediction for a target time that is indicated by a timestamp included in the at least one input; (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The generation could be made manually by an individual.)
generating, by the at least one processor, a set of common background data for each time window based on the at least one input; (This is an abstract idea of a "Mental Process." The "generating" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The generation could be made manually by an individual.)
determining, by the at least one processor, at least one respective mode explanation for each time window based on the corresponding set of common background data, the corresponding at least one trained model, and the corresponding at least one prediction, wherein each respective mode explanation of the at least one respective mode explanation includes at least one respective feature attribution that is generated for each time window with respect to a respective corresponding background data distribution of the generated set of common background data associated with the corresponding respective time window; (This is an abstract idea of a "Mental Process." The "determining" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The determination could be made manually by an individual.)
determining, by the at least one processor, at least one reconciled explanation for a target prediction that corresponds to the target time based on the at least one input and the at least one respective mode explanation; and (This is an abstract idea of a "Mental Process." The "determining" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The determination could be made manually by an individual.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application
Additional elements:
receiving, by the at least one processor via an application programming interface, at least one input; (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
temporally segmenting, by the at least one processor, the at least one input to generate a finite set of time windows; (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
training, by the at least one processor, at least one model for each time window of the set of time windows, wherein the at least one model includes a neural network that uses interconnected nodes in a layered structure for transforming the at least one input for predictive analytics, and wherein the at least one model is trained using raw data from each time window that is included in the at least one input; (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
by the at least one processor using each of the at least one trained model, (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
outputting, by the at least one processor, the at least one reconciled explanation from the at least one model. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception
The additional elements “receiving…”, “temporally segmenting…” and “outputting…” add insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).The additional elements “training…” and “by the at least one processor…” are recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
When considered in combination, these additional elements represent insignificant extra-solution activity and mere instructions to apply an expectation, which do not provide an inventive concept.
Therefore, claim 1 is ineligible
With respect to claim 2:
Step 2A Prong 1: claim 2, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the at least one input includes at least one from among a parameter, a weighting function that prioritizes a plurality of temporal time windows for consistency evaluation, a data sampling strategy, and a consistency factor. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 2 is ineligible.
With respect to claim 3:
Step 2A Prong 1: claim 3, which incorporates the rejection of claim 2, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the raw data includes a series of data that represents an evolution of information over time, and wherein the parameter includes a required number of the time windows. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 3 is ineligible.
With respect to claim 4:
Step 2A Prong 1: claim 4, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the at least one respective feature attribution is computed for each of a plurality of segmented time windows with respect to a specific corresponding background data distribution. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 4 is ineligible.
With respect to claim 5:
Step 2A Prong 1: claim 5, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the at least one reconciled explanation corresponds to a consistent explanation for each of the at least one prediction over time. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional elements add insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 5 is ineligible.
With respect to claim 6:
Step 2A Prong 1: claim 6, which incorporates the rejection of claim 1, recites an additional abstract idea:
determining, by the at least one processor, a consistency value for a plurality of features in each of the at least one respective mode explanation; and (This is an abstract idea of a "Mental Process." The "determining" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The determination could be made manually by an individual.)
removing, by the at least one processor, at least one feature from the plurality of features based on the consistency value and a consistency factor. (This is an abstract idea of a "Mental Process." The "removing" step under its broadest reasonable interpretation, covers concepts that can be practically performed by a human using a pen and paper.)
Step 2A Prong 2: claim 6 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 6 does not recite an additional element.
Therefore, claim 6 is ineligible.
With respect to claim 7:
Step 2A Prong 1: claim 7, which incorporates the rejection of claim 6, recites an additional abstract idea:
weighting, by the at least one processor using a weighting function, at least one remaining feature from the plurality of features based on a result of the removing; and (this is an abstract idea of a “mathematical concept”. The recited “weighting function” represents a mathematical function that would fall under the “mathematical concepts” grouping.)
determining, by the at least one processor, at least one respective reconciled feature score for each of the at least one remaining feature. (This is an abstract idea of a "Mental Process." The "determining" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The determination could be made manually by an individual.)
Step 2A Prong 2: claim 7 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 7 does not recite an additional element.
Therefore, claim 7 is ineligible.
With respect to claim 8:
Step 2A Prong 1: claim 8, which incorporates the rejection of claim 6, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the consistency value relates to a feature value distance of a plurality of proximate features that is determined based on a graphical projection of the at least one respective mode explanation. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional elements add insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 8 is ineligible.
With respect to claim 10:
The claim recites similar limitations as corresponding to claim 1. Therefore, the same subject matter analysis that was utilized for claim 1, as described above, is equally applicable to claim 10. Therefore, claim 10 is ineligible.
With respect to claim 11:
The claim recites similar limitations as corresponding to claim 2. Therefore, the same subject matter analysis that was utilized for claim 2, as described above, is equally applicable to claim 11. Therefore, claim 11 is ineligible.
With respect to claim 12:
The claim recites similar limitations as corresponding to claim 3. Therefore, the same subject matter analysis that was utilized for claim 3, as described above, is equally applicable to claim 12. Therefore, claim 12 is ineligible.
With respect to claim 13:
The claim recites similar limitations as corresponding to claim 4. Therefore, the same subject matter analysis that was utilized for claim 4, as described above, is equally applicable to claim 13. Therefore, claim 13 is ineligible.
With respect to claim 14:
The claim recites similar limitations as corresponding to claim 5. Therefore, the same subject matter analysis that was utilized for claim 5, as described above, is equally applicable to claim 14. Therefore, claim 14 is ineligible.
With respect to claim 15:
The claim recites similar limitations as corresponding to claim 6. Therefore, the same subject matter analysis that was utilized for claim 6, as described above, is equally applicable to claim 15. Therefore, claim 15 is ineligible.
With respect to claim 16:
The claim recites similar limitations as corresponding to claim 7. Therefore, the same subject matter analysis that was utilized for claim 7, as described above, is equally applicable to claim 16. Therefore, claim 16 is ineligible.
With respect to claim 17:
The claim recites similar limitations as corresponding to claim 8. Therefore, the same subject matter analysis that was utilized for claim 8, as described above, is equally applicable to claim 17. Therefore, claim 17 is ineligible.
With respect to claim 19:
The claim recites similar limitations as corresponding to claim 1. Therefore, the same subject matter analysis that was utilized for claim 1, as described above, is equally applicable to claim 19. Therefore, claim 19 is ineligible.
With respect to claim 20:
The claim recites similar limitations as corresponding to claim 2. Therefore, the same subject matter analysis that was utilized for claim 2, as described above, is equally applicable to claim 20. Therefore, claim 20 is ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Nayebi (NPL: ‘WindowSHAP: An Efficient Framework for Explaining Time-series Classifiers based on Shapley Values’ (2023)) in view of Song (US 12,518,201 B1), Raykar (US 2023/0419136 A1) and Bento (NPL: ‘TimeSHAP: Explaining Recurrent Models through Sequence Perturbations’ (2021)).
Regarding claim 1, Nayebi teaches:
A method for providing explanations of predictive outputs, the method being implemented by at least one processor, the method comprising: (Abstract)
receiving, by the at least one processor via an application programming interface, at least one input; (Algorithm 1 on pages 8-9 shows input sequence).
temporally segmenting, by the at least one processor, the at least one input to generate a finite set of time windows; (Page 8, 3.3.1 Stationary WindowSHAP “In this approach, the time-axis is segmented into fixed-length windows.”)
training, by the at least one processor, at least one model for each time window of the set of time windows, wherein the at least one model includes a neural network that uses interconnected nodes in a layered structure for transforming the at least one input for predictive analytics, and wherein the at least one model is trained using raw data from each time window that is included in the at least one input; (Page 13. 3.5 Data sources “To test the model-agnostic explanation methods (e.g., WindowSHAP), we used three distinct clinical time-series data sets to develop and train three different deep learning prediction models.” And Page 15 mentions using recurrent neural networks and table 3 shows the information (number of layer, neurons etc.) about the recurrent neural networks.)
generating, by the at least one processor using each of the at least one trained model, at least one prediction for a target time that is indicated by a timestamp included in the at least one input; (Page 13. 3.5 Data sources ” To test the model-agnostic explanation methods (e.g., WindowSHAP), we used three distinct clinical time-series data sets to develop and train three different deep learning prediction models.” Prediction model is making prediction for the windows)
Nayebi does not teach:
Prediction model for each time window
Model is trained using raw data
Input data includes a timestamp
generating, by the at least one processor, a set of common background data for each of the at least one time window based on the at least one input;
determining, by the at least one processor, at least one respective mode explanation for each time window based on the corresponding set of common background data, the corresponding at least one trained model, and the corresponding at least one prediction wherein each respective mode explanation of the at least one respective mode explanation includes at least one respective feature attribution that is generated for each time window with respect to a respective corresponding background data distribution of the generated set of common background data associated with the corresponding respective time window
determining, by the at least one processor, at least one reconciled explanation for a target prediction that corresponds to the target time based on the at least one input and the at least one respective mode explanation; and
Outputting, by the at least one processor, the at least one reconciled explanation from the at least one model.
Song teaches:
Prediction model for each time window (Col 5 lines 31-35 “To train the time-restricted entity analysis model using the segmented training data sets at block 106, the computer system can train multiple different time-segmented machine learning models corresponding to the multiple segments of the training data set.”)
Model is trained using raw data from each time window(Col 5 lines 37-50 “The computer system uses the first training data set to train a first random forest model, which is configured to use attributes of a first set of entities to predict whether the target action will occur with respect to entities in the first set during the time period between t1 and t2. A second set of the segmented training data includes training data with timestamps falling between time t3 and time t4. Like the first training data set, the computer system uses the second training data set to train a second random forest model that, when trained, is configured to use attributes of a second set of entities to predict whether the target action will occur with respect to the entities in the second set during the time period between t3 and t4.” Song also mentions the model can be a neural network)
Input data includes a timestamp (Col 5 lines 35-37 “For example, a first set of the segmented training data includes training data with timestamps falling between time t1 and time t2.”)
Nayebi and Song are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine one model for each time period of Song with the explanation system of Nayebi and include the background data in the explanation determination. One would be motivated to do this to account for time-restricted variability (Song background).
Raykar teaches:
generating, by the at least one processor, a set of common background data for each of the at least one time window based on the at least one input; ([0040] “In operation 810, the model component 130 generates a surrogate data set. The surrogate data set may be generated by backtesting one or more time series forecasting predictions of the set of time series forecasting predictions.”)
Nayebi, Song and Raykar are considered analogous art to the claimed invention because they are in the same field of endeavor being AI Explanation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the background data of Raykar with the explanation system of Nayebi and include the background data in the explanation determination. One would want to do this to create better explanations (Raykar background).
Bento teaches:
determining, by the at least one processor, at least one respective mode explanation for each time window based on the corresponding set of common background data, the corresponding at least one trained model, and the corresponding at least one prediction wherein each respective mode explanation of the at least one respective mode explanation includes at least one respective feature attribution that is generated for each time window with respect to a respective corresponding background data distribution of the generated set of common background data associated with the corresponding respective time window (Section 3 on page 3 teaches their explanation method. More specifically equation 7 is generating explanation features which uses input data X and background data B. As mentioned above Song teaches the model for each time window.)
determining, by the at least one processor, at least one reconciled explanation for a target prediction that corresponds to the target time based on the at least one input and the at least one respective mode explanation. (Page 7 Section 4.2 Global Explanations “We compute global explanations by applying TimeSHAP to all sequences that contained a positive prediction, and explaining the first positive prediction of each sequence (hence referred as 𝑡 = 0). We use 𝜂 = 0.025 as the tolerance of the temporal coalition pruning algorithm.”)
Outputting, by the at least one processor, the at least one reconciled explanation from the at least one model. (Page 6 section 4.2 “Supplying a data scientist with global explanations provides an overview of the model’s decision process, revealing which features, or, in the case of TimeSHAP, events, are relevant to the model and which ones are not. TimeSHAP provides this type of explanations via aggregations and visualizations of local explanations.” Giving the explanations to a data scientist implies they are outputted).
Nayebi, Song, Raykar, and Bento are considered analogous art to the claimed invention because they are in the same field of endeavor being AI Explanation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the time window explanation system of Nayebi with the global explanation system of Bento. One would want to understand the model’s decision process as a whole (Bento 4.2 Global Explanations)
Regarding claim 2, Nayebi in view of Song, Raykar and Bento teaches claim 1 as outlined above. Nayebi further teaches:
the at least one input includes at least one from among a parameter, a weighting function that prioritizes a plurality of temporal time windows for consistency evaluation, a data sampling strategy, and a consistency factor. (Algorithm 1 on pages 8-9 shows input sequence).
Regarding claim 3, , Nayebi in view of Song, Raykar and Bento teaches claim 2 as outlined above. Nayebi further teaches:
the raw data includes a series of data that represents an evolution of information over time, and wherein the parameter includes a required number of the time windows. (Nayebi is working with time-series data. Page 8 Section 3.3.1 “In this approach, the time-axis is segmented into fixed-length windows. Even though all time windows have the same length, if the length of the sequence is not divisible by the length of the time window, the last time window may be smaller than the others.”)
Regarding claim 4, , Nayebi in view of Song, Raykar and Bento teaches claim 1 as outlined above. Raykar further teaches:
the at least one respective feature attribution is computed for each of a plurality of segmented time windows with respect to a specific corresponding background data distribution. ([0030] “The subset of surrogate models may be selected based on features used in the models, feature importance, model simplicity, or any other suitable interpretable aspect.” And [0031] “The one or more explanation outputs may be generated based on the subset of surrogate models.”)
Regarding claim 5, , Nayebi in view of Song, Raykar and Bento teaches claim 1 as outlined above. Bento further teaches:
the at least one reconciled explanation corresponds to a consistent explanation for each of the at least one prediction over time. (Page 4 Section 3 TimeSHAP “Additionally, we aim to explain sequential models while preserving the three desirable properties of importance attribution stemming from the Shapley values: local accuracy, missingness ,and consistency”)
Regarding claim 6, , Nayebi in view of Song, Raykar and Bento teaches claim 1 as outlined above. Bento further teaches:
determining, by the at least one processor, a consistency value for a plurality of features in each of the at least one respective mode explanation; and (Page 4 Section 3 TimeSHAP “TimeSHAP attributes an importance value to each feature/event in the input that reflects the degree to which that feature/event affected the final prediction.”
removing, by the at least one processor, at least one feature from the plurality of features based on the consistency value and a consistency factor. (Page 5 Section 3.2 Pruning “One glaring issue with TimeSHAP is that the number of event (temporal) coalitions scales exponentially with the length of the observed sequence, just as in KernelSHAP the number of feature coalitions scales exponentially with the number of input features. Moreover, in a recurrent setting, the input sequence can be arbitrarily long. We address this issue by proposing a temporal coalition pruning algorithm.”)
Regarding claim 7, , Nayebi in view of Song, Raykar and Bento teaches claim 6 as outlined above. Bento further teaches:
weighting, by the at least one processor using a weighting function, at least one remaining feature from the plurality of features based on a result of the removing; and (Page 4. Section 3 TimeSHAP “the bias term 𝑤0=𝑓(ℎ𝑋(0)) corresponds to the model’s output with all features toggled off (dubbed base score), the weights 𝑤𝑖,𝑖 ∈{1,...,𝑚}, correspond to the importance of each feature”)
determining, by the at least one processor, at least one respective reconciled feature score for each of the at least one remaining feature. (Page 7 Section 4.2 Global Explanations “We compute global explanations by applying TimeSHAP to all sequences that contained a positive prediction, and explaining the first positive prediction of each sequence (hence referred as 𝑡 = 0). We use 𝜂 = 0.025 as the tolerance of the temporal coalition pruning algorithm.”)
Regarding claim 8, , Nayebi in view of Song, Raykar and Bento teaches claim 6 as outlined above. Bento further teaches:
the consistency value relates to a feature value distance of a plurality of proximate features that is determined based on a graphical projection of the at least one respective mode explanation. (Page 8 Section 4.3 Local Explanations “As expected, the aggregate importance of older events (from the beginning of the sequence up to index𝑡) suffers a steep decrease as its distance to the current event increases”)
Regarding claim 10, Nayebi teaches:
receive, via an application programming interface, at least one input; (Algorithm 1 on pages 8-9 shows input sequence).
temporally segment the at least one input to generate a finite set of time windows; (Page 8, 3.3.1 Stationary WindowSHAP “In this approach, the time-axis is segmented into fixed-length windows.”)
train, by the at least one processor, at least one model for each time window of the set of time windows, wherein the at least one model includes a neural network that uses interconnected nodes in a layered structure for transforming the at least one input for predictive analytics, and wherein the at least one model is trained using raw data from each time window that is included in the at least one input; (Page 13. 3.5 Data sources “To test the model-agnostic explanation methods (e.g., WindowSHAP), we used three distinct clinical time-series data sets to develop and train three different deep learning prediction models.” And Page 15 mentions using recurrent neural networks and table 3 shows the information (number of layer, neurons etc.) about the recurrent neural networks.)
generate, by the at least one processor using each of the at least one trained model, at least one prediction for a target time that is indicated by a timestamp included in the at least one input; (Page 13. 3.5 Data sources ” To test the model-agnostic explanation methods (e.g., WindowSHAP), we used three distinct clinical time-series data sets to develop and train three different deep learning prediction models.” Prediction model is making prediction for the windows)
Nayebi does not teach:
A computing device configured to implement an execution of a method for providing explanations of predictive outputs, the computing device comprising: a processor; a memory; and
Prediction model for each time window
Model is trained using raw data
Input data includes a timestamp
generate, by the at least one processor, a set of common background data for each of the at least one time window based on the at least one input;
determine, by the at least one processor, at least one respective mode explanation for each time window based on the corresponding set of common background data, the corresponding at least one trained model, and the corresponding at least one prediction wherein each respective mode explanation of the at least one respective mode explanation includes at least one respective feature attribution that is generated for each time window with respect to a respective corresponding background data distribution of the generated set of common background data associated with the corresponding respective time window
determine, by the at least one processor, at least one reconciled explanation for a target prediction that corresponds to the target time based on the at least one input and the at least one respective mode explanation; and
Output, by the at least one processor, the at least one reconciled explanation from the at least one model.
Song teaches:
Prediction model for each time window (Col 5 lines 31-35 “To train the time-restricted entity analysis model using the segmented training data sets at block 106, the computer system can train multiple different time-segmented machine learning models corresponding to the multiple segments of the training data set.”)
Model is trained using raw data from each time window(Col 5 lines 37-50 “The computer system uses the first training data set to train a first random forest model, which is configured to use attributes of a first set of entities to predict whether the target action will occur with respect to entities in the first set during the time period between t1 and t2. A second set of the segmented training data includes training data with timestamps falling between time t3 and time t4. Like the first training data set, the computer system uses the second training data set to train a second random forest model that, when trained, is configured to use attributes of a second set of entities to predict whether the target action will occur with respect to the entities in the second set during the time period between t3 and t4.” Song also mentions the model can be a neural network)
Input data includes a timestamp (Col 5 lines 35-37 “For example, a first set of the segmented training data includes training data with timestamps falling between time t1 and time t2.”)
Nayebi and Song are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine one model for each time period of Song with the explanation system of Nayebi and include the background data in the explanation determination. One would be motivated to do this to account for time-restricted variability (Song background).
Raykar teaches:
A computing device configured to implement an execution of a method for providing explanations of predictive outputs, the computing device comprising: a processor; a memory; and ([0047] “As shown in the figure, computer system/server 900 is shown in the form of a general-purpose computing device. The components of computer system/server 900 may include, but are not limited to, one or more processors 902 (e.g., processing units), a system memory 904 (e.g., a computer-readable storage medium coupled to the one or more processors), and a bus 906 that couple various system components including system memory 904 to the processor 902.”)
generate, by the at least one processor, a set of common background data for each of the at least one time window based on the at least one input; ([0040] “In operation 810, the model component 130 generates a surrogate data set. The surrogate data set may be generated by backtesting one or more time series forecasting predictions of the set of time series forecasting predictions.”)
Nayebi, Song and Raykar are considered analogous art to the claimed invention because they are in the same field of endeavor being AI Explanation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the background data of Raykar with the explanation system of Nayebi and include the background data in the explanation determination. One would want to do this to create better explanations (Raykar background).
Bento teaches:
determine, by the at least one processor, at least one respective mode explanation for each time window based on the corresponding set of common background data, the corresponding at least one trained model, and the corresponding at least one prediction wherein each respective mode explanation of the at least one respective mode explanation includes at least one respective feature attribution that is generated for each time window with respect to a respective corresponding background data distribution of the generated set of common background data associated with the corresponding respective time window (Section 3 on page 3 teaches their explanation method. More specifically equation 7 is generating explanation features which uses input data X and background data B. As mentioned above Song teaches the model for each time window.)
determine, by the at least one processor, at least one reconciled explanation for a target prediction that corresponds to the target time based on the at least one input and the at least one respective mode explanation. (Page 7 Section 4.2 Global Explanations “We compute global explanations by applying TimeSHAP to all sequences that contained a positive prediction, and explaining the first positive prediction of each sequence (hence referred as 𝑡 = 0). We use 𝜂 = 0.025 as the tolerance of the temporal coalition pruning algorithm.”)
Output, by the at least one processor, the at least one reconciled explanation from the at least one model. (Page 6 section 4.2 “Supplying a data scientist with global explanations provides an overview of the model’s decision process, revealing which features, or, in the case of TimeSHAP, events, are relevant to the model and which ones are not. TimeSHAP provides this type of explanations via aggregations and visualizations of local explanations.” Giving the explanations to a data scientist implies they are outputted).
Nayebi, Song, Raykar, and Bento are considered analogous art to the claimed invention because they are in the same field of endeavor being AI Explanation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the time window explanation system of Nayebi with the global explanation system of Bento. One would want to understand the model’s decision process as a whole (Bento 4.2 Global Explanations)
Regarding claim 11, Nayebi in view of Song, Raykar and Bento teaches claim 10 as outlined above. Claim 11 recites similar limitations corresponding to claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding claim 12, Nayebi in view of Song, and Bento teaches claim 11 as outlined above. Claim 12 recites similar limitations corresponding to claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding claim 13, Nayebi in view of Song, Raykar and Bento teaches claim 10 as outlined above. Claim 13 recites similar limitations corresponding to claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding claim 14, Nayebi in view of Song, Raykar and Bento teaches claim 10 as outlined above. Claim 14 recites similar limitations corresponding to claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Regarding claim 15, Nayebi in view of Song, Raykar and Bento teaches claim 10 as outlined above. Claim 15 recites similar limitations corresponding to claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Regarding claim 16, Nayebi in view of Song, Raykar and Bento teaches claim 15 as outlined above. Claim 16 recites similar limitations corresponding to claim 7 and is rejected for similar reasons as claim 7 using similar teachings and rationale.
Regarding claim 17, Nayebi in view of Song, Raykar and Bento teaches claim 15 as outlined above. Claim 17 recites similar limitations corresponding to claim 8 and is rejected for similar reasons as claim 8 using similar teachings and rationale.
Regarding claim 19, Nayebi teaches:
receive, via an application programming interface, at least one input; (Algorithm 1 on pages 8-9 shows input sequence).
temporally segment the at least one input to generate a finite set of time windows; (Page 8, 3.3.1 Stationary WindowSHAP “In this approach, the time-axis is segmented into fixed-length windows.”)
train, by the at least one processor, at least one model for each time window of the set of time windows, wherein the at least one model includes a neural network that uses interconnected nodes in a layered structure for transforming the at least one input for predictive analytics, and wherein the at least one model is trained using raw data from each time window that is included in the at least one input; (Page 13. 3.5 Data sources “To test the model-agnostic explanation methods (e.g., WindowSHAP), we used three distinct clinical time-series data sets to develop and train three different deep learning prediction models.” And Page 15 mentions using recurrent neural networks and table 3 shows the information (number of layer, neurons etc.) about the recurrent neural networks.)
generate, by the at least one processor using each of the at least one trained model, at least one prediction for a target time that is indicated by a timestamp included in the at least one input; (Page 13. 3.5 Data sources ” To test the model-agnostic explanation methods (e.g., WindowSHAP), we used three distinct clinical time-series data sets to develop and train three different deep learning prediction models.” Prediction model is making prediction for the windows)
Nayebi does not teach:
A non-transitory computer readable storage medium storing instructions for providing explanations of predictive outputs, the storage medium comprising executable code which, when executed by a processor, causes the processor to: Prediction model for each time window
Model is trained using raw data
Input data includes a timestamp
generate, by the at least one processor, a set of common background data for each of the at least one time window based on the at least one input;
determine, by the at least one processor, at least one respective mode explanation for each time window based on the corresponding set of common background data, the corresponding at least one trained model, and the corresponding at least one prediction wherein each respective mode explanation of the at least one respective mode explanation includes at least one respective feature attribution that is generated for each time window with respect to a respective corresponding background data distribution of the generated set of common background data associated with the corresponding respective time window
determine, by the at least one processor, at least one reconciled explanation for a target prediction that corresponds to the target time based on the at least one input and the at least one respective mode explanation; and
Output, by the at least one processor, the at least one reconciled explanation from the at least one model.
Song teaches:
Prediction model for each time window (Col 5 lines 31-35 “To train the time-restricted entity analysis model using the segmented training data sets at block 106, the computer system can train multiple different time-segmented machine learning models corresponding to the multiple segments of the training data set.”)
Model is trained using raw data from each time window(Col 5 lines 37-50 “The computer system uses the first training data set to train a first random forest model, which is configured to use attributes of a first set of entities to predict whether the target action will occur with respect to entities in the first set during the time period between t1 and t2. A second set of the segmented training data includes training data with timestamps falling between time t3 and time t4. Like the first training data set, the computer system uses the second training data set to train a second random forest model that, when trained, is configured to use attributes of a second set of entities to predict whether the target action will occur with respect to the entities in the second set during the time period between t3 and t4.” Song also mentions the model can be a neural network)
Input data includes a timestamp (Col 5 lines 35-37 “For example, a first set of the segmented training data includes training data with timestamps falling between time t1 and time t2.”)
Nayebi and Song are considered analogous art to the claimed invention because they are in the same field of endeavor being machine learning. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine one model for each time period of Song with the explanation system of Nayebi and include the background data in the explanation determination. One would be motivated to do this to account for time-restricted variability (Song background).
Raykar teaches:
A non-transitory computer readable storage medium storing instructions for providing explanations of predictive outputs, the storage medium comprising executable code which, when executed by a processor, causes the processor to: ([0070] “These computer readable program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.”)
generate, by the at least one processor, a set of common background data for each of the at least one time window based on the at least one input; ([0040] “In operation 810, the model component 130 generates a surrogate data set. The surrogate data set may be generated by backtesting one or more time series forecasting predictions of the set of time series forecasting predictions.”)
Nayebi, Song and Raykar are considered analogous art to the claimed invention because they are in the same field of endeavor being AI Explanation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the background data of Raykar with the explanation system of Nayebi and include the background data in the explanation determination. One would want to do this to create better explanations (Raykar background).
Bento teaches:
determine, by the at least one processor, at least one respective mode explanation for each time window based on the corresponding set of common background data, the corresponding at least one trained model, and the corresponding at least one prediction wherein each respective mode explanation of the at least one respective mode explanation includes at least one respective feature attribution that is generated for each time window with respect to a respective corresponding background data distribution of the generated set of common background data associated with the corresponding respective time window (Section 3 on page 3 teaches their explanation method. More specifically equation 7 is generating explanation features which uses input data X and background data B. As mentioned above Song teaches the model for each time window.)
determine, by the at least one processor, at least one reconciled explanation for a target prediction that corresponds to the target time based on the at least one input and the at least one respective mode explanation. (Page 7 Section 4.2 Global Explanations “We compute global explanations by applying TimeSHAP to all sequences that contained a positive prediction, and explaining the first positive prediction of each sequence (hence referred as 𝑡 = 0). We use 𝜂 = 0.025 as the tolerance of the temporal coalition pruning algorithm.”)
Output, by the at least one processor, the at least one reconciled explanation from the at least one model. (Page 6 section 4.2 “Supplying a data scientist with global explanations provides an overview of the model’s decision process, revealing which features, or, in the case of TimeSHAP, events, are relevant to the model and which ones are not. TimeSHAP provides this type of explanations via aggregations and visualizations of local explanations.” Giving the explanations to a data scientist implies they are outputted).
Nayebi, Song, Raykar, and Bento are considered analogous art to the claimed invention because they are in the same field of endeavor being AI Explanation systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the time window explanation system of Nayebi with the global explanation system of Bento. One would want to understand the model’s decision process as a whole (Bento 4.2 Global Explanations)
Regarding claim 20, Nayebi in view of Song, Raykar and Bento teaches claim 19 as outlined above. Claim 20 recites similar limitations corresponding to claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL P GRUSZKA whose telephone number is (571)272-5259. The examiner can normally be reached M-F 9:00 AM - 6:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL GRUSZKA/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights