Detailed Action
This action is in response to the application filed 10/04/2022, in which:
Claims 1, 11, and 20 are the independent claims.
Claims 1-20 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/20/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 1:
Claim 1 recites a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 1 further recites the computer-implemented method for detecting positivity violations within a dataset, the method comprising:
generating … a plurality of propensity scores based on observational data associated with a group of entities, wherein, for each entity included in the group of entities, the observational data includes a plurality of attribute values associated with the entity, and wherein the group of entities comprises a subset of first entities that received a treatment and a subset of second entities that did not receive the treatment (a human being can mentally apply evaluation to apply a mathematical relationship between variables and/or numbers using a mathematical formula/equations to generate propensity scores based on specific associations and restrictions)
analyzing the plurality of propensity scores to identify one or more potential positivity violations (a human being can mentally apply evaluation to analyze propensity scores to identify potential positivity violations)
determining, based on the trained first decision tree, a first positivity violation comprising a first combination of attribute values that is associated with at least one entity included in the subset of first entities and is not associated with any entity included in the subset of second entities (a human being can mentally apply evaluation to determine positivity violations containing specific attribute values with specific association restrictions)
Claim 1 thus recites an abstract idea (that falls into the “mathematical concepts” or “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the additional elements consist of:
… using a trained machine learning model … (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
performing one or more training operations on the observational data based on the one or more potential positivity violations to generate a first trained decision tree associated with the one or more potential positivity violations (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself. Additional elements a and b are merely applying the abstract idea on a computer (MPEP 2106.05(f)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claim 2:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 2 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 2 further recites the method comprising of … determine a likelihood that the entity received the treatment (a human being can mentally apply evaluation to determine a likelihood that an entity received treatment). Claim 2 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the sole additional element consists of wherein the trained machine learning model is trained to receive one or more attribute values associated with an entity … (which is insignificant extra-solution activity of data gathering, by MPEP 2106.05(g)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the new sole additional element recited, alone or in combination, does not provide significantly more than the abstract idea itself. The additional element falls within MPEP 2106.05(d) as well-understood, routine and conventional activities of receiving or transmitting data over a network (MPEP 2106.05(d)(II): buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).. Thus, the claim is subject-matter ineligible.
Regarding Claim 3:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 3 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 3 further recites the method comprising of wherein analyzing the plurality of propensity scores comprises dividing the plurality of propensity scores into a first subset of propensity scores associated with the subset of first entities and a second subset of propensity scores associated with the subset of second entities (a human being can mentally apply evaluation to divide the scores into different subsets due to specific associations with entities). Claim 3 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 4 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 4 further recites the method comprising of:
wherein analyzing the plurality of propensity scores comprises: generating a plurality of histogram bins based on the plurality of propensity scores (a human being can mentally apply evaluation to generate a histogram bins based on propensity scores)
identifying at least one histogram bin that includes one or more propensity scores associated with the subset of first entities and does not include one or more propensity scores associated with the subset of second entities (a human being can mentally apply evaluation identify at least one histogram bin with specific propensity scores being associated/not-associated with specific subsets of entities)
Claim 4 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 5 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 5 further recites the method comprising of:
wherein analyzing the plurality of propensity scores comprises: generating a plurality of histogram bins based on the plurality of propensity scores (a human being can mentally apply evaluation to generate histogram bins based on propensity scores)
identifying at least one histogram bin that includes one or more propensity scores associated with the subset of second entities and does not include one or more propensity scores associated with the subset of first entities (a human being can mentally apply evaluation to identify at least one histogram bin with specific propensity scores being associated/not-associated with specific subsets of entities)
Claim 5 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 6 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 6 further recites the method comprising of … determine a significance associated with each potential positivity violation included in the one or more potential positivity violations (a human being can mentally apply evaluation to perform a statistical analysis operation to determine a significance associated with each violation violations with specific violations). Claim 6 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the additional elements consist of:
wherein performing one or more training operations on the observational data is further based on the significance determined for each potential positivity violation included in the one or more potential positivity violations (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
performing one or more statistical analysis operations on the one or more potential positivity violations to … (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the new sole additional element recited, alone or in combination, does not provide significantly more than the abstract idea itself. The additional element is merely applying the abstract idea on a computer (MPEP 2106.05(f)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claim 7:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 7 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 7 does not recite any additional abstract ideas and only inherits the abstract ideas from Claim 1. Claim 7 thus recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the sole additional element consists of wherein each node included in the first decision tree corresponds to a different attribute included in the observational data and is associated with a subset of observational data that includes one or more attribute values for the corresponding attribute (which is restricting the abstract idea to a Particular Technological Environment, by MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the new sole additional element recited, alone or in combination, does not provide significantly more than the abstract idea itself. The additional element is only restricting the abstract idea to a Particular Technological Environment (MPEP 2106.05(h)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 8 recites the method of Claim 7. Claim 7 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 8 further recites the method comprising of:
wherein performing the one or more training operations comprises: determining, for a first node included in the first decision tree, that a number of data points that are associated with the first node and correspond to the one or more potential positivity violations satisfies a threshold level (a human being can mentally apply evaluation to determine that a number of data points are associated with a first node in a first decision tree which also correspond with potential positivity violations that satisfy a specific threshold)
in response to determining that the number of data points satisfies the threshold level, selecting the first node as a leaf node of the first decision tree (a human being can mentally apply evaluation to select the first node a leaf node of the first decision tree)
Claim 8 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 9 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 9 does not recite any additional abstract ideas and only inherits the abstract ideas from Claim 1. Claim 9 thus recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the sole additional element consists of causing a visual representation of the first positivity violation to be displayed to a user via a graphical user interface (which is insignificant extra-solution activity of data display or output, by MPEP 2106.05(g)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the new sole additional element recited, alone or in combination, does not provide significantly more than the abstract idea itself. The additional element falls within MPEP 2106.05(d) as well-understood, routine and conventional activities of presenting offers and gathering statistics (MPEP 2106.05(d)(II)).Thus, the claim is subject-matter ineligible.
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 10 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 10 further recites the method comprising of modifying the observational data based on the first positivity violation to generate a modified set of observational data that does not include the first positivity violation (a human being can mentally apply evaluation to generate histogram bins based on propensity scores). Claim 10 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claims 11 and 12
Claims 11 and 12 incorporate substantively all the limitations of Claims 1 and 2 in a non-transitory computer-readable media (thus a manufacture) and further recites a new additional elements including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of (these claim limitations appear to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f)) and does not appear to integrate the abstract idea into a particular application; thus, the claim is subject-matter ineligible as it does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself); thus, Claims 11 and 12 are rejected for reasons set forth in the rejection of Claims 1 and 2, respectively.
Regarding Claim 13:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 13 recites the non-transitory computer-readable media of Claim 11. Claim 11 is a non-transitory computer-readable media, thus a manufacture, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 13 further recites the non-transitory computer-readable media comprising of:
wherein analyzing the plurality of propensity scores comprises: generating a first propensity score distribution based on a first subset of propensity scores associated with the subset of first entities and a second propensity score distribution based on a second subset of propensity scores associated with the subset of second entities (a human being can mentally apply evaluation to generate propensity score distributions based on specific subset of scores with specific associations)
comparing the first propensity score distribution with the second propensity score distribution (a human being can mentally apply evaluation to compare propensity score distributions)
Claim 13 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 14:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 14 recites the non-transitory computer-readable media of Claim 11. Claim 11 is a non-transitory computer-readable media, thus a manufacture, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 14 further recites the non-transitory computer-readable media comprising of:
wherein analyzing the plurality of propensity scores comprises: generating a plurality of histogram bins based on the plurality of propensity scores (a human being can mentally apply evaluation to generate histogram bins based on propensity scores)
for each histogram bin included in the plurality of histogram bins: determining a first number of propensity scores included in the histogram bin that correspond to the subset of first entities and a second number of propensity scores included in the histogram bin that correspond to the subset of second entities (a human being can mentally apply evaluation to determine propensity scores including specific histogram bin correspondences)
comparing the first number of propensity scores and the second number of propensity scores to determine whether the histogram bin includes a positivity violation (a human being can mentally apply evaluation to compare propensity scores to determine if a histogram bin comprises a positivity violation)
Claim 14 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 15:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 15 recites the non-transitory computer-readable media of Claim 11. Claim 11 is a non-transitory computer-readable media, thus a manufacture, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 15 further recites the non-transitory computer-readable media comprising of generating, for each data point included in the observational data, a corresponding label indicating whether the data point is associated with a positivity violation based on the one or more potential positivity violations (a human being can mentally apply evaluation to generate a corresponding label with specific indications for associations with violations with specific requirements for each data point in the observational data). Claim 15 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 16:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 16 recites the non-transitory computer-readable media of Claim 11. Claim 11 is a non-transitory computer-readable media, thus a manufacture, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 16 further recites the non-transitory computer-readable media comprising of wherein the first decision tree is trained to identify one or more attribute values included in the observational data that are associated with the one or more potential positivity violations (a human being can mentally apply evaluation to identify attribute values in observational data with specific potential positivity violation associations). Claim 16 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 17:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 17 recites the non-transitory computer-readable media of Claim 11. Claim 11 is a non-transitory computer-readable media, thus a manufacture, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 17 further recites the non-transitory computer-readable media comprising of pruning the first decision tree based on a percentage of data points included in a first node that correspond to the one or more potential positivity violations (a human being can mentally apply evaluation to prune a decision tree based on a percentage that correspond to specific potential positivity violations). Claim 17 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the sole additional element consists of wherein each node of the first decision tree is associated with one or more data points included in the observational data, and wherein performing the one or more training operations comprises (which is restricting the abstract idea to a Particular Technological Environment, by MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the new sole additional element recited, alone or in combination, does not provide significantly more than the abstract idea itself. The additional element is only restricting the abstract idea to a Particular Technological Environment (MPEP 2106.05(h)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claim 18:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 18 recites the non-transitory computer-readable media of Claim 11. Claim 11 is a non-transitory computer-readable media, thus a manufacture, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 18 further recites the non-transitory computer-readable media comprising of pruning the first decision tree based on a percentage of data points that correspond to the one or more potential positivity violations that are included in a first node (a human being can mentally apply evaluation to prune a decision tree based on a percentage that correspond to specific potential positivity violations ). Claim 18 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the sole additional element consists wherein each node of the first decision tree is associated with one or more data points included in the observational data, and wherein performing the one or more training operations comprises (which is restricting the abstract idea to a Particular Technological Environment, by MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the new sole additional element recited, alone or in combination, does not provide significantly more than the abstract idea itself. The additional element is only restricting the abstract idea to a Particular Technological Environment (MPEP 2106.05(h)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claim 19:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 19 recites the non-transitory computer-readable media of Claim 11. Claim 11 is a non-transitory computer-readable media, thus a manufacture, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 19 further recites the non-transitory computer-readable media comprising of determining, based on the trained second decision tree, a second positivity violation comprising a second combination of attribute values that is associated with at least one entity included in the subset of second entities and is not associated with any entity included in the subset of first entities (a human being can mentally apply evaluation to determine positivity violations containing specific attribute values with specific association restrictions). Claim 19 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the additional elements consist of:
wherein the first trained decision tree is associated with the subset of first entities (which is restricting the abstract idea to a Particular Technological Environment, by MPEP 2106.05(h)).
wherein the steps further comprise: performing the one or more training operations on the observational data based on the one or more potential positivity violations to generate a second trained decision tree associated with the one or more potential positivity violations and the subset of second entities (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself. Additional element a is only restricting the abstract idea to a Particular Technological Environment (MPEP 2106.05(h)) which cannot provide significantly more. Additional element b is merely applying the abstract idea on a computer (MPEP 2106.05(f)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claim 20:
Claim 20 incorporate substantively all the limitations of Claim 1 in an system (thus a machine) and further recites a new additional elements one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, perform the steps of (these claim limitations appear to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f)) and does not appear to integrate the abstract idea into a particular application; thus, the claim is subject-matter ineligible as it does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself); thus, Claim 20 is rejected for reasons set forth in the rejection of Claim 1.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 7-9, and 11-16, and 19-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Karavani et al., “A discriminative approach for finding and characterizing positivity violations using decision trees”.
Regarding Claim 1:
A computer-implemented method (Karavani, Page 5, Paragraph 6, “We used Python 3.6 together with scikit-learn [19] for the main analysis and tree construction”. The examiner interprets utilizing python for the main analysis as a computer-implemented method to process the algorithms and finalize determinations) for detecting positivity violations within a dataset, the method comprising: (Karavani, Page 9, Paragraph 6, “… We present a scalable approach for detecting if positivity violations occur in a dataset and characterizing the subspaces these violations originate from”.)
generating, using a trained machine learning model, (Karavani, Page 3, Paragraph 5-7, “… we apply a machine learning model to detect positivity violations … We train a decision tree classifier using the treatment assignment as the target variable.”; Page 10, Paragraph 5, “… It allows an easy overview of the scale and severity of violations in a dataset …”.) a plurality of propensity scores (Karavani, Page 14, Figure S2; Page 3, Paragraph 2, “… it also has two main drawbacks. Firstly, propensity scores can be viewed as a dimensionality reduction from the covariate space onto a single number, and almost always bear some information loss … Secondly, once a violation is detected, it is usually hard to characterize the covariate subspace … as it depends on the interpretability of the model used to obtain those propensity scores … In this work, we introduce a method to identify whether violations of positivity exist in a given dataset. While current methods seem to trade-off between the two, our method can both scale for complex high-dimensional spaces …”. Figure S2 shows a plurality of propensity scores (the examiner interprets propensity scores as an estimated probability of an entity to receive a treatment using statistics) within a score distribution between the treatments groups. Karavani mentions the related work of Marginally comparing covariates & Propensity based methods; where both methods are incorporated into Karavani’s alternate score generation which is a type of propensity score as it a treatment probability score via measuring leaf depth for analysis (shown in Figure 2B)) based on observational data associated with a group of entities, (Karavani, Page 7, Figure 2; Page 5 Paragraph 5, “Figure 2, shows a real world example of an observational study about the …”. Figure 2 depicts the observational data associated with a group of entities (NHEFS study) where each point corresponds to one entity which is a subset of a treatment group type (illustrated via the coloring in the figure)) wherein, for each entity included in the group of entities, the observational data includes a plurality of attribute values associated with the entity (Karavani, Page 3, Paragraph 4, “… lack of common support can be detected by measuring large statistical distance between the covariate distribution of treated and untreated groups”; Page 1, Abstract, “… Most common methods to date are insufficient for discovering nonpositivity, as they do not scale for modern high-dimensional covariate spaces, or they cannot pinpoint the subpopulation violating positivity. To overcome these issues, we suggest to harness decision trees for detecting violations”. The covariate distribution implies a plurality of attribute/feature values over all entities; thus, for each entity included in the group of entities. The method taught by Karavani is to overcome a modern high-dimensional covariate spaces which is interpreted by the examiner as each entity comprising multiple features due to high-dimensionality spaces; thus, for each observational entity there being a plurality of attribute values being associated with the entity to determine positivity violations), and wherein the group of entities comprises a subset of first entities that received a treatment and a subset of second entities that did not receive the treatment; (Karavani, Page 3, Paragraph 4, “…measuring large statistical distance between the covariate distribution of treated and untreated groups”; Page 7, Figure 2. Figure 2 shows the different subset of entities (first entities and second entities via coloring of the treatment type groups)).
analyzing the plurality of propensity scores to identify one or more potential positivity violations;
(Karavani, Page 14, Figure S2: B; Page 3, Paragraph 3, “… In this work, we introduce a method to identify whether violations of positivity exist in a given dataset …”. Figure S2: B shows the classic approach of propensity score analysis to identify one or more potential positivity violations; where, Figure 2B shows the application of Karavani’s method to analyze the treatment probability scores visually (which is a type of propensity score) to identity one or more potential positivity violations).
performing one or more training operations on the observational data based on the one or more potential positivity violations to generate a first trained decision tree associated with the one or more potential positivity violations; and
(Karavani, Page 7, Figure 2; Page 5 Paragraph 5, “Figure 2, shows a real world example of an observational study about the …”; Page 3, Paragraph 7, “… We train a decision tree classifier using the treatment assignment as the target variable. While being constructed, the tree tries to maximize group discrimination by finding regions in the covariate space populated exclusively by only one treatment group. These covariate subspaces are, by definition, violating the positivity assumption”. Figure 2 shows the visual analysis of the performing of the training operations on the observational study data which is based on the first and second subsets of entities to generate a decision tree; where the decision tree is shown in two different visuals within Figure 2A and 2B for reviewing and analyzing the potential positivity violations).
determining, based on the trained first decision tree, a first positivity violation comprising a first combination of attribute values that is associated with at least one entity included in the subset of first entities and is not associated with any entity included in the subset of second entities.
(Karavani, Page 7, Paragraph 2, “Probability is a measure for leaves, which we model using hypergeometric distribution. Given two possible interventions A = 0, 1, we can denote the number of samples from each group in the entire dataset as N0,N1 accordingly. For a given leaf (or a covariate subspace), we can have n0 untreated and n1 treated patients belong to it. These samples (n0,n1) originate from the root samples (N0,N1). Therefore, we can model the leaf as a sample from the population (root) the following way: - N = N0 + N1 the population size, - K = N1 the number of successes in the population, - n = n0 + n1 the number of draws, - k = n1 the number of observed successes. And the probability of a leaf is the probability mass function Pr[X = k] given X ~ Hypergeometric(N, k, n). The smaller the probability the rarer it is to get that subspace by chance”. The model is used to determine probability via measuring leaves within a trained decision tree (where each leaf represent a subset of entities (N0 for untreated and N1 for treated; where n0 and n1 are units within the entity/leaf with similar features/attribute values, respectively)) which is trained on the observational data and the and is best depicted in Figure 1 to denote determining positivity violations and determining whether a combination of attribute values associated with one entity within the first entities subset (treated group) is not associated with any entity in the second entities subset (untreated group)).
Regarding Claim 2:
Karavani teaches the method of Claim 1 and further teaches:
wherein the trained machine learning model is trained to receive one or more attribute values associated with an entity and determine a likelihood that the entity received the treatment.
(Karavani, Page 1, Paragraph 1, “Causal inference … estimating an effect of an intervention is done by comparing two groups, one that received the intervention, and another that received a different intervention … ”; Page 9, Paragraph 6, “Positivity is a necessary assumption for causal inference, ensuring treatment groups are comparable, so causal conclusions could be drawn … We present a scalable approach for detecting if positivity violations occur in a dataset and characterizing the subspaces these violations originate from”; Page 8, [3.2.4: Soft violations]. The soft violations (imbalance) denotes that the data is not incorrect but the attribute values for probability of intervention/treatment (likelihood) is not exactly 0 or 1; thus, a violation for causally inferring. The method determines the entity receiving treatment correctly but the model was unable to determine comparable entities within the opposite treatment group (n0 >> n1 or n0 >> n1)).
Regarding Claim 3:
Karavani teaches the method of Claim 1 and further teaches:
wherein analyzing the plurality of propensity scores comprises dividing the plurality of propensity scores into a first subset of propensity scores associated with the subset of first entities and a second subset of propensity scores associated with the subset of second entities.
(Karavani, Page 6-7, Figures 1 & 2; Page 14, Figure S2. Figures 1, 2, and S2 all denote the division of the propensity scores into the two different subsets (where the first and second subsets are denoted in different colors based on treatment group association)).
Regarding Claim 4:
Karavani teaches the method of Claim 1 and further teaches:
wherein analyzing the plurality of propensity scores comprises:
generating a plurality of histogram bins based on the plurality of propensity scores; and
(Karavani, Page 6-7, Figures 1 & 2. Figure 1B and 2B show the application of the method to generate histogram bins via binning (as the plots show the intervals to consider overlap) the received treatment probability scores (a type of propensity score)).
identifying at least one histogram bin that includes one or more propensity scores associated with the subset of first entities and does not include one or more propensity scores associated with the subset of second entities.
(Karavani, Page 7, Figure 2: “… (A) contains a scatter plot … Overlap of points from group 0 (orange) and group 1 (blue) hints there are no major violations of nonpositivity … Applying our method on the original domain space and visualizing it in (B) confirms this as the rectangles are so transparent”. Figure 2 shows a visualization of how the system determines overlap of entities (data points within the scatter plot Figure 2A). The generated histogram bins/intervals within Figure 2B show the identification of histogram bins that include propensity scores associated with one entity and NOT the other subset when the bin is NOT transparent).
Regarding Claim 5:
Karavani teaches the method of Claim 1 and further teaches:
wherein analyzing the plurality of propensity scores comprises:
generating a plurality of histogram bins based on the plurality of propensity scores; and
(Karavani, Page 6-7, Figures 1 & 2. Figure 1B and 2B show the application of the method to generate histogram bins via binning (as the plots show the intervals to consider overlap) the received treatment probability scores (a type of propensity score)).
identifying at least one histogram bin that includes one or more propensity scores associated with the subset of second entities and does not include one or more propensity scores associated with the subset of first entities.
(Karavani, Page 7, Figure 2: “… (A) contains a scatter plot … Overlap of points from group 0 (orange) and group 1 (blue) hints there are no major violations of nonpositivity … Applying our method on the original domain space and visualizing it in (B) confirms this as the rectangles are so transparent”. Figure 2 shows a visualization of how the system determines overlap of entities (data points within the scatter plot Figure 2A). The generated histogram bins/intervals within Figure 2B show the identification of histogram bins that include propensity scores associated with one entity AND the other subset when the bin is transparent).
Regarding Claim 7:
Karavani teaches the method of Claim 1 and further teaches:
wherein each node included in the first decision tree corresponds to a different attribute included in the observational data and is associated with a subset of observational data that includes one or more attribute values for the corresponding attribute.
(Karavani, Page 9, Paragraph 3, “
PNG
media_image1.png
303
669
media_image1.png
Greyscale
”; Page 7, Paragraph 2, “… samples from each group in the entire dataset as N0,N1 accordingly … n0 untreated and n1 treated patients belong to it. These samples (n0,n1) originate from the root samples (N0,N1). Therefore, we can model the leaf as a sample from the population (root) the following way: - N = N0 + N1 the population size, - K = N1 the number of successes in the population, - n = n0 + n1 the number of draws, - k = n1 the number of observed successes”. Sections 3.2.2 and 3.4 teach the decision tree to contain leaves which are split into treated and untreated entity groups which contain subset of entities which share covariate attributes with one another; thus, each node (leaf) is corresponds to a different attribute as leaves/nodes are not identical and grouped based on similarity within their respective treatment type group).
Regarding Claim 8:
Karavani teaches the method of Claim 7 and further teaches:
wherein performing the one or more training operations comprises:
determining, for a first node included in the first decision tree, that a number of data points that are associated with the first node and correspond to the one or more potential positivity violations satisfies a threshold level; and
(Karavani, Page 7, Figure 2; Page 8, Paragraph 3, “… subspaces where unbalancing is simply large, i.e. the ratio n1/n0 is either very large or very small. This can be obtained by setting some small positive threshold t, for which leaves with impurity below will be flagged as violating”. Figure 2A and 2B show the scatter plots that denote the number of data points that are associated with the node that are associated with one or more potential positivity violations via overlap and color density, respectively. Thus, the examiner interprets this process as being determined as they are represented visually by apply the methods).
in response to determining that the number of data points satisfies the threshold level, selecting the first node as a leaf node of the first decision tree.
(Karavani, Page 8, Paragraph 3, “We provide … visualizations of the decision tree … to get a concise overview about how severe the violations in the dataset are. The interactive one can also further explore the leaves to obtain more details … about the subspaces they represent. … Additional color opacity is determined by the average consistency values of the samples belonging to the leaf …”; Page 7, Figure 2; Page 8, Paragraph 3, “… subspaces where unbalancing is simply large, i.e. the ratio n1/n0 is either very large or very small. This can be obtained by setting some small positive threshold t, for which leaves with impurity below will be flagged as violating”. As the method utilizes the threshold level to indicate positivity violation via color labeling within Figure 1 & 2; thus, the density based colored points denote a number of data points that satisfy a threshold level and are thus selected as each point is colored with the visual representation of the decision tree).
Regarding Claim 9:
Karavani teaches the method of Claim 1 and further teaches:
comprising causing a visual representation of the first positivity violation to be displayed to a user via a graphical user interface.
(Karavani, Page 6-7, Figures 1 & 2; Page 8, Paragraph 5, “We provide both static and interactive visualizations of the decision tree. The goal of the static visualization is to get a concise overview about how severe the violations in the dataset are” The figures and explanations of the visualizations within Karavani are interpreted by examiner to be displayed to a user via a graphical interface).
Regarding Claims 11 and 12:
Claims 11 and 12 incorporate substantively all the limitations of Claims 1 and 2 in a non-transitory computer-readable media (thus a manufacture) and further recites a new additional elements including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of (Karavani, Page 5, Paragraph 6, “We used Python 3.6 together with scikit-learn [19] for the main analysis and tree construction”. The method of Karavani utilizes Python for the main analysis which implies that the analysis is done on a computing device, in which a processor and computer-readable media are inherent); thus, Claims 11 and 12 are rejected for reasons set forth in the rejection of Claims 1 and 2, respectively.
Regarding Claim 13:
Karavani teaches the non-transitory computer-readable media of Claim 11 and further teaches:
wherein analyzing the plurality of propensity scores comprises:
generating a first propensity score distribution based on a first subset of propensity scores associated with the subset of first entities and a second propensity score distribution based on a second subset of propensity scores associated with the subset of second entities; and comparing the first propensity score distribution with the second propensity score distribution.
(Karavani, Pages 6-14, Figures 1, 2, and S2. Figures 1, 2, and S2 show the comparisons of the different propensity score distributions within different visualizations for identifications and analysis; thus, the visualization is comparing the two propensity score distributions which are generated based on first and second subsets of propensity scores (treated and untreated groups)).
Regarding Claim 14:
Karavani teaches the non-transitory computer-readable media of Claim 11 and further teaches:
wherein analyzing the plurality of propensity scores comprises:
generating a plurality of histogram bins based on the plurality of propensity scores;
(Karavani, Page 13, Figures S1. Figure S1 shows a snapshot of Figure 1B’s plot within the visualization to depict the histogram bins generated from the propensity scores).
for each histogram bin included in the plurality of histogram bins: determining a first number of propensity scores included in the histogram bin that correspond to the subset of first entities and a second number of propensity scores included in the histogram bin that correspond to the subset of second entities; and
(Karavani, Page 13, Figures S1. Figure S1 shows a snapshot of Figure 1B’s plot within the visualization to depict the histogram bins generated from the propensity scores (‘probability’ within the hovering window of the example leaf where a leaf is a subset of the entities). The differentiation between the first and second subsets of entities is done via coloring and the color opacity is based on the average consistency values which is treatment probability (a type of propensity score)).
comparing the first number of propensity scores and the second number of propensity scores to determine whether the histogram bin includes a positivity violation.
(Karavani, Page 14, Page 7, Figure 2; Figure S2. Figure 2 shows the depiction of overlap of points and histogram bins to determine whether a histogram bin includes a positivity violation. S2 shows the comparison within the classical propensity based distribution plots while 2B shows the histogram bins comparison).
Regarding Claim 15:
Karavani teaches the non-transitory computer-readable media of Claim 11 and further teaches:
comprising generating, for each data point included in the observational data, a corresponding label indicating whether the data point is associated with a positivity violation based on the one or more potential positivity violations.
(Karavani, Page 8, Paragraph 3, “We provide … visualizations of the decision tree … to get a concise overview about how severe the violations in the dataset are. The interactive one can also further explore the leaves to obtain more details … about the subspaces they represent. … Additional color opacity is determined by the average consistency values of the samples belonging to the leaf …”; Pages 6-7, Figures 1-2. As the method utilizes the threshold level to indicate positivity violation via color labeling within Figure 1 & 2; thus, the density based colored points denote the probability that the data points is associated with one or more potential positivity violations).
Regarding Claim 16:
Karavani teaches the non-transitory computer-readable media of Claim 11 and further teaches:
wherein the first decision tree is trained to identify one or more attribute values included in the observational data that are associated with the one or more potential positivity violations.
(Karavani, Page 3, Paragraph 7-9, We harness this property for the detection of positivity violations. We train a decision tree classifier using the treatment assignment as the target variable. While being constructed, the tree tries to maximize group discrimination by finding regions in the covariate space populated exclusively by only one treatment group. These covariate subspaces are, by definition, violating the positivity assumption. To characterize the subspaces violating positivity, we utilize another useful property of decision trees which is their interpretability”. The decision tree classifier is trained on treatment assignment on the target variable; where the tree is identify the covariate subspaces (interpreted as the feature/attribute values subspaces) which contain potential (probable) positivity violations of treatment).
Regarding Claim 19:
Karavani teaches the non-transitory computer-readable media of Claim 11 and further teaches:
wherein the first trained decision tree is associated with the subset of first entities, and wherein the steps further comprise:
performing the one or more training operations on the observational data based on the one or more potential positivity violations to generate a second trained decision tree associated with the one or more potential positivity violations and the subset of second entities; and
(Karavani, Page 7, Figure 2; Page 5 Paragraph 5, “Figure 2, shows a real world example of an observational study about the …”; Page 3, Paragraph 7, “… We train a decision tree classifier using the treatment assignment as the target variable. While being constructed, the tree tries to maximize group discrimination by finding regions in the covariate space populated exclusively by only one treatment group. These covariate subspaces are, by definition, violating the positivity assumption”. Figure 2 shows the visual analysis of the performing of the training operations on the observational study data which is based on the first and second subsets of entities to generate a decision tree; where the decision tree is shown in two different visuals within Figure 2A and 2B for reviewing and analyzing the potential positivity violations).
determining, based on the trained second decision tree, a second positivity violation comprising a second combination of attribute values that is associated with at least one entity included in the subset of second entities and is not associated with any entity included in the subset of first entities.
(Karavani, Page 7, Paragraph 2, “Probability is a measure for leaves, which we model using hypergeometric distribution. Given two possible interventions A = 0, 1, we can denote the number of samples from each group in the entire dataset as N0,N1 accordingly. For a given leaf (or a covariate subspace), we can have n0 untreated and n1 treated patients belong to it. These samples (n0,n1) originate from the root samples (N0,N1). Therefore, we can model the leaf as a sample from the population (root) the following way: - N = N0 + N1 the population size, - K = N1 the number of successes in the population, - n = n0 + n1 the number of draws, - k = n1 the number of observed successes. And the probability of a leaf is the probability mass function Pr[X = k] given X ~ Hypergeometric(N, k, n). The smaller the probability the rarer it is to get that subspace by chance”. The model is used to determine probability via measuring leaves within a trained decision tree (where each leaf represent a subset of entities (N0 for untreated and N1 for treated; where n0 and n1 are units within the entity/leaf with similar features/attribute values, respectively)) which is trained on the observational data and the and is best depicted in Figure 1 to denote determining positivity violations and determining whether a combination of attribute values associated with one entity within the first entities subset (treated group) is not associated with any entity in the second entities subset (untreated group)).
Regarding Claim 20:
Claim 20 incorporate substantively all the limitations of Claim 1 in an system (thus a machine) and further recites a new additional elements one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, perform the steps of (Karavani, Page 5, Paragraph 6, “We used Python 3.6 together with scikit-learn [19] for the main analysis and tree construction”. The method of Karavani utilizes Python for the main analysis which implies that the analysis is done on a computing device, in which a processor and memories are inherent to execute the code for algorithm 1); thus, Claim 20 is rejected for reasons set forth in the rejection of Claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 6 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Karavani et al., “A discriminative approach for finding and characterizing positivity violations using decision trees”, in view of Bak et al., US 11,354,297 B2.
Regarding Claim 6:
Karavani teaches the method of Claim 1 and further teaches:
performing one or more statistical analysis operations on the one or more potential positivity violations to determine a significance associated with each potential positivity violation included in the one or more potential positivity violations; and
(Karavani, Page, 2, Paragraph 4, “… to find violations of positivity … can be broadly cast into … based on statistical tests comparing the two groups on each covariate marginally, and another based on comparing propensity scores across these two groups …”; Page 3, Paragraph 4, “As its name suggests, lack of common support can be detected by measuring large statistical distance between the covariate distribution of treated and untreated groups. This comparison of distributions is known in statistics as two-sample test … ”; Page 6-8, [Significance Testing]. The significance testing section within Karavani highlights the performing of the statistical analysis operations via implementing the probability mass function Pr[X=k] which determine the probability; where the smaller the probability the more probable/significant positivity violation).
Karavani does teach performing one or more training operations on the observational data but does not explicitly teach:
wherein performing one or more training operations on the observational data is further based on the significance determined for each potential positivity violation included in the one or more potential positivity violations.
However, Bak teaches:
wherein performing one or more training operations on the observational data is further based on the significance determined for each potential positivity violation included in the one or more potential positivity violations.
(Bak, Fig. 1: 110 -> 112 -> 102; Fig. 5: 512 -> 514. Figures 1 and 5 denote the training operations which includes generating positivity problem insights which are used to refine the cohort by making the model more precise based on the positivity problem insight (which is interpreted by the examiner as the significance of the positivity violation (problem). Thus, the training operations on the observational data is further based on the significance determined for each potential positivity violation included in the positivity violations)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to use the computer-implemented method of Karavani for determining and detecting positivity violations within a dataset with the explicit further training operations of Bak to improve the machine learning algorithm/model via cohort refinement (Bak, Column 3, Lines 32-44, “… enable efficient identification and characterization of sub-populations in one group that have no counterpart in another group to be compared. In this manner, a cohort refinement may automatically be executed. For example, once the positivity violations are identified, the subpopulation with combinations of features causing the positivity violation may be removed. The refined cohort may then be used to train a machine learning algorithm for detecting causality in data sets. By removing positivity violations from the cohort used to train the machine learning algorithm, the accuracy of the machine learning algorithm of detecting causality may be improved”).
Regarding Claim 10:
Karavani teaches the method of Claim 1 and teaches the observational data and positivity violations but does not explicitly teach:
comprising modifying the observational data based on the first positivity violation to generate a modified set of observational data that does not include the first positivity violation.
However, Bak teaches:
comprising modifying the observational data based on the first positivity violation to generate a modified set of observational data that does not include the first positivity violation.
(Bak, Figure 5: 514; Column 3, Lines 39-44, “…The refined cohort may then be used to train a machine learning algorithm for detecting causality in data sets. By removing positivity violations from the cohort used to train the machine learning algorithm, the accuracy of the machine learning algorithm of detecting causality may be improved”. Bak teaches refining the cohort by automatically removing positivity violations (Fig. 5: 514) from the cohort (observational data containing a plurality of entities) that is used for training; thus, interpreted by the examiner as modifying the observational data based on positivity violations to generate a modified observational dataset that does not include the positivity violations).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to use the computer-implemented method of Karavani for determining and detecting positivity violations within an observational dataset with the explicit further training operations of refining from Bak to improve the machine learning algorithm/model via cohort refinement (Bak, Column 3, Lines 32-44, “… enable efficient identification and characterization of sub-populations in one group that have no counterpart in another group to be compared. In this manner, a cohort refinement may automatically be executed. For example, once the positivity violations are identified, the subpopulation with combinations of features causing the positivity violation may be removed. The refined cohort may then be used to train a machine learning algorithm for detecting causality in data sets. By removing positivity violations from the cohort used to train the machine learning algorithm, the accuracy of the machine learning algorithm of detecting causality may be improved”).
Claims 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Karavani et al., “A discriminative approach for finding and characterizing positivity violations using decision trees”, in view of Liu et al., “A Robust Decision Tree Algorithm for Imbalanced Data Sets”.
Regarding Claim 17:
Karavani teaches the non-transitory computer-readable media of Claim 11 and further teaches:
wherein each node of the first decision tree is associated with one or more data points included in the observational data,
(Karavani, Page 9, Paragraph 3, “
PNG
media_image1.png
303
669
media_image1.png
Greyscale
”; Page 7, Paragraph 2, “… samples from each group in the entire dataset as N0,N1 accordingly … n0 untreated and n1 treated patients belong to it. These samples (n0,n1) originate from the root samples (N0,N1). Therefore, we can model the leaf as a sample from the population (root) the following way: - N = N0 + N1 the population size, - K = N1 the number of successes in the population, - n = n0 + n1 the number of draws, - k = n1 the number of observed successes”. Sections 3.2.2 and 3.4 teach the decision tree to contain leaves which are split into treated and untreated entity groups; the leaves contain subsets of the observational data which share covariate attributes with one another; thus, each node (leaf) contains one or more data points included in the observational data). and wherein performing the one or more training operations comprises … a first node that correspond to the one or more potential positivity violations (Karavani, Page 8, Paragraph 3, “… subspaces where unbalancing is simply large, i.e. the ratio n1/n0 is either very large or very small. This can be obtained by setting some small positive threshold t, for which leaves with impurity below will be flagged as violating”.)
Karavani does teach performing one or more training operations comprising the first decision tree, potential positivity violations and even a percentage of data points but does not explicitly disclose pruning based on a percentage:
…pruning the first decision tree based on a percentage of data points included in a first node…
However, Liu teaches:
…pruning the first decision tree based on a percentage of data points included in a first node…
(Liu, Abstract, “…This allows us to immediately explain why Information Gain, like confidence, results in rules which are biased towards the majority class. To overcome this bias, we introduce a new measure, Class Confidence Proportion (CCP), which forms the … rules which are statistically significant we … prune branches of the tree which are not statistically significant … ”; Page 769, Equations 3.10-3.12, “
PNG
media_image2.png
125
369
media_image2.png
Greyscale
…
PNG
media_image3.png
58
361
media_image3.png
Greyscale
”; Page 772, Column 1, Paragraph 4, “While CCP helps to select which branch of a tree are “good” to discriminate between classes, we also want to evaluate the statistical significance of each branch. This is done by the Fisher’s exact test (FET) … Therefore, given a threshold for the p value, we can find and keep the tree branches that are statistically significant (with lower p values), and discard those tree nodes that are not”. Class Confidence Proportion (CCP) is used to prune branches of the decision tree that are not statistically significant (which is based on a percentage) and can be seen within equation 3.12 which is comprised of equations 3.10 and 3.11. Thus, the pruning of the decision tree is based on a percentage of data points included in a first node (as the CCP equation is a class confidence value calculated for branch significance while Fisher’s Exact Test (FET) is utilized to determine which nodes are for removal as they do not meet a significance threshold based on a percent)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to use the computer-implemented method of Karavani for determining and detecting positivity violations with the pruning methodology of Liu to improve accuracy, balance datasets, significant rules (Liu, Abstract, “… To generate rules which are statistically significant we design a novel and efficient top-down and bottom-up approach which uses Fisher’s exact test to prune branches of the tree which are not statistically significant. Together these two changes yield a classifier that performs statistically better than not only traditional decision trees but also trees learned from data that has been balanced by well-known sampling techniques …”; Page 776, Column 2, Paragraph 3, “We address the problem of designing a decision tree algorithm for classification which is robust against class imbalance in the data. We first explain why traditional decision tree measures, like information gain, are sensitive to class imbalance. We do that by expressing information gain in terms of the confidence of a rule. Information gain, like confidence, is biased towards the majority class. Having identified the cause of the problem, we propose a new measure, Class Confidence Proportion (CCP). Using both theoretical and geometric arguments we show that CCP is insensitive to class distribution. We then embed CCP in information gain and use the improvised measure to construct the decision tree. Using a wide array of experiments we demonstrate the effectiveness of CCP when the data sets are imbalanced. We also propose the use of Fisher exact test as a method for pruning the decision tree. Besides improving the accuracy, an added benefit of the Fisher exact test is that all the rules found are statistically significant”).
Regarding Claim 18:
Karavani teaches the non-transitory computer-readable media of Claim 11 and further teaches:
wherein each node of the first decision tree is associated with one or more data points included in the observational data, and wherein performing the one or more training operations comprises … that correspond to the one or more potential positivity violations that are included in a first node.
(Karavani, Page 8, Paragraph 3, “… subspaces where unbalancing is simply large, i.e. the ratio n1/n0 is either very large or very small. This can be obtained by setting some small positive threshold t, for which leaves with impurity below will be flagged as violating”.)
Karavani does teach performing one or more training operations comprising the first decision tree associated with observational data points and performing discussed above. However, Karavani does not explicitly disclose:
…pruning the first decision tree based on a percentage of data points …
However, Liu teaches:
…pruning the first decision tree based on a percentage of data points …
(Liu, Abstract, “…This allows us to immediately explain why Information Gain, like confidence, results in rules which are biased towards the majority class. To overcome this bias, we introduce a new measure, Class Confidence Proportion (CCP), which forms the … rules which are statistically significant we … prune branches of the tree which are not statistically significant … ”; Page 769, Equations 3.10-3.12, “
PNG
media_image2.png
125
369
media_image2.png
Greyscale
…
PNG
media_image3.png
58
361
media_image3.png
Greyscale
”; Page 772, Column 1, Paragraph 4, “While CCP helps to select which branch of a tree are “good” to discriminate between classes, we also want to evaluate the statistical significance of each branch. This is done by the Fisher’s exact test (FET) … Therefore, given a threshold for the p value, we can find and keep the tree branches that are statistically significant (with lower p values), and discard those tree nodes that are not”. Class Confidence Proportion (CCP) is used to prune branches of the decision tree that are not statistically significant (which is based on a percentage) and can be seen within equation 3.12 which is comprised of equations 3.10 and 3.11. Thus, the pruning of the decision tree is based on a percentage of data points (as the CCP equation is a class confidence value calculated for branch significance while Fisher’s Exact Test (FET) is utilized to determine which nodes are for removal as they do not meet a significance threshold based on a percent)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to use the computer-implemented method of Karavani for determining and detecting positivity violations with the pruning methodology of Liu to improve accuracy, balance datasets, significant rules (Liu, Abstract, “… To generate rules which are statistically significant we design a novel and efficient top-down and bottom-up approach which uses Fisher’s exact test to prune branches of the tree which are not statistically significant. Together these two changes yield a classifier that performs statistically better than not only traditional decision trees but also trees learned from data that has been balanced by well-known sampling techniques …”; Page 776, Column 2, Paragraph 3, “We address the problem of designing a decision tree algorithm for classification which is robust against class imbalance in the data. We first explain why traditional decision tree measures, like information gain, are sensitive to class imbalance. We do that by expressing information gain in terms of the confidence of a rule. Information gain, like confidence, is biased towards the majority class. Having identified the cause of the problem, we propose a new measure, Class Confidence Proportion (CCP). Using both theoretical and geometric arguments we show that CCP is insensitive to class distribution. We then embed CCP in information gain and use the improvised measure to construct the decision tree. Using a wide array of experiments we demonstrate the effectiveness of CCP when the data sets are imbalanced. We also propose the use of Fisher exact test as a method for pruning the decision tree. Besides improving the accuracy, an added benefit of the Fisher exact test is that all the rules found are statistically significant”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM RAHMAN whose telephone number is (703)756-1646. The examiner can normally be reached M-F 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.R./ Examiner, Art Unit 2122
/KAKALI CHAKI/ Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights