DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Applicant's response, filed on 12/22/2025, has been fully considered. The following rejections and/or objections are either reiterated or newly applied. They constitute the complete set presently being applied to the instant application.
Status of claims
Canceled:
1-20, 26
Amended:
21, 24-25, 27-28, 31-34, 36-38
Pending:
21-25, 27-40
Withdrawn:
none
Examined:
21-25 and 27-40
Independent:
21, 32, 37
Allowable:
none
Priority
As detailed on the 03/08/2021 filing receipt, this application claims priority to as early as 03/30/2016.
Drawings
The drawings filed 02/15/2021 are accepted.
Information Disclosure Statement
The Information Disclosure Statements filed on 12/22/2025 is not in compliance with the provisions of 37 CFR 1.97 and have not been considered because legible copies of the non-patent literature document have not been provided in the instant application. 37 CFR 1.98(a)(2) requires that a legible copy of each foreign patent and non-patent literature must be provided. It is also noted that certain references lack appropriate page numbers and/or dates as is required under 37 CFR 1.97.
Election/Restrictions
Applicant’s election of Group I (claims 21-36), drawn to a method of obtaining clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to estimate one or more outcomes for a disease, in the reply filed on 06/30/2025 is acknowledged. Because applicant did not distinctly and specifically point out the supposed errors in the restriction requirement, the election has been treated as an election without traverse (MPEP § 818.01(a)). However, after further consideration, it has been determined that the claims are not restrictable and the restriction/election requirement in the office action mailed 05/01/2025 is withdrawn.
Withdrawn Rejections/Objections
The objection of the disclosure in the Office action mailed 08/21/2025 is withdrawn in view of the amendments filed 12/22/2025.
The rejection of claims 21-40 under 35 U.S.C. §112(b), Second Paragraph, in the Office action mailed 08/21/2025 is withdrawn in view of the amendments filed 12/22/2025. However, a new rejection is applied in view of claim amendments as indicated below.
The rejection of claims 21-36 under 35 U.S.C. §112(a), in the Office action mailed 08/21/2025 is withdrawn in view of the amendments filed 12/22/2025. However, a new rejection is applied in view of claim amendments as indicated below.
The rejection of claims 21-40 under 35 U.S.C. §103 as being unpatentable over De Leon (US2009/0150134 A1, published Jun. 11, 2009; cited on the 10/26/2023 IDS Document), in view of Barsoum (US 2012/0271612 A1, published Oct. 25, 2012; cited on the 10/26/2023 IDS Document) and Barhak (US 9,858,390 B2, Date of Patent: Jan. 2, 2018, prior publication data: Oct. 2, 2014; cited on the 10/26/2023 IDS Document), in the Office action mailed 08/21/2025 is withdrawn in view of the amendments filed 12/22/2025. However, a new rejection is applied in view of claim amendments as discussed below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 21-25 and 27-40 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Independent claims 21 (page 11, para. 7, determining step) and 32 (page 14, para. 4, determining step) recite “…a plurality of components that each correspond to one of the one or more observers…” The metes and bounds are not clearly defined because it is unclear what the component that corresponds to observers encompasses. The specification does not define “components that each correspond to one of the one or more observers” as recited in the claims.
Claim 31 recites “…a set of initial weights for the outcome definition input are equal.” The metes and bounds are not clearly defined because it is unclear what value is being compared to the recited “a set of initial weights for the outcome definition input” to determine whether the values are equal.
Claim 34 (page 15, lines 4-5) recites “second individual weightings for a set of individual components that correspond to individual ones of the plurality of observers…” The metes and bounds are not clearly defined because it is unclear what the component that corresponds to observers encompasses. The specification does not define “components that correspond to individual ones of the plurality of observers” as recited in the claim.
Dependent claims are rejected for depending on rejected claims.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 21-36 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Independent claims 21 (page 11, para. 7, determining step) and 32 (page 14, para. 4, determining step) recite “…a plurality of components that each correspond to one of the one or more observers…” The specification does not disclose “components that each correspond to one of the one or more observers” as recited in the claims.
Claim 34 (page 15, lines 4-5) recites “second individual weightings for a set of individual components that correspond to individual ones of the plurality of observers…” The specification does not disclose “components that correspond to individual ones of the plurality of observers” as recited in the claim.
Dependent claims are rejected for depending on rejected claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 21-25 and 27-40 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
In accordance with MPEP § 2106, claims found to recite statutory subject matter (Step 1: YES) are then analyzed to determine if the claims recite any concepts that equate to an abstract idea, law of nature or natural phenomenon (Step 2A, Prong 1). In the instant application, the claims recite the following limitations that equate to an abstract idea:
Mental processes recited include:
Claim 21 recites: "predict one or more outcomes for a disease, and the one or more outcomes having a respective definition …wherein the virtual outcomes are selected from a set of virtual outcomes and each virtual outcome of the set of virtual outcomes corresponds to a definition associated with the aggregate model; obtaining, by the computing system, outcome definition input from a plurality of observers, the outcome definition input indicating measures of difference between the respective definitions of the one or more outcomes included to the plurality of clinical studies and additional definitions of the virtual outcomes included in the set of virtual outcomes; determining, by the computing system, a difference between the virtual outcomes and the outcome definition input; wherein determining the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models included therein to minimize the difference between the virtual outcomes and the outcome definition input, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models included therein, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques; determining, by the computing system, a modified aggregate model that is responsive to the plurality of models and a plurality of components that each correspond to one of the one or more observers; determining, by the computing system, measures of difference between the modified virtual outcomes and observed outcomes included in the clinical study data; and determining, by the computing system and based on the measures of difference, individual weightings for individual ones of the plurality of components; wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the outcome definition input, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying. Selecting, determining, modifying and predicting are acts of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III). Modifying weights involves making changes to the weights of the model that could be practically performed in the human mind and/or with pen and paper.
Claim 23 recites: “wherein the virtual outcomes for the clinical study are determined using the virtual population; and the number of virtual individuals associated with each outcome are determined and multiple measures of difference are determined by determining a difference between the number of virtual individuals having the outcome in relation to the number of observed outcomes for the population of the clinical study.” Determining is an act of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper.
Claim 24 recites: "wherein the outcome definition input includes a value indicating a first measure of difference between a definition of a first outcome related to the aggregate model and a definition of a first additional outcome included in the clinical study." Measure of difference is an act of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III).
Claim 25 recites: “iteratively modifying a set of weightings associated with the observers by calculating a difference between the virtual outcomes and the observed outcomes.”
Claim 30 recites: wherein the scalar number is greater than one, less than one, or equal to one. This limitation requires comparing scalar numbers that is involved with evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper.
Claim 31 recites: wherein initial weights for the user input are equal. This limitation requires comparing weights that is involved with evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper.
Claim 32 recites: …estimate one or more outcomes for a disease…; wherein generating the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models to minimize a difference between the virtual outcomes and the additional definition associated with the aggregate model, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques; obtaining outcome definition input from a plurality of observers, the outcome definition input indicating measures of difference between the respective definitions of the outcomes included to the plurality of clinical studies and a set of definitions of the virtual outcomes included in the set of virtual outcomes; determining modified virtual outcomes for the virtual individuals included in the plurality of virtual populations based on the virtual outcomes and the outcome definition input; determining a modified aggregate model that is responsive to the plurality of models and a plurality of components that each correspond to one of the one or more observers; determining measures of difference between the modified virtual outcomes and observed outcomes included in the clinical study data; and determining, based on the measures of difference, individual weightings for individual components of the plurality of components.” Estimating, determining, modifying weights and measures of difference are acts of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III).
Claim 33 recites: determining a first fitness of a first set of initial conditions based at least partly on first results of a first number of simulations for a plurality of virtual populations with regard to the observed outcomes; and determining a second fitness of a second set of initial conditions based at least partly on second results of a second number of simulations for the plurality of virtual populations with regard to the observed outcomes. Determining is an act of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III).
Claim 35 recites: one or more user interface elements that are selectable to modify at least one of one or more first individual weightings or one or more second individual weightings. Selecting and modifying are acts of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III).
Claim 36 recites: receiving input via the user interface to modify an individual first weighting; and determining modified fitness scores based on the input received via the user interface. Determining and modifying are acts of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III).
Claim 37 recites: …determining, by the computing system, a plurality of models from the clinical study data, wherein individual models of the plurality of models estimate progression of a disease and the progression of the disease includes a plurality of states…; wherein generating the aggregate model includes iteratively modifying the coefficients to minimize the at least one characteristic, the iteratively modifying including applying, to the coefficients, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques; wherein determining the correction factors includes second iteratively modifying the correction factors in response to the user input, the second iteratively modifying being responsive to a minimum value of the minimized at least one characteristic, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying. Determining, modifying weights and estimating are acts of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III).
Claim 38 recites: using a query; and filtering, by the computing system, the summary data information according to import instructions to produce filtered population information, wherein the query is included in the import instructions used to filter the population information… Querying and filtering are acts of evaluating, analyzing and judging data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III).
Claim 39 recites: formatting, by the computing system, the filtered population information according to a predetermined template to produce formatted population information; and merging, by the computing system, the formatted population information with prior population information stored in a template file. Formatting involves arranging and organizing data that could be practically performed in the human mind and/or with pen and paper (See MPEP 2106.04(a)(2) subsection III).
Claim 40 recites: generating a first object that includes one or more first rules related to determining values of characteristics and includes one or more first objectives defining statistics for a first additional virtual population of the plurality of populations; and generating a second object that includes one or more second rules related to determining values of characteristics and includes one or more second objectives defining statistics related to a second additional virtual population of the plurality of populations, wherein the virtual population is an object that inherits from the first object and the second object.
Mathematical concepts recited include:
Claims 21 and 32 recite: wherein determining the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models included therein to minimize the difference between the virtual outcomes and the outcome definition input, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models included therein, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques; and wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the outcome definition input, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying. Gradient descent techniques or evolutionary computation techniques and iteratively modifying to minimize the difference are mathematical concepts and/or formulas.
Claim 25 recites: “iteratively modifying a set of weightings associated with the observers by calculating a difference between the virtual outcomes and the observed outcomes”. Calculating is a mathematical concept.
Claim 27 recites: " virtual outcomes are generated by performing one or more simulations with respect to at least portion of the plurality of populations using one or more Monte Carlo techniques." Monte Carlo techniques are mathematical concepts and/or formula.
Claim 30 recites: wherein the scalar number is greater than one, less than one, or equal to one. This limitation involves mathematical concepts.
Claim 32 recites: …estimate one or more outcomes for a disease… and measures of difference… are mathematical concepts.
Claim 37 recites: …estimate progression of a disease… and wherein generating the aggregate model includes iteratively modifying the coefficients to minimize the at least one characteristic, the iteratively modifying including applying, to the coefficients, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques; wherein determining the correction factors includes second iteratively modifying the correction factors in response to the user input, the second iteratively modifying being responsive to a minimum value of the minimized at least one characteristic, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying. Estimating, gradient descent techniques or evolutionary computation techniques and iteratively modifying to minimize the difference are mathematical concepts and/or formulas.
Claims 21, 23-25, 30-33 and 35-40, as indicated above, recite mental processes, such as determining, modifying and predicting. The claim elements as indicated above are involved with acts of evaluating, analyzing, observing and judging data as discussed above. Acts of evaluating and analyzing data could be practically performed in the human mind and/or with pen and paper because they merely require making observations, evaluations, judgments, and opinions (See MPEP 2106.04(a)(2) subsection III). Although, claims 21, 32 and 37 recites performing the method as part of a method executed on a computer, there are no additional limitations to indicate that anything other than a generic computer is required. However, merely requiring that the steps are carried out with a generic computer does not negate the mental nature of these steps and equates rather to merely using a computer as a tool to perform the mental process. Therefore, under the broadest reasonable interpretation, the indicated claims above can be practically carried out in the human mind or with pen and paper as claimed, which falls under the "Mental processes" grouping of abstract ideas.
Claims 21, 25, 27, 30-32 and 37, as indicated above, recite mathematical concepts and/or formulas. For instance, claim 21 recites gradient descent techniques or evolutionary computation techniques and iteratively modifying to minimize the difference; claim 25 recites calculating a difference and 27 recites Monte Carlo techniques that are mathematical concepts and/or formula. Therefore, under the broadest reasonable interpretation, the indicated claims above falls under the “mathematical concepts” grouping of abstract ideas.
As such, claims 21-25 and 27-40 recite an abstract idea (Step 2A, Prong 1: YES).
Claims found to recite a judicial exception under Step 2A, Prong 1 are then further analyzed to determine if the claims as a whole integrate the recited judicial exception into a practical application or not (Step 2A, Prong 2). The above indicated judicial exceptions are not integrated into a practical application because the claims do not recite an additional elements that apply, rely on or use the judicial exception in such a manner to amount to integration into a practical application. For example, there are no limitations that reflect an improvement to technology or applies or uses the recited judicial exception in some other meaningful way. Rather, the instant claims recite additional elements that equate to mere instructions to implement an abstract idea or insignificant extra solution activity. Specifically, the instant claims recite the following additional elements:
Claim 21 recites "obtaining, by a computing system including one or more hardware processors and memory, clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to predict one or more outcomes for a disease…," and "…obtaining, by the computing system, outcome definition input…" Obtaining clinical study data and outcome definition input are data gathering activities and inputting and outputting data. The computing system with hardware processors and memory equate to generic computer components.
Claim 23 recites “…outcome in relation to the number of observed outcomes for the population of the clinical study.” This limitation is outputting data.
Claim 24 recites: "outcome definition input includes a value indicating a first measure of difference between a definition of a first outcome related to the aggregate model and a definition of a first additional outcome included in the clinical study" This element is inputting and outputting data.
Claim 27 recites: "the virtual outcomes are generated by performing one or more simulations…" These limitations are outputting data.
Claim 28 recites: “observer input indicates a proportion of a clinical study outcome that corresponds to a model outcome” This limitation is inputting and outputting data.
Claim 32 recites: A computing system comprising: one or more processing units; one or more computer-readable storage media storing computer-readable instructions that, when executed by the one or more processing units… obtaining clinical study data that corresponds to a plurality of clinical studies related to a disease… indicating a model to estimate one or more outcomes for a disease, and the one or more outcomes having a respective definition… the virtual outcomes are selected from a set of virtual outcomes and each virtual outcome of the set of virtual outcomes corresponds to a definition associated with the aggregate model; obtaining outcome definition input from a plurality of observers, the outcome definition input indicating measures of difference between the respective definitions of the outcomes included to the plurality of clinical studies and additional definitions of the virtual outcomes included in the set of virtual outcomes… virtual outcomes for the virtual individuals included in the plurality of virtual populations based on the virtual outcomes and the outcome definition input… virtual outcomes and observed outcomes included in the clinical study data. The limitations include obtaining, inputting and outputting data. The computing system with hardware processors and memory equate to generic computer components.
Claims 33-34 and 36 recite: computer-readable media store instructions that, when executable by the one or more processing units, cause the computing system to perform operations
Claims 35-36 recite: user interface
Claim 37 recites "obtaining, by a computing system including one or more hardware processors and memory, clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to predict one or more outcomes for a disease…," and "obtaining, by the computing system, user input from one or more observers, the user input indicating differences between first definitions of outcomes associated with the aggregate model and second definitions of outcomes associated with the plurality of clinical studies…" Obtaining clinical study data and user input are data gathering. The computing system with hardware processors and memory equate to generic computer components.
Claim 38 recites: obtaining, by the computing system, the summary data from at least one online database using a query; and filtering, by the computing system, the summary data according to import instructions to produce filtered population information, wherein the query is included in the import instructions used to filter the population information. The claim limitations are involved with data gathering and outputting.
Claim 39 recites: formatting, by the computing system, the filtered population information according to a predetermined template to produce formatted population information; and merging, by the computing system, the formatted population information with prior population information stored in a template file. The limitations are involved with data processing for output.
The elements of claims 21, 23-24, 27-28 and 32-39 as indicated above equate to insignificant extra solutional activities. Extra-solution activity includes both pre-solution and post-solution activity. An example of pre-solution activity is a step of gathering data for use in a claimed process, e.g., a step of obtaining information about credit card transactions, which is recited as part of a claimed process of analyzing and manipulating the gathered information by a series of steps in order to detect whether the transactions were fraudulent. An example of post-solution activity is an element that is not integrated into the claim as a whole, e.g., a printer that is used to output a report of fraudulent transactions, which is recited in a claim to a computer programmed to analyze and manipulate information about credit card transactions in order to detect whether the transactions were fraudulent (See MPEP 2106.05(g)). Claim 21 and 32-39 recites a computer system comprising a computer readable storage medium, processor, memory and storing computer-readable instructions. These elements of claims 21 and 32-39 equate to generic computer components. Claims 21 and 32-39 invoke the computer components merely as tools to execute the abstract idea. The use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. (see MPEP 2106.05(f)). Additionally, the listed additional elements are mere instructions to apply an exception because they recite no more than an idea of a solution or outcome and does not recite a technological solution to a technological problem. (See MPEP 2106.05(f)(1)). As such, as currently recited, the claims do not appear to recite an improvement to technology or apply or use the recited judicial exception in some other meaningful way. Therefore, claims 21-25 and 27-40 are directed to an abstract idea (Step 2A, Prong 2: NO).
Claims found to be directed to a judicial exception are then further evaluated to determine if the claims recite an inventive concept that provides significantly more than the judicial exception itself (Step 2B). The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claims recite additional elements that equate to well-understood, routine and conventional activities, insignificant extra-solution activity or mere instructions to implement the abstract idea on a generic computer. The instant claims recite the following additional elements:
Claim 21 recites "obtaining, by a computing system including one or more hardware processors and memory, clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to predict one or more outcomes for a disease…," and "…obtaining, by the computing system, outcome definition input…" Obtaining clinical study data and outcome definition input are data gathering activities and inputting and outputting data. The computing system with hardware processors and memory equate to generic computer components.
Claim 23 recites “…outcome in relation to the number of observed outcomes for the population of the clinical study.” This limitation is outputting data.
Claim 24 recites: "outcome definition input includes a value indicating a first measure of difference between a definition of a first outcome related to the aggregate model and a definition of a first additional outcome included in the clinical study" This element is inputting and outputting data.
Claim 27 recites: "the virtual outcomes are generated by performing one or more simulations…" These limitations are outputting data.
Claim 28 recites: “observer input indicates a proportion of a clinical study outcome that corresponds to a model outcome” This limitation is inputting and outputting data.
Claim 32 recites: A computing system comprising: one or more processing units; one or more computer-readable storage media storing computer-readable instructions that, when executed by the one or more processing units… obtaining clinical study data that corresponds to a plurality of clinical studies related to a disease… indicating a model to estimate one or more outcomes for a disease, and the one or more outcomes having a respective definition… the virtual outcomes are selected from a set of virtual outcomes and each virtual outcome of the set of virtual outcomes corresponds to a definition associated with the aggregate model; obtaining outcome definition input from a plurality of observers, the outcome definition input indicating measures of difference between the respective definitions of the outcomes included to the plurality of clinical studies and additional definitions of the virtual outcomes included in the set of virtual outcomes… virtual outcomes for the virtual individuals included in the plurality of virtual populations based on the virtual outcomes and the outcome definition input… virtual outcomes and observed outcomes included in the clinical study data. The limitations include obtaining, inputting and outputting data. The computing system with hardware processors and memory equate to generic computer components.
Claims 33-34 and 36 recite: computer-readable media store instructions that, when executable by the one or more processing units, cause the computing system to perform operations
Claims 35-36 recite: user interface
Claim 37 recites "obtaining, by a computing system including one or more hardware processors and memory, clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to predict one or more outcomes for a disease…," and "obtaining, by the computing system, user input from one or more observers, the user input indicating differences between first definitions of outcomes associated with the aggregate model and second definitions of outcomes associated with the plurality of clinical studies…" Obtaining clinical study data and user input are data gathering. The computing system with hardware processors and memory equate to generic computer components.
Claim 38 recites: obtaining, by the computing system, the summary data from at least one online database using a query; and filtering, by the computing system, the summary data according to import instructions to produce filtered population information, wherein the query is included in the import instructions used to filter the population information. The claim limitations are involved with data gathering and outputting.
Claim 39 recites: formatting, by the computing system, the filtered population information according to a predetermined template to produce formatted population information; and merging, by the computing system, the formatted population information with prior population information stored in a template file. The limitations are involved with data processing for output.
The additional elements indicated above do not comprise an inventive concept when considered individually or as an ordered combination that transforms the claimed judicial exception into a patent-eligible application of the judicial exception. The limitations equate to mere data gathering activities and outputting, which are insignificant extra solutional activities. As explained by the Supreme Court, the addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional. (see MPEP 2106.05(g)). Also, limitations that equate to mere data gathering and outputting via generic computer components, such as receiving data at a computer or outputting data, amount to insignificant extra-solution activity as set forth by the courts in Mayo, 566 U.S. at 79, 101 USPQ2d at 1968 and OIP Techs., Inc, v, Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015). Also, the use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more as identified by the courts in Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Also, obtaining virtual outcomes from virtual individuals are known methods as discussed in Figure 2 (Page 3) of Saka (Saka, Görkem, et al. "Use of dynamic microsimulation to predict disease progression in patients with pneumonia-related sepsis." Critical Care 11.3 (2007): R65.; as cited on the attached 892 form). Therefore, the claims do not amount to significantly more than the judicial exception itself (Step 2B: No). As such, claims 21-25 and 27-40 are not patent eligible.
Response to 35 USC § 101 Arguments (Remarks filed 12/22/2025, pages 19-21)
Applicant amened claims 21, 24-25, 27-28, 31-34, 36-38.
It is noted that Applicant’s remarks are based on amended claims.
In the remarks, Applicant does not agree with the office action that the claimed method recites mental processes, mathematical concepts and methods of organizing human activities.
Regarding mental processes, Applicant states that the recited steps include performing a gradient descent technique or an evolutionary computation technique that are far too detailed to be practically performed in the mind, or even with pencil and paper. Applicant also states that such a computation would take unduly long even when performed by a computer, unless the advances in simulation described in the Application are used. Applicant similarly states that the construction and simulation of the virtual population is impractical to perform in the human mind and optimization involves many simulation runs that is also impractical to perform in the human mind. Applicant further states that the techniques allow performance of optimization with far less computation than previously known, an advance in computation which by itself is clearly not a "mental step" and are advances in computation which are technical advances that "promote the progress of science and useful arts", U.S. Const. Art. I, § 8, cl. 8, and which are not subject to any judicial "exception".
Regarding mathematical concepts, Applicant states that the courts only state that a "mathematical formula as such" (that is, without anything more) is not patent-eligible. Diamond v. Diehr, 490 U.S. 175, 191 (1981) (emphasis added). Applicant asserts that the claims which the Office Action asserts recite "mathematical concepts" make substantial additional recitals and do not preempt any mathematical concept.
Regarding methods of organizing human activities, Applicant states that the claimed method does not recite any fundamental economic principles, commercial or legal transactions, or methods for managing personal behavior.
In response, Applicant’s arguments are not persuasive because there is no indication in the claims that the process or amount of data is too complicated or too large to be performed by the human mind and/or with pen and paper. Also, the additional elements of the claims do not integrate the JEs into a practical application as seen in Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981)). In Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981)), the improvement was provided by one or more additional elements. However, the claims do not recite methods of organizing human activities due to claim amendments and remarks. Therefore, the claims recite mental processes and mathematical concepts as discussed in the 101 rejection section above.
Additionally, Applicant’s statement of improvement is not persuasive because it is a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art. From the asserted improvement, it is not clear how the claimed invention improves over existing technology and it is also not clear how one would gauge the improvement since there are no metrics for comparison between the claimed technology and previous technology. Overall, one of ordinary skill in the art cannot gauge whether the improvements asserted are delivered by the claims because the details provided in the specification do not provide sufficient details such that the improvement would be apparent, do not explain the details of an unconventional technical solution expressed in the claim, or identify technical improvements realized by the claim over the prior art. As stated in MPEP 2106.05(a) and MPEP 2106.04(d), the disclosure must provide sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. Furthermore, if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology. An indication that the claimed invention provides an improvement can include a discussion in the specification that identifies a technical problem and explains the details of an unconventional technical solution expressed in the claim, or identifies technical improvements realized by the claim over the prior art. (see MPEP 2106.05(a) and MPEP 2106.04(d)).
Applicant further states that the Office has recited some of the claim recitations and made the conclusory statement that the claims do not recite "significantly more", the Office Action fails to make the factual findings and meet the evidentiary standard of Berkheimer v. HP, Inc., 881 F.3d 1360 (Fed. Cir. 2018). See also Robert Bahr, Changes in Examination Procedure Pertaining to Subject Matter Eligibility, Recent Subject Matter Eligibility Decision (Berkheimer v. HP, Inc.) (Apr. 19, 2018) ("Bahr Memo").
In response, Applicant’s arguments are not persuasive. According to MPEP 2106.07(a), when the examiner has concluded that certain claim elements recite well understood, routine, conventional activities in the relevant field, the examiner must expressly support the rejection in writing with one of the four options specified in Subsection III of MPEP 2106.07(a). One of the listed supports that could be utilized is a citation to one or more of the court decisions discussed in MPEP § 2106.05(d), subsection II, as noting the well-understood, routine, conventional nature of the additional element(s). Therefore, citing case law meets the burden of establishing the conventionality of the additional elements as indicated in the 101 Claims Rejections section under Step 2B above. Additionally, the prior art Saka as indicated above, is evidence for the conventionality of obtaining virtual outcomes from virtual individuals as indicated in the 101 Claims Rejections section under Step 2B above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 21-25 and 27-40 are rejected under 35 U.S.C. 103 as being unpatentable over De Leon (US2009/0150134 A1, published Jun. 11, 2009; cited on the 10/26/2023 IDS Document), in view of Barsoum (US 2012/0271612 A1, published Oct. 25, 2012; cited on the 10/26/2023 IDS Document); Barhak (US 9,858,390 B2, Date of Patent: Jan. 2, 2018, prior publication data: Oct. 2, 2014; cited on the 10/26/2023 IDS Document) and Lacy (Modeling Movement Disorders in Parkinson's Disease using Computational Intelligence. Diss. University of York, 2015; cited on the attached 892 form).
Regarding independent claim 21, De Leon teaches obtaining, by a computing system including one or more hardware processors and memory, clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to predict one or more outcomes for a disease, and the one or more outcomes having a respective definition with “One aspect of the invention provides computer-implemented methods of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject;… (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user. In certain implementations, each virtual patient of the virtual population has an associated prevalence. In such cases, applying one or more virtual protocols to the one or more selected virtual patients can comprise calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients. The set of output can comprise the likelihood of each clinical outcome or the likelihood of a selected set of clinical outcomes, for example the most likely outcomes or a subset of outcomes representative of the range of clinical outcomes.” (Para. [0010]) and with “The identification of the one or more biological processes can be based on, for example, experimental or clinical data, scientific literature, genetic studies, results of a computer model, model sensitivity analysis, or a combination of them.” (para. [0058]).
De Leon teaches generating, by the computing system and using the aggregate model, virtual outcomes for virtual individuals included in a plurality of virtual populations, wherein the virtual outcomes are selected from a set of virtual outcomes and each virtual outcome of the set of virtual outcomes corresponds to a definition associated with the aggregate model with “(d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient” (Claim 9 of De Leon).
De Leon teaches obtaining, by the computing system, outcome definition input from a plurality of observers, the outcome definition input indicating measures of difference between the respective definitions of the one or more outcomes included to the plurality of clinical studies and additional definitions of the virtual outcomes included in the set of virtual outcomes with “To provide for interaction with a user, the invention can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.” (Para. [0095]); “The computer model can be built using a “top-down” approach that begins by defining a general set of behaviors indicative of a biological condition, e.g. a disease.” (Para. [0044]) and “One aspect of the invention provides computer-implemented methods of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject; (c) selecting one or more virtual patients from the virtual population based on a similarity between each of the selected virtual patients and the input data; (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user. In certain implementations, each virtual patient of the virtual population has an associated prevalence. In such cases, applying one or more virtual protocols to the one or more selected virtual patients can comprise calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients. The set of output can comprise the likelihood of each clinical outcome or the likelihood of a selected set of clinical outcomes, for example the most likely outcomes or a subset of outcomes representative of the range of clinical outcomes. In certain implementations, the virtual population is a prevalence-weighted virtual population, wherein each virtual patient of the virtual population has an associated prevalence weight. In a preferred implementation, the virtual protocol represents a stimulus selected from the group consisting of a therapeutic regimen, passage of time, exercise, weight gain, diet, a lifestyle choice and a combination of two or more of the same.” (Para. [0010]).
De Leon teaches determining, by the computing system, modified virtual outcomes for the virtual individuals included in the plurality of virtual populations based on the virtual outcomes and the outcome definition input with “The method of claim 9, wherein each virtual patient of the virtual population has an associated prevalence.” (De Leon, claim 10); “The method of claim 10, wherein applying one or more virtual protocols to the one or more selected virtual patients comprises calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients.” (De Leon, claim 11) and “The method of claim 11, wherein the set of outputs comprises the likelihood of each clinical outcome.” (De Leon, claim 12).
De Leon teaches modifying a computer model with “…(d) modifying the computer model to reflect the function of the one or more identified genes; and (e) storing the modified computer model in a computer-readable storage medium. The computer model can modified to directly reflect the function of the one or more identified genes, to directly reflect absence of the function of the one or more identified genes or to indirectly reflect the downstream function of the one or more identified genes. In certain implementations, the method can further comprise (f) executing the modified computer model to generate a simulated outcome; and (g) comparing the simulated outcome with the known association between the genetic marker and clinical phenotype to confirm the validity of the modified computer model.” (Para. [0011]).
De Leon teaches determining, by the computing system, measures of difference between the modified virtual outcomes and observed outcomes included in the clinical study data with “…(f) executing the modified computer model to generate a simulated outcome; and (g) comparing the simulated outcome with the known association between the genetic marker and clinical phenotype to confirm the validity of the modified computer model. In such a case, comparing the simulated outcome with the known associate between the genetic marker and clinical phenotype optionally can comprise comparing a virtual population with a clinical population.” (Para. [0011]).
De Leon teaches a computer model of a biological system to reflect genomic information (Para. 0011) and a computer model to generate a simulated outcome (De Leon, Claim 20(f)).
De Leon does not teach generating, by the computing system, an aggregate model that includes a plurality of models included in the clinical study data and a respective weighting for each model of the plurality of models and determining, by the computing system, a modified aggregate model that includes the plurality of models and a plurality of components that each correspond to an observer of the one or more observers of claim 21. However, this limitation is taught by Barsoum.
De Leon also does not teach determining, by the computing system and based on the measures of difference, individual weightings for individual components of the plurality of components of claim 21. However, this limitation is taught by Barhak.
De Leon also does not teach wherein determining the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models included therein to minimize the difference between the virtual outcomes and the outcome definition input, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models included therein, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques and wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the outcome definition input, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying of claim 21. However, these limitations are taught by Lacy.
Barsoum teaches generating, by the computing system, an aggregate model that includes a plurality of models included in the clinical study data and a respective weighting for each model of the plurality of models with “FIG. 3 depicts an example of an aggregated model generation system 100 that can be utilized to create an aggregate model 102.” (para. [0060]) and with “The predictor variables can be analyzed to generate a model having a portion of the predictor variables with weighted coefficients according to an event or outcome for which the model is generated. A prediction tool can employ the model to predict the even or outcome for one or more patients.” (Abstract).
Barsoum teaches determining, by the computing system, a modified aggregate model that includes the plurality of models and a plurality of components that each correspond to an observer of the one or more observers with “FIG. 3 depicts an example of an aggregated model generation system 100 that can be utilized to create an aggregate model 102. The model generation system 100 can include a model modification method 104 that is programmed to modify an encounter-specific model 106, such as a corresponding model generated by the systems and methods of FIGS. 1 and 2 (e.g., model 38). The encounter-specific model 106 thus is generated for predicting a patient outcome based on analysis of model data generated from a plurality of patients' final coded data set that is stored in one or more sources of patient data. The encounter-specific model 106 thus can be utilized to predict an outcome generally for any patient. In some examples, longitudinal patient data 108 for a given patient may be relevant to determining coefficients and predictor variables relevant to predicting an outcome for the given patient. Thus, the model modification function 104 can modify the encounter-specific model 106 (e.g., generated by the model generator 36 of FIG. 2) and provide the aggregate model based upon the longitudinal patient data 108 for a given patient. That is, the model modification function 104 can adjust the model for a given patient depending on the given patient's circumstances.” (Para. [0060]).
Barhak teaches determining, by the computing system and based on the measures of difference, individual weightings for individual components of the plurality of components with “C3) Weighing differences between model and observed study results allow the user to emphasize the importance of each specific outcome with a weight. For example it allows the user do emphasize the weight of a stroke or the weight of a death from stroke to be more important than that of Death from other causes.” (Col. 7. Para. 1).
Lacy teaches the claim limitation of wherein determining the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models included therein to minimize the difference between the virtual outcomes and the outcome definition input, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models included therein, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques and wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the outcome definition input, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying with “The data instances are weighted according to their classification accuracy, so that the second model to be fitted will be rewarded more highly if it manages to correctly predict the instances that the first model failed on. In this manner, the criteria of having an ensemble of classifiers who make their errors on disjoint parts of the data set is explicitly managed. After each iteration, the weights are updated to reflect the current ensemble’s performance.” (page 58, para. 2) and “Once a predetermined number of iterations has completed, the ensemble is formed and can be used to predict new patterns by aggregating all the members’ votes. AdaBoost (Freund et al., 1996, 1999) is the most well known algorithm in this field although numerous similar techniques exist under the name gradient boosting. It uses a WMV aggregation scheme, with the weights derived from the base classifiers’ performance on the training set. While boosting can be employed with any learning algorithm, it is frequently used to strong effect with decision trees; Breiman (1996b) explains the reasons for this success. Referring back to the bias-variance decomposition discussed in Section 3.3, boosting (as with many ensemble techniques including bagging) works most effectively by reducing the variance of low-bias classifiers (Johnson & Rayens, 2007).” (page 58, para. 2).
It would have been prima facia obvious to combine the teachings of De Leon and Barsoum to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include an aggregated model as taught by Barsoum for use in predicting one or more patient outcomes (Barsoum, Para. 0063). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barsoum teach methods that pertain to predicting patient outcomes.
It would also have been prima facia obvious to combine the teachings of De Leon and Barhak to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include determining the measures of difference by weight for models as taught by Barhak to emphasize the importance of each specific outcome (Barhak, Col. 7. Para. 1). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barhak teach methods that pertain to predicting patient outcomes.
It would also have been prima facia obvious to combine the teachings of De Leon and Lacy to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include iteratively modifying the weights of the different of models with gradient descent techniques as taught by Lacy for the benefit of optimizing the models’ performance. Furthermore, there would have been a reasonable expectation of success, since both De Leon and Lacy teach methods that pertain to the analysis of genetic data to predict disease.
Regarding claim 22, De Leon teaches wherein the clinical study data includes, for individual clinical studies, population summary data indicating values of characteristics of one or more populations that participated in the individual clinical study, and the method comprising: generating, by the computing system, the characteristics of the virtual individuals included in a virtual population of the plurality of virtual populations based on the summary data for a clinical study of the plurality of clinical studies with “The American Diabetes Association released an extension of Archimedes, called Diabetes PHD. When a person uses Diabetes PHD and provides information about himself or herself, the system creates a simulated person who has the same characteristics (e.g., sex, age, race/ethnicity), same features (e.g., height, weight, blood pressure), same laboratory test results (e.g., glucose, cholesterol), the same past medical history, family history, symptoms, complications, and the same treatments as the person providing the information. The system then takes this simulated version of the person and creates a thousand “identical looking” people.” (Para. [0008]); “The term “population,” as used herein, refers to a group or collection of individuals, either real or virtual. The individuals in the collection of individuals can be from or represent, for example, a group of subjects having a particular disease, treatment history, physiologic or genotypic characteristic(s), and the like. A population is typically a collection of individuals about which one wants to generalize, e.g., the inhabitants of Greenland, cancer patients receiving chemotherapy, severe diabetics, hypertensive rats, etc. The population is typically comprised of mammals of a similar species, e.g. humans.” (para. [0020]); “The term “population characteristics,” as used herein, refers to any qualitative or quantitative features, behaviors, or aspects of the population that are of interest. For example, if the population is cancer patients receiving chemotherapy, the population characteristics may include tumor mass, five-year survival rate, red blood cell (“RBC”) count, and white blood cell (“WBC”) count; if the population is severe diabetics, the population characteristics may include fasting glucose, HbAlc, circulating free fatty acids (“FFA”) concentrations; and if the population is hypertensive rats, the population characteristics may include mean arterial pressure (“MAP”), diastolic blood pressure (“DBP”), systolic blood pressure (“SBP”).” (para. [0022]); “The term “virtual patient population,” as used herein, represents the population characteristics of a population of real subjects, such as a clinical population of interest. The virtual patient population has statistical properties or behaviors (e.g., mean, median, variance, dynamics, etc.) that approximate the statistical properties or behavior of a sample population of real subjects.” (para. [0025]) and “In certain implementations, the system of the invention comprises a computer readable storage medium storing a plurality of variables, parameters and/or mathematical representations that comprise a computer model of a biological system. The computer readable storage medium also can comprises a plurality of variable and/or parameters representing one or more virtual patients. In the case that the computer readable storage medium comprises a plurality of variables and/or parameters representing more than one virtual patient, the computer readable medium can also comprise a plurality of variables and/or parameters representing population characteristics of each of the virtual patients and of the virtual population as a whole.” (para. [0091]).
Regarding claim 23, De Leon teaches wherein the virtual outcomes for the clinical study are determined using the virtual population; and the number of virtual individuals associated with each outcome are determined and multiple measures of difference are determined by determining a difference between the number of virtual individuals having the outcome in relation to the number of observed outcomes for the population of the clinical study with “One aspect of the invention provides computer-implemented methods of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject; (c) selecting one or more virtual patients from the virtual population based on a similarity between each of the selected virtual patients and the input data; (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user. In certain implementations, each virtual patient of the virtual population has an associated prevalence. In such cases, applying one or more virtual protocols to the one or more selected virtual patients can comprise calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients. The set of output can comprise the likelihood of each clinical outcome or the likelihood of a selected set of clinical outcomes, for example the most likely outcomes or a subset of outcomes representative of the range of clinical outcomes. In certain implementations, the virtual population is a prevalence-weighted virtual population, wherein each virtual patient of the virtual population has an associated prevalence weight. In a preferred implementation, the virtual protocol represents a stimulus selected from the group consisting of a therapeutic regimen, passage of time, exercise, weight gain, diet, a lifestyle choice and a combination of two or more of the same.” (para. [0069]).
Regarding claim 24, De Leon teaches wherein the outcome definition input includes a value indicating a first measure of difference between a definition of a first outcome related to the aggregate model and a definition of a first additional outcome included in the clinical study with “Another aspect of the invention provides methods for modifying a computer model of a biological system to reflect genomic information, the method comprising: (a) providing a computer model of a biological system in a computer-readable storage medium; (b) providing a genetic marker having a known association with a clinical phenotype, wherein the genetic marker has a known locus on a chromosome; (c) identifying one or more genes of known biological function that have linkage disequilibrium with the locus of the genetic marker; (d) modifying the computer model to reflect the function of the one or more identified genes; and (e) storing the modified computer model in a computer-readable storage medium. The computer model can be modified to directly reflect the function of the one or more identified genes, to directly reflect absence of the function of the one or more identified genes or to indirectly reflect the downstream function of the one or more identified genes. In certain implementations, the method can further comprise (f) executing the modified computer model to generate a simulated outcome; and (g) comparing the simulated outcome with the known association between the genetic marker and clinical phenotype to confirm the validity of the modified computer model. In such a case, comparing the simulated outcome with the known associate between the genetic marker and clinical phenotype optionally can comprise comparing a virtual population with a clinical population.” (para. [0076]); “The term “goodness-of-fit,” as used herein, refers to the similarity of two or more distributions, such as a prediction or simulation compared to an actual observation. Measures of goodness-of-fit include any method or process by which one quantifies and/or qualifies such similarity.” (para. [0027]) and “To evaluate similarity or correlation, a measure of goodness-of-fit between the common features for the virtual patients and the common features for the real subjects can be calculated. A measure of goodness-of-fit between the combinations of the common features for the virtual patients and the combinations of the common features for the real subjects can be calculated.” (para. [0064]) and “Common features can include at least one continuous dependent variable and similarity can be evaluated by calculating one or more summary statistics for the continuous dependent variable for the real subjects, calculating the one or more summary statistics for the continuous dependent variable for the virtual patients according to their respective prevalences, and comparing the one or more summary statistics for the real subjects with the summary statistics for the virtual patients” (paragraph [0065]).
Regarding claim 25, De Leon teaches iteratively modifying a set of weightings associated with the observers by calculating a difference between the virtual outcomes and the observed outcomes with “The term “goodness-of-fit,” as used herein, refers to the similarity of two or more distributions, such as a prediction or simulation compared to an actual observation. Measures of goodness-of-fit include any method or process by which one quantifies and/or qualifies such similarity.” (para. [0027]) and “To evaluate similarity or correlation, a measure of goodness-of-fit between the common features for the virtual patients and the common features for the real subjects can be calculated. A measure of goodness-of-fit between the combinations of the common features for the virtual patients and the combinations of the common features for the real subjects can be calculated.” (para. [0064]) and “The term “prevalence,” as used herein to describe a virtual patient, indicates the occurrence, e.g. the frequency of occurrence, of that virtual patient in a virtual patient population. The prevalence of any particular virtual patient in a virtual patient population can be defined by a weighting factor or weight, wherein each weight adjusts for over- or under-representation of the characteristics of the virtual patient in the population. The prevalence of a virtual patient relates to the likelihood that there is a real subject in the population with characteristics of or similar to the virtual patient.” (Para. 0026).
Regarding claim 28, De Leon teaches wherein the observer input indicates a proportion of a clinical study outcome that corresponds to a model outcome with “ One aspect of the invention provides computer-implemented methods of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject; (c) selecting one or more virtual patients from the virtual population based on a similarity between each of the selected virtual patients and the input data; (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user. In certain implementations, each virtual patient of the virtual population has an associated prevalence. In such cases, applying one or more virtual protocols to the one or more selected virtual patients can comprise calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients. The set of output can comprise the likelihood of each clinical outcome or the likelihood of a selected set of clinical outcomes, for example the most likely outcomes or a subset of outcomes representative of the range of clinical outcomes. In certain implementations, the virtual population is a prevalence-weighted virtual population, wherein each virtual patient of the virtual population has an associated prevalence weight. In a preferred implementation, the virtual protocol represents a stimulus selected from the group consisting of a therapeutic regimen, passage of time, exercise, weight gain, diet, a lifestyle choice and a combination of two or more of the same.” (para. [0068]) and “A computer-implemented method of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject; (c) selecting one or more virtual patients from the virtual population based on a similarity between each of the selected virtual patients and the input data; (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user.” (Claim 9 of De Leon)
De Leon does not teach wherein the proportion of the clinical study outcome that corresponds to the model outcome is a scalar number of claim 29 and wherein the scalar number is greater than one, less than one, or equal to one of claim 30. However, these limitations are taught by Barsoum.
De Leon also does not teach teaches wherein the virtual outcomes are generated by performing one or more simulations with respect to at least portion of the plurality of populations using one or more Monte Carlo techniques of claim 27 and wherein initial weights for the user input are approximately equal of claim 31. However, these limitations are taught by Barhak.
Regarding claim 29, Barsoum teaches wherein the proportion of the clinical study outcome that corresponds to the model outcome is a scalar number with “For instance, the output generator 42 can compare the predicted outcome to one more thresholds, such as can vary depending on the outcome for which the model has been generated.” (Para. [0050]) and “As one example, some types of models, such as for diagnosing a medical condition, may have a single threshold (e.g., a risk threshold), which if the value of the predicted outcome computed by the prediction tool 40 exceeds the threshold, the output generator 42 can provide an output identifying the diagnosis for the given patient. The output generator 42 can employ multiple thresholds for models generated for other types of outcomes (e.g., readmission risk, patient satisfaction, length of stay and the like). For assessments based on these types of predicted outcomes, the output generator 42 can vary the output that is generated based on the value of the predicted outcome relative to the predicted outcome to the thresholds that have been set. Thus, as the risk of such an outcome increases (as determined relative to predetermined thresholds), the output can increase in scale commensurately with such risk. For instance, different graphical representations of such risk can be provided and/or can be color coded (e.g., yellow, orange, red) to indicate the level of severity. Other types of severity scales and risk indicators can be utilized, which can include providing a normalized scale of the value of the predicted outcome (e.g., as a percentage). By employing a variety of models various types of outcomes can be predicted for each patient in real-time during a patient's stay in the hospital and thereby help to mitigate the risk of negative outcomes and increase the likelihood of positive outcomes.” (Para. [0051]).
Regarding claim 30, Barsoum teaches wherein the scalar number is greater than one, less than one, or equal to one with “For instance, the output generator 42 can compare the predicted outcome to one more thresholds, such as can vary depending on the outcome for which the model has been generated.” (Para. [0050]) and “As one example, some types of models, such as for diagnosing a medical condition, may have a single threshold (e.g., a risk threshold), which if the value of the predicted outcome computed by the prediction tool 40 exceeds the threshold, the output generator 42 can provide an output identifying the diagnosis for the given patient. The output generator 42 can employ multiple thresholds for models generated for other types of outcomes (e.g., readmission risk, patient satisfaction, length of stay and the like). For assessments based on these types of predicted outcomes, the output generator 42 can vary the output that is generated based on the value of the predicted outcome relative to the predicted outcome to the thresholds that have been set. Thus, as the risk of such an outcome increases (as determined relative to predetermined thresholds), the output can increase in scale commensurately with such risk. For instance, different graphical representations of such risk can be provided and/or can be color coded (e.g., yellow, orange, red) to indicate the level of severity. Other types of severity scales and risk indicators can be utilized, which can include providing a normalized scale of the value of the predicted outcome (e.g., as a percentage). By employing a variety of models various types of outcomes can be predicted for each patient in real-time during a patient's stay in the hospital and thereby help to mitigate the risk of negative outcomes and increase the likelihood of positive outcomes.” (Para. [0051]).
It would have been prima facia obvious to combine the teachings of De Leon and Barsoum to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to modify the weights of the different models of the aggregated model as taught by Barsoum to adjust the model for a given patient depending on the given patient's circumstances. (Barsoum, Para. 0060). A person of ordinary skill in the art would also have been motivated to modify the method of De Leon to represent the clinical outcome as a scalar number as taught by Barsoum to easily provide a representation of the level of risk and severity (Barsoum, Para. 0051). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barsoum teach methods that pertain to predicting patient outcomes.
Regarding claim 27, Barhak teaches wherein the virtual outcomes are generated by performing one or more simulations with respect to at least a portion of the plurality of populations using one or more Monte Carlo techniques with “The Reference Model is built from publicly available data, using MIST (Micro-Simulation Tool)—a Python based modeling framework which is available under General Public License (GPL), and does not require access to proprietary, or individual patient, information. The software uses Monte Carlo simulations that are executed in parallel, and in the data shown in this application, the system can run on a single machine, on a cluster of machines, and on a cluster in the cloud. With the model utilizing computer power/techniques, such as parallel processing/high performance computing, cross validation, competition among alternative equations and/or hypotheses combinations, and ranks results based on fitness via a fitness engine multiple studies can be compared and large amounts of study data, which was inaccessible before, is now available to determine disease progression over large populations comprising large data sets.” (Col. 3, para. 6).
Regarding claim 31, Barhak teaches wherein a set of initial weights for the outcome definition input are equal with “To make sure all equations use the same coefficients, a first order Taylor series expansion can be used, or the user can define those weights manually.” (Col. 6, para. 8).
It would also have been prima facia obvious to combine the teachings of De Leon and Barhak to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include using Monte Carlo techniques for simulations to generate virtual outcomes as taught by Barhak to execute simulations in parallel (Barhak, Col. 3. Para. 6). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barhak teach methods that pertain to predicting patient outcomes.
Regarding independent claim 32, De Leon teaches A computing system comprising: one or more processing units; one or more computer-readable storage media storing computer-readable instructions that, when executed by the one or more processing units with “The invention and all of the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structural means disclosed in this specification and structural equivalents thereof, or in combinations of them. The invention can be implemented as one or more computer program products, i.e., one or more computer programs tangibly embodied in an information carrier, e.g., in a machine-readable storage device, for execution by, or to control the operation of, data processing apparatus, e.g., a programmable processor, a computer, or multiple computers. A computer program (also known as a program, software, software application, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment” (Para. 0090).
De Leon teaches obtaining clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to predict one or more outcomes for a disease, and the one or more outcomes having a respective definition with “One aspect of the invention provides computer-implemented methods of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject;… (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user. In certain implementations, each virtual patient of the virtual population has an associated prevalence. In such cases, applying one or more virtual protocols to the one or more selected virtual patients can comprise calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients. The set of output can comprise the likelihood of each clinical outcome or the likelihood of a selected set of clinical outcomes, for example the most likely outcomes or a subset of outcomes representative of the range of clinical outcomes.” (Para. [0010]) and with “The identification of the one or more biological processes can be based on, for example, experimental or clinical data, scientific literature, genetic studies, results of a computer model, model sensitivity analysis, or a combination of them.” (para. [0058]).
De Leon teaches generating using the aggregate model, virtual outcomes for virtual individuals included in a plurality of virtual populations, wherein the virtual outcomes are selected from a set of virtual outcomes and each virtual outcome of the set of virtual outcomes corresponds to a definition associated with the aggregate model with “(d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient” (Claim 9 of De Leon).
De Leon teaches obtaining outcome definition input from a plurality of observers, the outcome definition input indicating measures of difference between the respective definitions of the one or more outcomes included to the plurality of clinical studies and additional definitions of the virtual outcomes included in the set of virtual outcomes with “To provide for interaction with a user, the invention can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.” (Para. [0095]); “The computer model can be built using a “top-down” approach that begins by defining a general set of behaviors indicative of a biological condition, e.g. a disease.” (Para. [0044]) and “One aspect of the invention provides computer-implemented methods of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject; (c) selecting one or more virtual patients from the virtual population based on a similarity between each of the selected virtual patients and the input data; (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user. In certain implementations, each virtual patient of the virtual population has an associated prevalence. In such cases, applying one or more virtual protocols to the one or more selected virtual patients can comprise calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients. The set of output can comprise the likelihood of each clinical outcome or the likelihood of a selected set of clinical outcomes, for example the most likely outcomes or a subset of outcomes representative of the range of clinical outcomes. In certain implementations, the virtual population is a prevalence-weighted virtual population, wherein each virtual patient of the virtual population has an associated prevalence weight. In a preferred implementation, the virtual protocol represents a stimulus selected from the group consisting of a therapeutic regimen, passage of time, exercise, weight gain, diet, a lifestyle choice and a combination of two or more of the same.” (Para. [0010]).
De Leon teaches determining modified virtual outcomes for the virtual individuals included in the plurality of virtual populations based on the virtual outcomes and the outcome definition input with “The method of claim 9, wherein each virtual patient of the virtual population has an associated prevalence.” (claim 10 of De Leon); “The method of claim 10, wherein applying one or more virtual protocols to the one or more selected virtual patients comprises calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients.” (claim 11 of De Leon) and “The method of claim 11, wherein the set of outputs comprises the likelihood of each clinical outcome.” (claim 12 of De Leon).
De Leon teaches …(d) modifying the computer model to reflect the function of the one or more identified genes; and (e) storing the modified computer model in a computer-readable storage medium. The computer model can modified to directly reflect the function of the one or more identified genes, to directly reflect absence of the function of the one or more identified genes or to indirectly reflect the downstream function of the one or more identified genes. In certain implementations, the method can further comprise (f) executing the modified computer model to generate a simulated outcome; and (g) comparing the simulated outcome with the known association between the genetic marker and clinical phenotype to confirm the validity of the modified computer model.” (Para. [0011]), but does not teach determining, by the computing system, a modified aggregate model that includes the plurality of models and a plurality of components that each correspond to an observer of the one or more observers.
De Leon teaches determining measures of difference between the modified virtual outcomes and observed outcomes included in the clinical study data with “…(f) executing the modified computer model to generate a simulated outcome; and (g) comparing the simulated outcome with the known association between the genetic marker and clinical phenotype to confirm the validity of the modified computer model. In such a case, comparing the simulated outcome with the known associate between the genetic marker and clinical phenotype optionally can comprise comparing a virtual population with a clinical population.” (Para. [0011]).
De Leon does not teach generating an aggregate model that includes a plurality of models included in the clinical study data and a respective weighting for each model of the plurality of models and determining a modified aggregate model that is responsive to the plurality of models and a plurality of components that each correspond to one of the one or more observers of claim 32. However, these limitations are taught by Barsoum.
De Leon also does not teach determining based on the measures of difference, individual weightings for individual models of the plurality of models and second individual weightings for individual components of the plurality of components of claim 32. However, this limitation is taught by Barhak.
De Leon also does not teach wherein determining the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models included therein to minimize the difference between the virtual outcomes and the outcome definition input, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models included therein, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques and wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the outcome definition input, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying of claim 32. However, these limitations are taught by Lacy.
Barsoum teaches generating an aggregate model that includes a plurality of models included in the clinical study data and a respective weighting for each model of the plurality of models with “FIG. 3 depicts an example of an aggregated model generation system 100 that can be utilized to create an aggregate model 102.” (para. [0060]) and with “The predictor variables can be analyzed to generate a model having a portion of the predictor variables with weighted coefficients according to an event or outcome for which the model is generated. A prediction tool can employ the model to predict the even or outcome for one or more patients.” (Abstract).
Barsoum teaches determining a modified aggregate model that is responsive to the plurality of models and a plurality of components that each correspond to the one or more observers with “FIG. 3 depicts an example of an aggregated model generation system 100 that can be utilized to create an aggregate model 102. The model generation system 100 can include a model modification method 104 that is programmed to modify an encounter-specific model 106, such as a corresponding model generated by the systems and methods of FIGS. 1 and 2 (e.g., model 38). The encounter-specific model 106 thus is generated for predicting a patient outcome based on analysis of model data generated from a plurality of patients' final coded data set that is stored in one or more sources of patient data. The encounter-specific model 106 thus can be utilized to predict an outcome generally for any patient. In some examples, longitudinal patient data 108 for a given patient may be relevant to determining coefficients and predictor variables relevant to predicting an outcome for the given patient. Thus, the model modification function 104 can modify the encounter-specific model 106 (e.g., generated by the model generator 36 of FIG. 2) and provide the aggregate model based upon the longitudinal patient data 108 for a given patient. That is, the model modification function 104 can adjust the model for a given patient depending on the given patient's circumstances.” (Para. [0060]).
Barhak teaches determining based on the measures of difference, individual weightings for individual components of the plurality of components with “C3) Weighing differences between model and observed study results allow the user to emphasize the importance of each specific outcome with a weight. For example it allows the user do emphasize the weight of a stroke or the weight of a death from stroke to be more important than that of Death from other causes.” (Col. 7. Para. 1).
Lacy teaches the claim limitation of wherein determining the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models included therein to minimize the difference between the virtual outcomes and the outcome definition input, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models included therein, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques and wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the outcome definition input, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying with “The data instances are weighted according to their classification accuracy, so that the second model to be fitted will be rewarded more highly if it manages to correctly predict the instances that the first model failed on. In this manner, the criteria of having an ensemble of classifiers who make their errors on disjoint parts of the data set is explicitly managed. After each iteration, the weights are updated to reflect the current ensemble’s performance.” (page 58, para. 2) and “Once a predetermined number of iterations has completed, the ensemble is formed and can be used to predict new patterns by aggregating all the members’ votes. AdaBoost (Freund et al., 1996, 1999) is the most well known algorithm in this field although numerous similar techniques exist under the name gradient boosting. It uses a WMV aggregation scheme, with the weights derived from the base classifiers’ performance on the training set. While boosting can be employed with any learning algorithm, it is frequently used to strong effect with decision trees; Breiman (1996b) explains the reasons for this success. Referring back to the bias-variance decomposition discussed in Section 3.3, boosting (as with many ensemble techniques including bagging) works most effectively by reducing the variance of low-bias classifiers (Johnson & Rayens, 2007).” (page 58, para. 2).
It would have been prima facia obvious to combine the teachings of De Leon and Barsoum to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include an aggregated model as taught by Barsoum for use in predicting one or more patient outcomes (Barsoum, Para. 0063). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barsoum teach methods that pertain to predicting patient outcomes.
It would have been also prima facia obvious to combine the teachings of De Leon and Barhak to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include determining the measures of difference by weight for models as taught by Barhak to emphasize the importance of each specific outcome (Barhak, Col. 7. Para. 1). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barhak teach methods that pertain to predicting patient outcomes.
It would also have been prima facia obvious to combine the teachings of De Leon and Lacy to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include iteratively modifying the weights of the different of models with gradient descent techniques as taught by Lacy for the benefit of optimizing the models’ performance. Furthermore, there would have been a reasonable expectation of success, since both De Leon and Lacy teach methods that pertain to the analysis of genetic data to predict disease.
Regarding claim 33, De Leon teaches determining a first fitness of a first set of initial conditions based at least partly on first results of a first number of simulations for a plurality of virtual populations with regard to the observed outcomes; and determining a second fitness of a second set of initial conditions based at least partly on second results of a second number of simulations for the plurality of virtual populations with regard to the observed outcomes with “The term “goodness-of-fit,” as used herein, refers to the similarity of two or more distributions, such as a prediction or simulation compared to an actual observation. Measures of goodness-of-fit include any method or process by which one quantifies and/or qualifies such similarity. Qualitative measures include visual inspection and comparison of plots or other graphical representations of the distributions. Quantitative measures include statistically rigorous methods by which one quantifies the total deviation of one set of values from another, for example, using a Chi-square test, G-test, Analysis of Covariance (ANCOVA), or Kolmogorov-Smimov test. Measures of goodness-of-fit can include both qualitative and quantitative aspects, such as non-parametric measures including ranked or categorized pairwise comparisons.” (Para. 0027).
Barhak also teaches determining a first fitness of a first set of initial conditions based at least partly on first results of a first number of simulations for a plurality of virtual populations with regard to the observed outcomes; and determining a second fitness of a second set of initial conditions based at least partly on second results of a second number of simulations for the plurality of virtual populations with regard to the observed outcomes with “…determining a first score for the first equation based on the first differences, the first score indicating a first fitness of the first equation to evaluate the progression of the disease in the plurality of populations; determining a second score for the second equations based on the second differences, the second score indicating a second fitness of the second equation to evaluate the progression of the disease in the plurality of populations;…” (Claim 11 of Barhak).
Regarding claim 34, De Leon teaches generating a user interface with “A diagram object can include documentation that provides a description of the diagram object, a collection of parameters, and methods which may define an equation or class or equations. The diagram objects each define a feature or object of a modeled system that is displayed within a diagram window presented by a graphical user interface (GUI) that interacts with the core.” (Para. 0041) and “The programmable processor, responsive to a request from a user interface to execute the computer model, retrieves at least a subset of variables, parameters and/or mathematical representations from the computer readable storage medium and applies the mathematical representations to the variables and parameters to generate a simulated effect.” (para. 0092).
De Leon teaches individual fitness scores for a population included in a clinical study of the plurality of clinical studies with “To evaluate similarity or correlation, a measure of goodness-of-fit between the common features for the virtual patients and the common features for the real subjects can be calculated. A measure of goodness-of-fit between the combinations of the common features for the virtual patients and the combinations of the common features for the real subjects can be calculated. The measure of goodness-of-fit can be a Chi-square test, G-test, Analysis of Covariance (ANCOVA), Kolmogorov-Smimov test, weighted coefficient of determination. The measure of goodness-of-fit can be a qualitative assessment of statistical properties of the common features for the virtual patients and the common features for the real subjects.” (Para 0064).
De Leon does not teach generating a user interface that indicates: the first individual weightings for individual models of the plurality of models; second individual weightings for individual components that correspond to individual users of the plurality of users of claim 34. However, this limitation is taught by Barsoum.
Barsoum teaches generating a user interface that indicates: the first individual weightings for individual models of the plurality of models; second individual weightings for individual components that correspond to individual users of the plurality of users with “A selection function 56 can in turn select from the available sets of weighting coefficients and predictor variables as determined by the weighting and ranking functions 54 and 52, respectively. The selection function 56, for example, can be utilized to select and generate the model 38.” (Para. 0056).
It would have been prima facia obvious to combine the teachings of De Leon and Barsoum to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include an interface that indicates weightings for individual models as taught by Barsoum to allow users to view the model’s weights. Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barsoum teach methods that pertain to predicting patient outcomes.
Regarding claim 35, De Leon teaches wherein the user interface includes one or more user interface elements that are selectable to modify at least one of one or more first individual weightings or one or more second individual weightings with “A diagram object can include documentation that provides a description of the diagram object, a collection of parameters, and methods which may define an equation or class or equations. The diagram objects each define a feature or object of a modeled system that is displayed within a diagram window presented by a graphical user interface (GUI) that interacts with the core.” (Para. 0041); “The programmable processor, responsive to a request from a user interface to execute the computer model, retrieves at least a subset of variables, parameters and/or mathematical representations from the computer readable storage medium and applies the mathematical representations to the variables and parameters to generate a simulated effect.” (para. 0092) and “…(c) generating virtual patients based on (i) the population constraints of the original virtual population and (ii) the one or more new axes of variation; (d) assigning prevalence weights to the virtual population, incorporating population statistics for the genetic marker as a constraint in the prevalence weighting process; and (e) generating an output comprising the virtual population and associated prevalence weights. In certain implementations, the virtual population is provided as data stored on a computer readable medium.” (Para. 0012) and “Once the one or more biological processes at issue have been identified, various virtual patients can be created by defining different modifications to one or more mathematical relations included in the computer model, which one or more mathematical relations represent the one or more biological processes. A modification to a mathematical relation can include, for example, a parametric change (e.g., altering or specifying one or more parameter values associated with the mathematical relation), altering or specifying behavior of one or more variables associated with the mathematical relation, altering or specifying one or more functions associated with the mathematical relation, or a combination of them. The computer model may be run based on a particular modification for a time sufficient to create a “stable” configuration of the computer model.” (para. 0058).
Regarding claim 36, De Leon teaches receiving input via the user interface to modify an individual first weighting with “(c) generating virtual patients based on (i) the population constraints of the original virtual population and (ii) the one or more new axes of variation; (d) assigning prevalence weights to the virtual population, incorporating population statistics for the genetic marker as a constraint in the prevalence weighting process; and (e) generating an output comprising the virtual population and associated prevalence weights. In certain implementations, the virtual population is provided as data stored on a computer readable medium.” Assigning weights of De Leon corresponds to input via the user interface.
De Leon teaches fitness scores with “To evaluate similarity or correlation, a measure of goodness-of-fit between the common features for the virtual patients and the common features for the real subjects can be calculated. A measure of goodness-of-fit between the combinations of the common features for the virtual patients and the combinations of the common features for the real subjects can be calculated. The measure of goodness-of-fit can be a Chi-square test, G-test, Analysis of Covariance (ANCOVA), Kolmogorov-Smimov test, weighted coefficient of determination. The measure of goodness-of-fit can be a qualitative assessment of statistical properties of the common features for the virtual patients and the common features for the real subjects.” (Para. 0064).
De Leon does not teach fitness scores based on the input received via the user interface. However, this limitation is taught by Barhak.
Barhak teaches determining modified fitness scores based on the input received via the user interface with “It is also possible to calculate the fitness score for a factor in the risk equation by using the coefficient of the factor as a weight in a weighted average. This way a factor with relatively larger coefficients in a certain equation be more influenced by the score of this equation. To make sure all equations use the same coefficients, a first order Taylor series expansion can be used, or the user can define those weights manually.” (col. 6, para. 8).
It would have been also prima facia obvious to combine the teachings of De Leon and Barhak to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include determining fitness scores as taught by Barhak to evaluate the progression of the disease in the plurality of populations (Barhak, Claim 11). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barhak teach methods that pertain to predicting patient outcomes.
Regarding independent claim 37, De Leon teaches obtaining, by a computing system including one or more hardware processors and memory, clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to predict one or more outcomes for a disease, and the one or more outcomes having a respective definition with “One aspect of the invention provides computer-implemented methods of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject;… (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user. In certain implementations, each virtual patient of the virtual population has an associated prevalence. In such cases, applying one or more virtual protocols to the one or more selected virtual patients can comprise calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients. The set of output can comprise the likelihood of each clinical outcome or the likelihood of a selected set of clinical outcomes, for example the most likely outcomes or a subset of outcomes representative of the range of clinical outcomes.” (Para. [0010]) and with “The identification of the one or more biological processes can be based on, for example, experimental or clinical data, scientific literature, genetic studies, results of a computer model, model sensitivity analysis, or a combination of them.” (para. [0058]).
De Leon teaches wherein the clinical study data for an individual clinical study of the plurality of clinical studies includes summary data indicating values of characteristics of one or more populations that participated in the individual clinical study with “Common features can include at least one continuous dependent variable and similarity can be evaluated by calculating one or more summary statistics for the continuous dependent variable for the real subjects, calculating the one or more summary statistics for the continuous dependent variable for the virtual patients according to their respective prevalences, and comparing the one or more summary statistics for the real subjects with the summary statistics for the virtual patients.” (Para. 0065).
De Leon teaches determining, by the computing system, a plurality of models from the clinical study data, wherein individual models of the plurality of models estimate progression of a disease and the progression of the disease includes a plurality of states with “The identification of the one or more biological processes can be based on, for example, experimental or clinical data, scientific literature, genetic studies, results of a computer model, model sensitivity analysis, or a combination of them. Once the one or more biological processes at issue have been identified, various virtual patients can be created by defining different modifications to one or more mathematical relations included in the computer model, which one or more mathematical relations represent the one or more biological processes.” (Para. 0058).
De Leon teaches generating, by the computing system, a plurality of virtual populations based on the summary data for the plurality of clinical studies, wherein individual virtual populations of the plurality of virtual populations: correspond to a respective clinical study of the plurality of clinical studies; include a plurality of virtual subjects; and individual subjects of the plurality of virtual subjects have values of at least one characteristic included in the summary data for the respective clinical study with “A collection of virtual patients, i.e. a virtual population, ideally is representative of a real population. If a sample population of real subjects is representative of the population, then the collection of virtual patients representing that sample population should be similar to the sample of real subjects from the population. For example, a collection of virtual patients has virtual patients that approximate the phenotypes observed in the sample population. In addition, the weighted frequency of each virtual patient in the virtual patient population is similar to the frequency of the corresponding real subject in the sample and, in this case, in the clinical population.” (para. 0059).
De Leon teaches generating, by the computing system, virtual outcomes for individual virtual subjects of the plurality of virtual subjects for the plurality of virtual populations based on the aggregate model; obtaining, by the computing system, user input from one or more observers, the user input indicating differences between first definitions of outcomes associated with the aggregate model and second definitions of outcomes associated with the plurality of clinical studies with “(d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient” (Claim 9 of De Leon).
De Leon does not teach generating, by the computing system, an aggregate model that includes a plurality of coefficients with individual coefficients of the plurality of coefficients corresponding to individual models of the plurality of models of claim of 37. However, this limitation is taught by Barsoum.
De Leon also does not teach determining, by the computing system, a number of correction factors for the virtual outcomes based on user input from the one or more observers of claim 37. However, this limitation is taught by Barhak.
De Leon also does not teach wherein generating the aggregate model includes iteratively modifying the coefficients to minimize the at least one characteristic, the iteratively modifying including applying, to the coefficients, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques and wherein determining the correction factors includes second iteratively modifying the correction factors in response to the user input, the second iteratively modifying being responsive to a minimum value of the minimized at least one characteristic, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying in claim 37. However, these limitations are taught by Lacy.
Barsoum teaches generating, by the computing system, an aggregate model that includes a plurality of coefficients with individual coefficients of the plurality of coefficients corresponding to individual models of the plurality of models with “FIG. 3 depicts an example of an aggregated model generation system 100 that can be utilized to create an aggregate model 102.” (para. [0060]) and with “The predictor variables can be analyzed to generate a model having a portion of the predictor variables with weighted coefficients according to an event or outcome for which the model is generated. A prediction tool can employ the model to predict the even or outcome for one or more patients.” (Abstract).
Barhak teaches determining, by the computing system, a number of correction factors for the virtual outcomes based on user input from the one or more observers with “FIG. 4 is a depiction of the correction term which accounts for model outdate and in the particular parameters of the study used in this example of The Reference Model. In this case, the parameters are adjusted for time past between; model year average of model data time interval, and simulated time stamp=simulated study year/s.” (col. 4, para. 4) and “The fitness score matrix uses color coding and ranking to visually demonstrate the fitness between 4/22 populations/cohorts and 64 combinations of published risk equations and hypotheses. The results show that different combinations of risk equations behave differently on different population cohorts. For each query, the system ranks the models. Models that implement the following two corrections generally behaved better: Temporal correction for treatment improvement; Biomarker change introduced in the first year.” (col. 5, para. 5).
Lacy teaches the claim limitation of teach wherein generating the aggregate model includes iteratively modifying the coefficients to minimize the at least one characteristic, the iteratively modifying including applying, to the coefficients, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques and wherein determining the correction factors includes second iteratively modifying the correction factors in response to the user input, the second iteratively modifying being responsive to a minimum value of the minimized at least one characteristic, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying with “The data instances are weighted according to their classification accuracy, so that the second model to be fitted will be rewarded more highly if it manages to correctly predict the instances that the first model failed on. In this manner, the criteria of having an ensemble of classifiers who make their errors on disjoint parts of the data set is explicitly managed. After each iteration, the weights are updated to reflect the current ensemble’s performance.” (page 58, para. 2) and “Once a predetermined number of iterations has completed, the ensemble is formed and can be used to predict new patterns by aggregating all the members’ votes. AdaBoost (Freund et al., 1996, 1999) is the most well known algorithm in this field although numerous similar techniques exist under the name gradient boosting. It uses a WMV aggregation scheme, with the weights derived from the base classifiers’ performance on the training set. While boosting can be employed with any learning algorithm, it is frequently used to strong effect with decision trees; Breiman (1996b) explains the reasons for this success. Referring back to the bias-variance decomposition discussed in Section 3.3, boosting (as with many ensemble techniques including bagging) works most effectively by reducing the variance of low-bias classifiers (Johnson & Rayens, 2007).” (page 58, para. 2).
It would have been prima facia obvious to combine the teachings of De Leon and Barsoum to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include an aggregated model as taught by Barsoum for use in predicting one or more patient outcomes (Barsoum, Para. 0063). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barsoum teach methods that pertain to predicting patient outcomes.
It would have been also prima facia obvious to combine the teachings of De Leon and Barhak to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include correction factors as taught by Barhak to improve the model’s performance (Barhak, Col. 5. Para. 5). Furthermore, there would have been a reasonable expectation of success, since both De Leon and Barhak teach methods that pertain to predicting patient outcomes.
It would also have been prima facia obvious to combine the teachings of De Leon and Lacy to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of De Leon to include iteratively modifying the weights of the different of models with gradient descent techniques as taught by Lacy for the benefit of optimizing the models’ performance. Furthermore, there would have been a reasonable expectation of success, since both De Leon and Lacy teach methods that pertain to the analysis of genetic data to predict disease.
Regarding claim 38, De Leon teaches obtaining, by the computing system, the summary data from at least one online database using a query with “One computer model that can be used in the clinical setting, called Archimedes, has been developed to simulate the complete healthcare environment, with every person, every doctor and every piece of equipment being represented and interacting as they do in reality. The Archimedes database contains vast amounts of data from numerous epidemiological and clinical trial studies. The data, in combination with the demographics of a virtual community health care system, and information about different treatments, progression of diabetes, medical personnel, facilities, and logistics of medical centers allow Archimedes users to evaluate multiple interventions, including; personal interventions like prevention, diagnosis, screening, treatment and support care, and organizational interventions such as quality improvement, care management, performance measurement, and changes in patient and practitioner behaviors.” (Para. 0007) and “The American Diabetes Association released an extension of Archimedes, called Diabetes PHD. When a person uses Diabetes PHD and provides information about himself or herself, the system creates a simulated person who has the same characteristics (e.g., sex, age, race/ethnicity), same features (e.g., height, weight, blood pressure), same laboratory test results (e.g., glucose, cholesterol), the same past medical history, family history, symptoms, complications, and the same treatments as the person providing the information. The system then takes this simulated version of the person and creates a thousand “identical looking” people.” (para. 0008).
De Leon teaches filtering, by the computing system, the summary data according to import instructions to produce filtered population information, wherein the query is included in the import instructions used to filter the population information with “In certain implementations of the invention, the virtual patients that most closely represent the real subject can be identified by (a) collecting data about the subject; (b) create a filter based on the subject's data; and (c) applying the filter to each virtual patient in a virtual population, wherein the filter identifies those virtual patients most closely representing the subject. The data collected from the subject can include physical data (such as height, weight or blood serum levels), genetic data (such as gender or genetic profile), environmental data (such as chemical exposure or location of residence) and lifestyle data (such as exercise regimen or diet). The data about the subject can be collected from manual entry, directly from devices or diagnostic equipment, or from an electronic data store such as an electronic medical record or personal health record system. In certain implementations, the filter created based on the subject's data can be a standard deviation or other representation of noise in the measurements to create a filter that has a range of values rather than requiring the virtual patient to match an absolute value.” (Para. 0067).
Regarding claim 39, De Leon teaches formatting, by the computing system, the filtered population information according to a predetermined template to produce formatted population information; and merging, by the computing system, the formatted population information with prior population information stored in a template file with “In certain implementations, the virtual population is provided as data stored on a computer readable medium.” (para. 0012).
Regarding claim 40, De Leon teaches generating a first object that includes one or more first rules related to determining values of characteristics and includes one or more first objectives defining statistics for a first additional virtual population of the plurality of populations; and generating a second object that includes one or more second rules related to determining values of characteristics and includes one or more second objectives defining statistics related to a second additional virtual population of the plurality of populations, wherein the virtual population is an object that inherits from the first object and the second object with “Accordingly, the core is shown to comprise classes of objects, namely diagram objects, access panel objects, layer panel objects, monitor panel objects, chart objects, configuration objects, experiment protocol objects, and measurement objects. As is well known within the art, each object within the core may comprise a collection of parameters (also commonly referred to as instances, variables or fields) and a collection of methods that utilize the parameters of the relevant object.” (Para. 0040) and “According to one implementation, the diagram objects may include state, function, modifier and link objects, which are represented respectively by state nodes, function nodes, modifier icons and link icons within the diagram window. Each object defined within the software core can have at least one parameter associated therewith which quantifies certain characteristics of the object, and which is used during simulation of the modeled system. It will also be appreciated that not all objects must include a parameter. In one implementation, several types of parameters are defined. Firstly, system parameters may be defined for each subject type. For example, a system parameter may be assigned an initial value for a state object, or a coefficient value for a link object. Other parameter types include object parameters and diagram parameters that facilitate easy manipulation of values in simulation operations.” (para. 0042)
Response to 35 U.S.C. 103 Rejection Arguments (Remarks filed 12/22/2025, page 21-23)
Applicant amened claims 21, 24-25, 27-28, 31-34, 36-38.
It is noted that Applicant’s remarks are based on amended claims.
In the Remarks, Applicant states that none of the references show the recited amended claim limitations.
In response, Applicant’s remarks are persuasive. Therefore, the rejections have been withdrawn. However, upon further consideration, a new ground of rejection is made in view of claim amendments as discussed above.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 21-25, 32-33 and 37-40 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-4, 9-10, 12 and 15-17 of U.S. Patent No. 10923234 (App. No. 15466535). Although the claims at issue are not identical, they are not patentably distinct from each other because both sets of claims are directed towards generating aggregated models for determining outcomes of virtual individuals. Overall, the difference is that the claims of the instant application are broader in scope than the claims of the reference application and thus the instant claims are anticipated by the reference application (see MPEP 804.II.B.2). See table below for a mapping of the claims of U.S. Patent No. 10923234 (App. No. 15466535) that anticipate the claims of the instant application.
Instant App. 17176152, 06/30/2025
U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020)
Claims 21-25, 32, 37-38
Claims 1, 4, 9-10, 14-15
21. (currently amended) A method comprising:
obtaining, by a computing system including one or more hardware processors and memory, clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model
to predict one or more outcomes for a disease, and the one or more outcomes having a respective definition;
generating, by the computing system, an aggregate model that includes a plurality of models included in the clinical study data and a respective weighting for each model of the plurality of models;
generating, by the computing system and using the aggregate model, virtual outcomes for virtual individuals included in a plurality of virtual populations,
wherein the virtual outcomes are selected from a set of virtual outcomes and each virtual outcome of the set of virtual outcomes corresponds to an additional definition associated with the aggregate model;
obtaining, by the computing system, outcome definition input from a plurality of observers,
the outcome definition input indicating measures of difference between the respective definitions of the one or more outcomes included to the plurality of clinical studies and additional definitions of the virtual outcomes included in the set of virtual outcomes;
determining, by the computing system, a difference between the virtual outcomes and the outcome definition input;
wherein determining the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models included therein to minimize the difference between the virtual outcomes and the outcome definition input, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models included therein, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques;
determining, by the computing system, a modified aggregate model that is responsive to the plurality of models and a plurality of components that each correspond to one of the one or more observers;
determining, by the computing system, measures of difference between the modified virtual outcomes and observed outcomes included in the clinical study data; and
determining by the computing system and based on the measures of difference, individual weightings for individual ones of the plurality of components;
wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the outcome definition input, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying.
22. (previously presented) The method of The method of wherein the clinical study data includes, for individual clinical studies, population summary data indicating values of characteristics of one or more populations that participated in the individual clinical study, and the method comprising: generating, by the computing system, the characteristics of the virtual individuals included in a virtual population of the plurality of virtual populations based on the summary data for a clinical study of the plurality of clinical studies
23. (previously presented) The method of claim 22, wherein the virtual outcomes for the clinical study are determined using the virtual population; and the number of virtual individuals associated with each outcome are determined and multiple measures of difference are determined by determining a difference between the number of virtual individuals having the outcome in relation to the number of observed outcomes for the population of the clinical study.
24. (previously presented) The method of claim 22, wherein the outcome definition input includes a value indicating a first measure of difference between a definition of a first outcome related to the aggregate model and a definition of a first additional outcome included in the clinical study.
25. (previously presented) The method of claim 21, comprising iteratively modifying the weightings for the observers by calculating a difference between the virtual outcomes and the observed outcomes.
37. (currently amended) A method comprising:
obtaining, by a computing system including one or more hardware processors and memory, clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to predict one or more outcomes for a disease, and the one or more outcomes having a respective definition;
wherein the clinical study data for an individual clinical study of the plurality of clinical studies includes summary data indicating values of characteristics of one or more populations that participated in the individual clinical study;
determining, by the computing system, a plurality of models from the clinical study data, wherein individual models of the plurality of models estimate progression of a disease and the progression of the disease includes a plurality of states;
generating, by the computing system, a plurality of virtual populations based on the summary data for the plurality of clinical studies, wherein individual virtual populations of the plurality of virtual populations:
correspond to a respective clinical study of the plurality of clinical studies;
include a plurality of virtual subjects; and
individual subjects of the plurality of virtual subjects have values of at least one characteristic included in the summary data for the respective clinical study;
generating, by the computing system, an aggregate model that includes a plurality of coefficients with individual coefficients of the plurality of coefficients corresponding to individual models of the plurality of models,
wherein generating the aggregate model includes iteratively modifying the coefficients to minimize the at least one characteristic, the iteratively modifying including applying, to the coefficients, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques;
generating, by the computing system, virtual outcomes for individual virtual subjects of the plurality of virtual subjects for the plurality of virtual populations based on the aggregate model;
obtaining, by the computing system, user input from one or more observers, the user input indicating differences between first definitions of outcomes associated with the aggregate model and second definitions of outcomes associated with the plurality of clinical studies;
determining, by the computing system, a number of correction factors for the virtual outcomes based on user input from the one or more observers,
wherein determining the correction factors includes second iteratively modifying the correction factors in response to the user input, the second iteratively modifying being responsive to a minimum value of the minimized at least one characteristic, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying.
38. (currently amended) The method of claim 37, further comprising: obtaining, by the computing system, the summary data from at least one online database using a query; and filtering, by the computing system, the summary data according to import instructions to produce filtered population information, wherein the query is included in the import instructions used to filter the population information.
32. (currently amended) A computing system comprising:
one or more processing units;
one or more computer-readable storage media storing computer-readable instructions that, when executed by the one or more processing units, cause the computing system to perform operations comprising:
obtaining clinical study data that corresponds to a plurality of clinical studies related to a disease, the clinical study data for an individual clinical study of the plurality of clinical studies indicating a model to
estimate one or more outcomes for a disease, and the one or more outcomes having a respective definition;
generating an aggregate model that includes a plurality of models included in the clinical study data and a respective weighting for each model of the plurality of models;
generating, using the aggregate model, virtual outcomes for virtual individuals included in a plurality of virtual populations, wherein the virtual outcomes are selected from a set of virtual outcomes and each virtual outcome of the set of virtual outcomes corresponds to an additional definition associated with the aggregate model;
wherein generating the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models to minimize a difference between the virtual outcomes and the additional definition associated with the aggregate model, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques;
obtaining outcome definition input from a plurality of observers, the outcome definition input indicating measures of difference between the respective definitions of the outcomes included to the plurality of clinical studies and a set of definitions of the virtual outcomes included in the set of virtual outcomes;
determining modified virtual outcomes for the virtual individuals included in the plurality of virtual populations based on the virtual outcomes and the outcome definition input;
determining a modified aggregate model that is responsive to the plurality of models and a plurality of additional components that each correspond to one of the one or more observers;
determining measures of difference between the modified virtual outcomes and observed outcomes included in the clinical study data; and
determining, based on the measures of difference, individual weightings for individual components of the plurality of components;
wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the additional definition associated with the aggregate model, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying.
1. (Currently Amended) A method comprising: obtaining, from at least one online database, clinical study information, the clinical study information including information corresponding to a number of models that predict a progression of one or more diseases and population summary data that indicates characteristics of groups of individuals involved in a plurality of clinical studies with respect to the one or more diseases; identifying a first model from among the number of models that predicts [[a]] the progression of a disease of the one or more diseases, wherein the first model is derived from a first portion of the clinical study information and the progression of the disease includes a plurality of states; identifying a second model from among the number of models that predicts the progression of the disease, wherein the second model is derived from a second portion of the clinical study information; generating an aggregate model that includes a first coefficient corresponding to the first model and a second coefficient corresponding to the second model; generating, based on the population summary data, a virtual population including a number of virtual individuals; determining, based on data related to virtual individuals included in the virtual population, first simulated outcomes of the aggregate model using a first value of the first coefficient and a second value of the second coefficient, wherein: individual first simulated outcomes of the first simulated outcomes indicate a first probability of a virtual individual of the virtual population progressing from one state of the disease to another state of the disease over a period of time, the first value of the first coefficient corresponds to a first amount of contribution of the first model in determining the first simulated outcomes; and the second value of the second coefficient corresponds to a second amount of contribution of the second model in determining the first simulated outcomes; analyzing the first simulated outcomes with respect to first observed outcomes obtained from the clinical study information to determine first differences between the first simulated outcomes and the first observed outcomes; modifying, based on the first differences, at least one of the first coefficient or the second coefficient to determine second simulated outcomes of the aggregate model based on the data related to the virtual individuals included in the virtual population; and determining a measure of fitness of the aggregate model based on differences between the second simulated outcomes and second observed outcomes obtained from the clinical study information and based on a fitness function corresponding to the aggregate model moving toward a local minimum.
4. (Currently Amended) The method of claim 1, wherein: clinical study information includes at least one first clinical study that includes the first model and at least one second clinical study that includes the second model; and the population summary data includes summary information including at least one statistical measure for at least one characteristic of one or more groups of individuals included in at least one of the at least one first clinical study or the at least one second clinical study.
9. (Currently Amended) A method comprising: obtaining, from at least one online database, clinical study information, the clinical study information including information corresponding to a number of models that predict a progression of one or more biological conditions and population summary data that indicates characteristics of groups of individuals involved in a plurality of clinical studies with respect to the one or more biological conditions; identifying, from among the number of models, a plurality of models that predict a progression of a biological condition of the one or more biological conditions; generating an aggregate model that includes a plurality of coefficients, individual coefficients of the plurality of coefficients indicating an individual contribution of an individual model of the plurality of models to a fitness function of the aggregate model; generating one or more virtual populations based on the population summary data from at least a portion of the population information; performing a plurality of iterations of an optimization process for the fitness function, individual iterations of the plurality of iterations including: determining a respective value of one or more coefficients of the plurality of coefficients, determining simulated outcomes for the aggregate model for virtual individuals included in a virtual population of the one or more virtual populations, individual simulated outcomes being influenced by a probability of a virtual individual included in the virtual population progressing from at least one state of the biological condition to at least one additional state of the biological condition over a period of time; and determining a measure of fitness of the aggregate model for the individual iteration based on differences between observed outcomes of individuals included in the clinical study information and the simulated outcomes for the individual iteration of the optimization process, the measure of fitness to move the fitness function toward a local minimum; and determining, after completion of the optimization process, values of respective coefficients of the plurality of coefficients at the local minimum of the fitness function.
10. (Currently Amended) The method of claim 9, wherein first simulated outcomes of a first iteration of the plurality of iterations of the optimization process are determined using a first set of initial conditions that include a first value of a first coefficient of a first model included in the aggregate model and a second value of a second coefficient of a second model included in the aggregate model, and the operations further comprise: determining second simulated outcomes of a second iteration of the plurality of iterations of the optimization process utilizing the aggregate model and the virtual population and that utilize a second set of initial conditions that include a first additional value of the first coefficient of the first model and a second additional value of the second coefficient of the second model.
14. (Currently Amended) The method of claim 9, wherein the local minimum is determined using a gradient descent algorithm such that the individual models of the plurality of models cooperate during the optimization process.
15. (Currently Amended) A system comprising: one or more processing units; memory including computer-readable instructions that when executed by the one or more processing units perform operations comprising: obtaining, from at least one online database, clinical study information, the clinical study information including information corresponding to a number of models that predict a progression of one or more one or more biological conditions and population summary data that indicates characteristics of groups of individuals involved in a plurality of clinical studies with respect to the one or more biological conditions; identifying, from among the number of models, a plurality of models that predict a progression of a biological condition of the one or more biological conditions;
generating an aggregate model that includes a plurality of coefficients, individual coefficients of the plurality of coefficients indicating an individual contribution of an individual model of the plurality of models to a fitness function of the aggregate model; generating [[a]] one or more virtual populations based on the population summary data; performing a plurality of iterations of an optimization process for the fitness function, individual iterations of the plurality of iterations including: determining a respective value of one or more coefficients of the plurality of coefficients, determining simulated outcomes for the aggregate model for virtual individuals included in a virtual population of the one or more virtual populations, individual simulated outcomes influenced by a probability of a virtual individual included in the virtual population progressing from at least one state of the biological condition to at least one additional state of the biological condition over a period of time; and
determining a measure of fitness of the aggregate model for the individual iteration based on differences between observed outcomes of individuals included in the clinical study information and the simulated outcomes for the individual iteration of the optimization process, the measure of fitness to move the fitness function toward a local minimum; and
determining, after completion of the optimization process, values of respective coefficients of the plurality of coefficients at the local minimum of the fitness function.
Claim 33
Claim 12
33. (previously presented) The computing system of claim 32, wherein the one or more computer-readable media store instructions that, when executable by the one or more processing units, cause the computing system to perform operations comprising: determining a first fitness of a first set of initial conditions based at least partly on first results of a first number of simulations for a plurality of virtual populations with regard to the observed outcomes; and determining a second fitness of a second set of initial conditions based at least partly on second results of a second number of simulations for the plurality of virtual populations with regard to the observed outcomes.
12. determining a first measure of fitness for the aggregate model with respect to[[of]] the first set of initial conditions based at least partly on the first simulated outcomes with regard to the observed outcomes; determining a second measure of fitness for the aggregate model with respect to[[of]] the second set of initial conditions based at least partly on the second simulated outcomes with regard to the observed outcomes; and comparing the first measure of fitness with the second measure of fitness.
Claim 38
Claim 2
38. (currently amended) The method of claim 37, further comprising: obtaining, by the computing system, the summary data from at least one online database using a query; and filtering, by the computing system, the summary data according to import instructions to produce filtered population information, wherein the query is included in the import instructions used to filter the population information.
2. (Currently Amended) The method of claim 1, further comprising: obtaining the population summary data from at least one online database using a query; and filtering the population information according to import instructions to produce filtered population summary data, wherein the query is included in the import instructions used to filter the population summary data.
Claim 39
Claim 3
39. (original) The method of claim 38, further comprising: formatting, by the computing system, the filtered population information ac- cording to a predetermined template to produce formatted population information; and merging, by the computing system, the formatted population information with prior population information stored in a template file.
3. (Currently Amended) The method of claim 2, further comprising: formatting the filtered population summary data according to a predetermined template to produce formatted population summary data; and merging the formatted population summary data with prior population summary data stored in a template file.
Claim 40
Claims 16 and 17
40. (original) The method of claim 37, comprising: generating a first object that includes one or more first rules related to determining values of characteristics and includes one or more first objectives defining statistics for a first additional virtual population of the plurality of populations; and generating a second object that includes one or more second rules related to determining values of characteristics and includes one or more second objectives defining statistics related to a second additional virtual population of the plurality of populations. wherein the virtual population is an object that inherits from the first object and the second object.
16. (Currently Amended) The system of claim 15, wherein the operations further comprise: generating a first object that includes one or more first rules related to determining values of characteristics of first virtual individuals and includes one or more first objectives defining statistics for a first group of individuals included in the clinical study information; and generating a second object that includes one or more second rules related to determining values of characteristics second virtual individuals and includes one or more second objectives defining statistics related to a second group of individuals included in the clinical study information.
17. (Original) The system of claim 16, wherein the virtual population is an object that inherits from the first object and the second object.
Claims 27-31 and 34-36 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 4, 9 and 15 of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) in view of De Leon, Barsoum and Barhak.
U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) does not teach iteratively modifying a set of weightings associated with the observers by calculating a difference between the virtual outcomes and the observed outcomes of claim 25; wherein the observer input indicates a proportion of a clinical study outcome that corresponds to a model outcome of claim 28; generating a user interface of claim 34; wherein the user interface includes one or more user interface elements that are selectable to modify at least one of one or more first individual weightings or one or more second individual weightings of claim 35; and receiving input via the user interface to modify an individual first weighting of claim 36. However, these limitations are taught by De Leon.
U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) also does not teach wherein the proportion of the clinical study outcome that corresponds to the model outcome is a scalar number of claim 29 and wherein the scalar number is greater than one, less than one, or equal to one of claim 30; generating a user interface that indicates: the first individual weightings for individual models of the plurality of models; and second individual weightings for individual components that correspond to individual users of the plurality of users of claim 34. However, this limitation is taught by Barsoum.
U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) also does not teach wherein the virtual outcomes are generated by performing one or more simulations with respect to at least portion of the plurality of populations using one or more Monte Carlo techniques of claim 27 and wherein initial weights for the user input are approximately equal of claim 31; and determining modified fitness scores based on the input received via the user interface of claim 36. However, these limitations are taught by Barhak.
Regarding claim 25, De Leon teaches iteratively modifying a set of weightings associated with the observers by calculating a difference between the virtual outcomes and the observed outcomes with “The term “goodness-of-fit,” as used herein, refers to the similarity of two or more distributions, such as a prediction or simulation compared to an actual observation. Measures of goodness-of-fit include any method or process by which one quantifies and/or qualifies such similarity.” (para. [0027]) and “To evaluate similarity or correlation, a measure of goodness-of-fit between the common features for the virtual patients and the common features for the real subjects can be calculated. A measure of goodness-of-fit between the combinations of the common features for the virtual patients and the combinations of the common features for the real subjects can be calculated.” (para. [0064]) and “The term “prevalence,” as used herein to describe a virtual patient, indicates the occurrence, e.g. the frequency of occurrence, of that virtual patient in a virtual patient population. The prevalence of any particular virtual patient in a virtual patient population can be defined by a weighting factor or weight, wherein each weight adjusts for over- or under-representation of the characteristics of the virtual patient in the population. The prevalence of a virtual patient relates to the likelihood that there is a real subject in the population with characteristics of or similar to the virtual patient.” (Para. 0026).
Regarding claim 28, De Leon teaches wherein the observer input indicates a proportion of a clinical study outcome that corresponds to a model outcome with “One aspect of the invention provides computer-implemented methods of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject; (c) selecting one or more virtual patients from the virtual population based on a similarity between each of the selected virtual patients and the input data; (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user. In certain implementations, each virtual patient of the virtual population has an associated prevalence. In such cases, applying one or more virtual protocols to the one or more selected virtual patients can comprise calculating a likelihood of each clinical outcome based upon the prevalence of the one or more virtual patients. The set of output can comprise the likelihood of each clinical outcome or the likelihood of a selected set of clinical outcomes, for example the most likely outcomes or a subset of outcomes representative of the range of clinical outcomes. In certain implementations, the virtual population is a prevalence-weighted virtual population, wherein each virtual patient of the virtual population has an associated prevalence weight. In a preferred implementation, the virtual protocol represents a stimulus selected from the group consisting of a therapeutic regimen, passage of time, exercise, weight gain, diet, a lifestyle choice and a combination of two or more of the same.” (para. [0068]) and “A computer-implemented method of predicting a clinical outcome for a subject comprising: (a) providing a virtual population comprising a plurality of virtual patients; (b) receiving input data about a subject; (c) selecting one or more virtual patients from the virtual population based on a similarity between each of the selected virtual patients and the input data; (d) applying one or more virtual protocols to the one or more selected virtual patients to generate a set of outputs projecting a clinical outcome for the subject, wherein a set of outputs is generated for each selected virtual patient; and (e) reporting the set of outputs to a user.” (Claim 9 of De Leon)
Regarding claim 35, De Leon teaches wherein the user interface includes one or more user interface elements that are selectable to modify at least one of one or more first individual weightings or one or more second individual weightings with “A diagram object can include documentation that provides a description of the diagram object, a collection of parameters, and methods which may define an equation or class or equations. The diagram objects each define a feature or object of a modeled system that is displayed within a diagram window presented by a graphical user interface (GUI) that interacts with the core.” (Para. 0041); “The programmable processor, responsive to a request from a user interface to execute the computer model, retrieves at least a subset of variables, parameters and/or mathematical representations from the computer readable storage medium and applies the mathematical representations to the variables and parameters to generate a simulated effect.” (para. 0092) and “…(c) generating virtual patients based on (i) the population constraints of the original virtual population and (ii) the one or more new axes of variation; (d) assigning prevalence weights to the virtual population, incorporating population statistics for the genetic marker as a constraint in the prevalence weighting process; and (e) generating an output comprising the virtual population and associated prevalence weights. In certain implementations, the virtual population is provided as data stored on a computer readable medium.” (Para. 0012) and “Once the one or more biological processes at issue have been identified, various virtual patients can be created by defining different modifications to one or more mathematical relations included in the computer model, which one or more mathematical relations represent the one or more biological processes. A modification to a mathematical relation can include, for example, a parametric change (e.g., altering or specifying one or more parameter values associated with the mathematical relation), altering or specifying behavior of one or more variables associated with the mathematical relation, altering or specifying one or more functions associated with the mathematical relation, or a combination of them. The computer model may be run based on a particular modification for a time sufficient to create a “stable” configuration of the computer model.” (para. 0058).
It would have been prima facia obvious to combine the teachings of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) with De Leon to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) to include a user input and calculating a difference between the virtual outcomes and the observed outcomes as taught by De Leon to determine similarity or correlation, a measure of goodness-of-fit between the common features for the virtual patients and the common features for the real subjects. A person of ordinary skill in the art would have been motivated to modify the method of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) to take user inputs as taught by De Leon to predict a clinical outcome for a subject. Furthermore, there would have been a reasonable expectation of success, since U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) and De Leon teach methods that pertain to predicting patient outcomes.
Regarding claim 36, U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) teaches fitness scores with claims 1, 9, 12 and 15.
U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) does not teach receiving input via the user interface to modify an individual first weighting and determining modified fitness scores based on the input received via the user interface of claim 36. However, this limitation is taught by De Leon and Barsoum.
De Leon teaches receiving input via the user interface to modify an individual first weighting with “(c) generating virtual patients based on (i) the population constraints of the original virtual population and (ii) the one or more new axes of variation; (d) assigning prevalence weights to the virtual population, incorporating population statistics for the genetic marker as a constraint in the prevalence weighting process; and (e) generating an output comprising the virtual population and associated prevalence weights. In certain implementations, the virtual population is provided as data stored on a computer readable medium.” Assigning prevalence weights of De Leon corresponds to input via the user interface.
Barhak teaches determining modified fitness scores based on the input received via the user interface with “It is also possible to calculate the fitness score for a factor in the risk equation by using the coefficient of the factor as a weight in a weighted average. This way a factor with relatively larger coefficients in a certain equation be more influenced by the score of this equation. To make sure all equations use the same coefficients, a first order Taylor series expansion can be used, or the user can define those weights manually.” (col. 6, para. 8).
It would have been also prima facia obvious to combine the teachings of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) with De Leon and Barhak to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) to include user input to modify individual weightings as taught by De Leon to provide constraints in the weighting process. A person of ordinary skill in the art would have also been motivated to modify the method of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) to include determining modified fitness scores based on the input received via the user interface as taught by Barhak to ensure all equations use the same coefficients for determining fitness scores. Furthermore, there would have been a reasonable expectation of success, since U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020), De Leon and Barhak teach methods that pertain to predicting patient outcomes.
Regarding claim 34, U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) teaches individual fitness scores for a population included in a clinical study of the plurality of clinical studies with claims 1, 9, 12 and 15.
U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) does not teach generating a user interface that indicates: the first individual weightings for individual models of the plurality of models; second individual weightings for individual components that correspond to individual users of the plurality of users of claim 34. However, this limitation is taught by De Leon and Barsoum.
De Leon teaches generating a user interface with “A diagram object can include documentation that provides a description of the diagram object, a collection of parameters, and methods which may define an equation or class or equations. The diagram objects each define a feature or object of a modeled system that is displayed within a diagram window presented by a graphical user interface (GUI) that interacts with the core.” (Para. 0041) and “The programmable processor, responsive to a request from a user interface to execute the computer model, retrieves at least a subset of variables, parameters and/or mathematical representations from the computer readable storage medium and applies the mathematical representations to the variables and parameters to generate a simulated effect.” (para. 0092).
Barsoum teaches generating a user interface that indicates: the first individual weightings for individual models of the plurality of models; second individual weightings for individual components that correspond to individual users of the plurality of users with “A selection function 56 can in turn select from the available sets of weighting coefficients and predictor variables as determined by the weighting and ranking functions 54 and 52, respectively. The selection function 56, for example, can be utilized to select and generate the model 38.” (Para. 0056).
It would have been prima facia obvious to combine the teachings of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) with De Leon and Barsoum to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) to include an interface that indicates weightings for individual models as taught by De Leon and Barsoum to allow users to view the model’s weights. Furthermore, there would have been a reasonable expectation of success, since U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020), De Leon and Barsoum teach methods that pertain to predicting patient outcomes.
Regarding claim 29, Barsoum teaches wherein the proportion of the clinical study outcome that corresponds to the model outcome is a scalar number with “For instance, the output generator 42 can compare the predicted outcome to one more thresholds, such as can vary depending on the outcome for which the model has been generated.” (Para. [0050]) and “As one example, some types of models, such as for diagnosing a medical condition, may have a single threshold (e.g., a risk threshold), which if the value of the predicted outcome computed by the prediction tool 40 exceeds the threshold, the output generator 42 can provide an output identifying the diagnosis for the given patient. The output generator 42 can employ multiple thresholds for models generated for other types of outcomes (e.g., readmission risk, patient satisfaction, length of stay and the like). For assessments based on these types of predicted outcomes, the output generator 42 can vary the output that is generated based on the value of the predicted outcome relative to the predicted outcome to the thresholds that have been set. Thus, as the risk of such an outcome increases (as determined relative to predetermined thresholds), the output can increase in scale commensurately with such risk. For instance, different graphical representations of such risk can be provided and/or can be color coded (e.g., yellow, orange, red) to indicate the level of severity. Other types of severity scales and risk indicators can be utilized, which can include providing a normalized scale of the value of the predicted outcome (e.g., as a percentage). By employing a variety of models various types of outcomes can be predicted for each patient in real-time during a patient's stay in the hospital and thereby help to mitigate the risk of negative outcomes and increase the likelihood of positive outcomes.” (Para. [0051]).
Regarding claim 30, Barsoum teaches wherein the scalar number is greater than one, less than one, or equal to one with “For instance, the output generator 42 can compare the predicted outcome to one more thresholds, such as can vary depending on the outcome for which the model has been generated.” (Para. [0050]) and “As one example, some types of models, such as for diagnosing a medical condition, may have a single threshold (e.g., a risk threshold), which if the value of the predicted outcome computed by the prediction tool 40 exceeds the threshold, the output generator 42 can provide an output identifying the diagnosis for the given patient. The output generator 42 can employ multiple thresholds for models generated for other types of outcomes (e.g., readmission risk, patient satisfaction, length of stay and the like). For assessments based on these types of predicted outcomes, the output generator 42 can vary the output that is generated based on the value of the predicted outcome relative to the predicted outcome to the thresholds that have been set. Thus, as the risk of such an outcome increases (as determined relative to predetermined thresholds), the output can increase in scale commensurately with such risk. For instance, different graphical representations of such risk can be provided and/or can be color coded (e.g., yellow, orange, red) to indicate the level of severity. Other types of severity scales and risk indicators can be utilized, which can include providing a normalized scale of the value of the predicted outcome (e.g., as a percentage). By employing a variety of models various types of outcomes can be predicted for each patient in real-time during a patient's stay in the hospital and thereby help to mitigate the risk of negative outcomes and increase the likelihood of positive outcomes.” (Para. [0051]).
It would have been prima facia obvious to combine the teachings of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) and Barsoum to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) to modify the weights of the different models of the aggregated model as taught by Barsoum to adjust the model for a given patient depending on the given patient's circumstances. (Barsoum, Para. 0060). A person of ordinary skill in the art would also have been motivated to modify the method of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) to represent the clinical outcome as a scalar number as taught by Barsoum to easily provide a representation of the level of risk and severity (Barsoum, Para. 0051). Furthermore, there would have been a reasonable expectation of success, since U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) and Barsoum teach methods that pertain to predicting patient outcomes.
Regarding claim 27, Barhak teaches wherein the virtual outcomes are generated by performing one or more simulations with respect to at least a portion of the plurality of populations using one or more Monte Carlo techniques with “The Reference Model is built from publicly available data, using MIST (Micro-Simulation Tool)—a Python based modeling framework which is available under General Public License (GPL), and does not require access to proprietary, or individual patient, information. The software uses Monte Carlo simulations that are executed in parallel, and in the data shown in this application, the system can run on a single machine, on a cluster of machines, and on a cluster in the cloud. With the model utilizing computer power/techniques, such as parallel processing/high performance computing, cross validation, competition among alternative equations and/or hypotheses combinations, and ranks results based on fitness via a fitness engine multiple studies can be compared and large amounts of study data, which was inaccessible before, is now available to determine disease progression over large populations comprising large data sets.” (Col. 3, para. 6).
Regarding claim 31, Barhak teaches wherein a set of initial weights for the outcome definition input are equal with “To make sure all equations use the same coefficients, a first order Taylor series expansion can be used, or the user can define those weights manually.” (Col. 6, para. 8).
It would also have been prima facia obvious to combine the teachings of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) and Barhak to arrive at the claimed invention. A person of ordinary skill in the art would have been motivated to modify the method of U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) to include using Monte Carlo techniques for simulations to generate virtual outcomes as taught by Barhak to execute simulations in parallel (Barhak, Col. 3. Para. 6). Furthermore, there would have been a reasonable expectation of success, since both U.S. Patent No. 10923234 (App. No. 15466535, 07/29/2020) and Barhak teach methods that pertain to predicting patient outcomes.
Response to Double Patenting Rejection Arguments (Remarks filed 12/22/2025, page 24)
Applicant amened claims 21, 24-25, 27-28, 31-34, 36-38.
It is noted that Applicant’s remarks are based on amended claims.
In the Remarks, Applicant states that the rewritten claims 21, 32 and 37 have different claim recitations and are not obvious in view of the cited reference claims.
In response, Applicant’s remarks are not persuasive. The newly added limitations to independent claims 21, 32 and 37 are taught by claims 9-10 of reference application, U.S. Patent No. 10923234 (App. No. 15466535, claim set 07/29/2020) as indicated above.
For instance, the following new limitation in the instant amended independent claim 21 of wherein determining the aggregate model includes first iteratively modifying the weightings of the different ones of the plurality of models included therein to minimize the difference between the virtual outcomes and the outcome definition input, the first iteratively modifying including applying, to the respective weighting for each one of the plurality of models included therein, in response to simulation using the aggregate model, one or more of gradient descent techniques or evolutionary computation techniques and wherein determining the modified aggregate model includes second iteratively modifying the individual weightings for individual ones of the plurality of components in response to the measures of difference between the modified virtual outcomes and the observed outcomes, the second iteratively modifying being responsive to a minimum value of the difference between the virtual outcomes and the outcome definition input, the second iteratively modifying including applying, to the second individual weightings, one or more of gradient descent techniques or evolutionary computation techniques, independently of the first iteratively modifying corresponds to:
performing a plurality of iterations of an optimization process for the fitness function, individual iterations of the plurality of iterations including: determining a respective value of one or more coefficients of the plurality of coefficients, determining simulated outcomes for the aggregate model for virtual individuals included in a virtual population of the one or more virtual populations, individual simulated outcomes being influenced by a probability of a virtual individual included in the virtual population progressing from at least one state of the biological condition to at least one additional state of the biological condition over a period of time; and determining a measure of fitness of the aggregate model for the individual iteration based on differences between observed outcomes of individuals included in the clinical study information and the simulated outcomes for the individual iteration of the optimization process, the measure of fitness to move the fitness function toward a local minimum; and determining, after completion of the optimization process, values of respective coefficients of the plurality of coefficients at the local minimum of the fitness function in reference application claim 9; wherein first simulated outcomes of a first iteration of the plurality of iterations of the optimization process are determined using a first set of initial conditions that include a first value of a first coefficient of a first model included in the aggregate model and a second value of a second coefficient of a second model included in the aggregate model, and the operations further comprise: determining second simulated outcomes of a second iteration of the plurality of iterations of the optimization process utilizing the aggregate model and the virtual population and that utilize a second set of initial conditions that include a first additional value of the first coefficient of the first model and a second additional value of the second coefficient of the second model of reference application claim 10 and wherein the local minimum is determined using a gradient descent algorithm such that the individual models of the plurality of models cooperate during the optimization process of reference application claim 14.
Conclusion
No claims are allowed.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KETTIP KRIANGCHAIVECH whose telephone number is (571)272-1735. The examiner can normally be reached 8:30am-5:00pm EDT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Larry D. Riggs can be reached at (571) 270-3062. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.K./Examiner, Art Unit 1686
/LARRY D RIGGS II/Supervisory Patent Examiner, Art Unit 1686