DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in responsive to communication(s): original application filed on 06/30/2023, said application claims a priority filing date of 01/08/2021. Claims 1-23 are pending. Claims 1 and 22-23 are independent.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND PROGRAM FOR EXPLAINING THE PREDICTIONS.
Claim Objections
Claims 6-7 and 10-11 are objected to because of the following informalities:
In Claim 6, lines 8-10 of Page 2, "… selects a combination of at least two of the input variables as the reason for the prediction result" appears to be "… selects a combination of at least two of the plurality of input variables as the reason for the prediction result" according to Claims 1 and 2;
in Claim 7, lines 14-16 of Page 2, "… indicating strength of a relationship between the at least two input variables included in the combination and the prediction result …" appears to be "… indicating strength of a relationship between the at least two of the plurality of input variables included in the combination and the prediction result …" according to Claim 6;
in Claim 10, lines 2-5 of Page 3, "… an output variable indicating the prediction result as an objective variable for the plurality of the input variables, and selects the first explanatory variable as the reason from the input variables having …" appears to be "… an output variable indicating the prediction result as an objective variable for the plurality of input variables, and selects the first explanatory variable as the reason from the plurality of input variables having …";
in Claim 11, lines 13-15 of Page 3, "… selects the first explanatory variable as the reason from the input variables having a direct causal relationship with the objective variable" appears to be "… selects the first explanatory variable as the reason from the plurality of input variables having a direct causal relationship with the objective variable".
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: "control unit" in Claims 1-13, 16, and 18-21.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure (e.g., CPU 901 in ¶ [0202] that "The CPU 901 can form, for example, the control unit 130 illustrated in FIG. 1") described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-13, 15-16, 18, and 20-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 recites the limitation "… selects the first explanatory variable as a reason for the prediction result from among the plurality of input variables based on information as to whether the input variable and the prediction result are pseudo correlations in the prediction model generated by using machine learning, and outputs the information on the selected first explanatory variable" in Page 1, lines 14-21, which rendering the claim indefinite because "… outputting information on the selected first explanatory variable" is also recited in its based claim and (1) it is unclear whether the first instance of "information" recited here and "information" recited in its based claim are the same or different; (2) if they are different, which instance of "information" is referred by "the information" recited here. Clarification is required.
Claims 3-13 and 18 are rejected for fully incorporating the deficiency of their respective base claims.
Claim 8 recites the limitation "… the control unit … outputs the display screen" in in Page 2, lines 21-24, which rendering the claim indefinite because it is unclear how can "the control unit" outputs "the display screen"? and it appears somethings are missing between "outputs" and "the display screen"; i.e., it is unclear what information are outputted on "the display screen" by "the control unit". Clarification is required.
Claim 9 recites the limitation "… determining the combination of the input variables, and determines a combination of the input variables based on an operation …" in Page 2, lines 28-31, which rendering the claim indefinite because "… selects a combination of at least two of the input variables as the reason for the prediction result" is also recited in its based claim and (1) there is insufficient antecedent basis for the limitation "the combination of the input variables" in the claim; (2) it is unclear whether two instances of "combination of the input variables" are the same or not; and (3) it is unclear whether two instances of "combination of the input variables" recited here are the same as or different to "combination of at least two of the input variables" recited in its based claim. For examination, "… determining the combination of the at least two of the plurality of input variables, and determines the combination of the at least two of the plurality of input variables based on an operation …" is considered (see also Claim Objections to Claim 6) .
Claim 11 recites the limitation "... estimates a causal graph regarding a nearest node using the nearest node included in a hidden layer closest to the prediction model as an objective variable …" in Page 3, lines 10-12, which rendering the claim indefinite because it is unclear which layer is "a hidden layer closest to the prediction model" since "the prediction model" can have multiple layers (e.g., see FIG. 2 of the claimed invention). For examination purpose, "... estimates a causal graph regarding a nearest node using the nearest node included in a hidden layer closest to an output variable of the prediction model as an objective variable …" according to ¶¶ [0021] and [0035]-[0039] with FIG. 2 and ¶ [0043] with FIG. 3.
Claim 12 is rejected for fully incorporating the deficiency of their respective base claims.
Claim 12 recites the limitation "… selects the first explanatory variable serving as the positive reason on a basis of the causal graph related to the nearest node having a positive weight among the nearest nodes, and selects the first explanatory variable serving as the negative reason on a basis of the causal graph related to the nearest node having a negative weight among the nearest nodes" in Page 3, lines 18-25. There are insufficient antecedent basis for these limitations "the positive reason", "the nearest nodes" and "the negative reason" in the claim. For examination purposes, "… selects the first explanatory variable serving as positive reason on a basis of the causal graph related to the nearest node having a positive weight among nearest nodes in the hidden layer, and selects the first explanatory variable serving as negative reason on a basis of the causal graph related to the nearest node having a negative weight among the nearest nodes in the hidden layer" is considered.
Claim 15 recites the limitation "... the input variable includes information … acquired by a sensor" in Page 4, lines 6-8, which rendering the claim indefinite because "… the input variable includes information acquired by a sensor" is also recited in its based claim and it is unclear (1) whether the instance of "information" (acquired by "a sensor") recited here is the same as or different to the instance of "information" (acquired by "a sensor") recited in its based claim; and (2) whether the instance of "a sensor" recited here is the same as or different to the instance of "a sensor" recited in its based claim. Clarification is required.
Claim 16 is rejected for fully incorporating the deficiency of their respective base claims.
Claim 16 recites the limitation "… the input variable includes information acquired by a sensor … selects at least one of the information … acquired by the sensor as the first explanatory variable" in Page 4, lines 12-18, which rendering the claim indefinite because "… the input variable includes information acquired by a sensor" is also recited in its based Claim 14 and "... the input variable includes information … acquired by a sensor" is also recited in its based Claim 15, and (1) it is unclear whether the first instance of "information" (acquired by "a sensor") recited here is the same as or different to two instances of "information" (acquired by "a sensor") recited in its based claims; (2) if they are different, it is unclear which instance of "information" (acquired by "a sensor") is referred by "the information" (acquired by "the sensor") recited here; (3) it is unclear whether the first instance of "a sensor" recited here is the same as or different to the two instances of "a sensor" recited in its based claims; and (4) if they are different, it is unclear which instance of "sensor" is referred by "the sensor" recited here Clarification is required.
Claim 18 recites the limitation "… calculates an intervention effect for the first explanatory variable selected by the selection operation" in Page 4, lines 29-30, which rendering the claim indefinite because "… calculates an intervention effect in a case of intervening in the first explanatory variable selected as the reason" is also recited in its based claim, and it is unclear whether these two instances of "intervention effect" are the same or different. Clarification is required.
Claim 20 recites the limitation "... selects the second explanatory variable as a reason for the prediction result from among the plurality of input variables based on information as to whether the input variable and the prediction result are pseudo correlations in the prediction model generated by using machine learning, and outputs information on the selected second explanatory variable while distinguishing the information on the second explanatory variable from the information on the first explanatory variable… " in Page 5, lines 11-20, which rendering the claim indefinite because "… selecting, as a first explanatory variable, an input variable that affects a prediction result based on a causal model related to a causal relationship between a plurality of input variables and the prediction result in a prediction model using machine learning …" is also recited in its based Claim 1 and "… outputs information on the second explanatory variable while distinguishing the information on the second explanatory variable from the information on the first explanatory variable" is also recited in its based Claim 19, and (1) it is unclear whether "machine learning" recited here is the same as or different to "machine learning" recited in its based Claim 1; (2) it is unclear whether first two instances of "information" (regarding "second explanatory variable") recited here are the same as or different to two instances of "information" (regarding "second explanatory variable") recited in its based Claim 19; and (3) if they are different, which instance of "information" is referred by "the information" recited here. Clarification is required.
Claim 21 is rejected for fully incorporating the deficiency of their respective base claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 23 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Step 1: the claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the a "program" (i.e., software per se.) is claimed.
Claims 1-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more.
Independent Claims 1 and 22
Step 1: Claim 1 is an apparatus claim and Claim 22 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) recite(s) "selecting, as a first explanatory variable, an input variable that affects a prediction result based on a causal model related to a causal relationship between a plurality of input variables and the prediction result in a prediction model" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) recite(s) additional elements/limitations of "an information processing apparatus" (Claim 1), "a control unit" (Claim 1), "using machine learning", and "outputting information on the selected first explanatory variable" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because (a) the additional limitation/element of "using machine learning" is well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"); and (b) the additional limitation/element of "outputting information on the selected first explanatory variable" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 2
Step 1: Claim 2 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects the first explanatory variable as a reason for the prediction result from among the plurality of input variables based on information as to whether the input variable and the prediction result are pseudo correlations in the prediction model" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "the prediction model generated by using machine learning", and "outputs information on the selected first explanatory variable" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because (a) the additional limitation/element of "the prediction model generated by using machine learning" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"); and (b) the additional limitation/element of "outputs information on the selected first explanatory variable" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 3
Step 1: Claim 3 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects the input variable that is not in a pseudo correlation relationship with the prediction result as the first explanatory variable" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 4
Step 1: Claim 4 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects the input variable that is not conditionally independent of the prediction result as the first explanatory variable" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 5
Step 1: Claim 5 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "indicating strength of a relationship between the first explanatory variable selected as the reason and the prediction result" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "outputs strength information" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "outputs strength information" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 6
Step 1: Claim 6 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects a combination of at least two of the input variables as the reason for the prediction result" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 7
Step 1: Claim 7 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "indicating strength of a relationship between the at least two input variables included in the combination and the prediction result in association with information regarding the combination" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "outputs strength information" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "outputs strength information" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 8
Step 1: Claim 8 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "determines an order or a color corresponding to the first explanatory variable based on the strength information" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "a display screen" and "outputs the display screen" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "outputs the display screen" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 9
Step 1: Claim 9 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "determining the combination of the input variables" and "determines a combination of the input variables based on an operation" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "outputs an interface" and "an operation corresponding to the interface" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because (a) the additional limitation/element of "outputs an interface" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"); and (b) the additional limitation/element of "an operation corresponding to the interface" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 10
Step 1: Claim 10 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "estimates a causal graph with an output variable indicating the prediction result as an objective variable for the plurality of the input variables" and "selects the first explanatory variable as the reason from the input variables having a direct causal relationship with the objective variable" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 11
Step 1: Claim 11 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "for the plurality of input variables, estimates a causal graph regarding a nearest node using the nearest node included in a hidden layer closest to the prediction model as an objective variable" and "selects the first explanatory variable as the reason from the input variables having a direct causal relationship with the objective variable" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 12
Step 1: Claim 12 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects the first explanatory variable serving as the positive reason on a basis of the causal graph related to the nearest node having a positive weight among the nearest nodes" and "selects the first explanatory variable serving as the negative reason on a basis of the causal graph related to the nearest node having a negative weight among the nearest nodes" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 13
Step 1: Claim 13 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "calculates an intervention effect in a case of intervening in the first explanatory variable selected as the reason" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 14
Step 1: Claim 14 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) does/do not further recite(s) elements/limitations which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/algorithms/calculations.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "a sensor" and "the input variable includes information acquired by a sensor" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "the input variable includes information acquired by a sensor" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 15
Step 1: Claim 15 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) does/do not further recite(s) elements/limitations which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/algorithms/calculations.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "a device", "a sensor" and "the input variable includes information on an operating environment or an operating state of a device acquired by a sensor" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "the input variable includes information on an operating environment or an operating state of a device acquired by a sensor" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 16
Step 1: Claim 16 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects at least one of the information regarding temperature, humidity, voltage, current, electric power, or vibration acquired by the sensor as the first explanatory variable" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "a sensor" and "the input variable includes information regarding temperature, humidity, voltage, current, electric power, or vibration acquired by a sensor" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "the input variable includes information regarding temperature, humidity, voltage, current, electric power, or vibration acquired by a sensor" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 17
Step 1: Claim 17 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) does/do not further recite(s) elements/limitations which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/algorithms/calculations.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "the input variable includes information about an age or a history of a person" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "the input variable includes information about an age or a history of a person" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 18
Step 1: Claim 18 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "calculates an intervention effect for the first explanatory variable selected by the selection operation" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "acquires a selection operation for the output first explanatory variable" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "acquires a selection operation for the output first explanatory variable" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 19
Step 1: Claim 19 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects the input variable that does not affect the prediction result as a second explanatory variable based on the causal model" and "distinguishing the information on the second explanatory variable from the information on the first explanatory variable" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "outputs information on the second explanatory variable" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "outputs information on the second explanatory variable" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 20
Step 1: Claim 20 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects the second explanatory variable as a reason for the prediction result from among the plurality of input variables based on information as to whether the input variable and the prediction result are pseudo correlations in the prediction model" and "distinguishing the information on the second explanatory variable from the information on the first explanatory variable" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "the prediction model generated by using machine learning", and "outputs information on the selected second explanatory variable" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because (a) the additional limitation/element of "the prediction model generated by using machine learning" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"); and (b) the additional limitation/element of "outputs information on the selected second explanatory variable" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 21
Step 1: Claim 21 is an apparatus claim. The claim falls within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "selects the input variable having a pseudo correlation with the prediction result or the input variable that becomes conditionally independent as the second explanatory variable" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 17, 19, and 22-23 are rejected under 35 U.S.C. 103 as being unpatentable over DAISUKE et al. (JP 2020-95398 A, pub. date: 06/18/2020), hereinafter DAISUKE in view of Jorstad et al. (US 2020/0285969 A1, pub. date: 09/10/2020), hereinafter Jorstad.
Independent Claims 1 and 22-23
DAISUKE discloses an information processing apparatus (DAISUKE, ¶ [0020] with 100 in FIG. 1: the prediction basis presentation system 100 consists of at least one computer (not shown), and a terminal 150 is connected to the prediction basis presentation system 100) comprising a control unit (DAISUKE, ¶ [0021] with FIG.1: a computer has a processor) identifying (DAISUKE, ABSTRACT: calculates the degree of influence showing the influence of the values of the multiple explanatory variables on the prediction result and generates prediction basis data consisting of the degree of influence of the multiple explanatory variables; generates groups from aggregable explanatory variables identified by performing statistical analysis processing using the history of input data and the history of prediction basis data, calculates aggregated influence from the degree of influence of the multiple explanatory variables included in the group; ¶ [0001]: explaining the basis of predictions made using models generated by machine learning; ¶ [0002]: generating a model that mathematically represents the relationship between the target variable and explanatory variables using machine learning with training data, and then using that model to predict the value of the target variable from the value of an unknown explanatory variable; ¶ [0005]: calculating the degree of influence (importance) of each explanatory variable included in the data input to the model, which indicates the impact on the prediction result, as information that shows the basis for the prediction result; ¶ [0006]: when there are multiple explanatory variables related to a single feature (a factor in the prediction result), the multicollinearity problem causes the influence of that single feature on the prediction result to be output as the influence of multiple related explanatory variables; ¶ [0012]: presenting easily understandable basis for prediction results obtained by inputting non-independent explanatory variables into a model; ¶ [0023]: data is generated as prediction basis data, which consists of values that evaluate the magnitude of the influence of each explanatory variable on the prediction result, i.e., the degree of influence of each explanatory variable; ¶ [0024]: identifies explanatory variables to be aggregated by analyzing the relationships between explanatory variables, and aggregates the degree of influence of the identified explanatory variables; converts the prediction basis data into aggregated prediction basis data consisting of unaggregated and aggregated impact values; ¶ [0026]: the model is generated by a model generation unit that performs machine learning; ¶ [0034]: calculates the degree of influence of each explanatory variable on the prediction result and generates prediction basis data that includes the degrees of influence of multiple explanatory variables; ¶¶ [0036]-[0038]: calculates related indicators using the data to be predicted and the data on which the prediction is based; calculates the absolute value of the correlation coefficient calculated from the regression analysis as a related indicator; determines the explanatory variables to be aggregated based on the related indicator data; generates group data for the groups generated from the determined explanatory variables; generates aggregated prediction basis data by aggregating the influence of the explanatory variables to be aggregated based on the prediction basis data and group data; ¶¶ [0040]-[0044]: consider a model that performs classification into two classes (Class 0 and Class 1); the model will output a value indicating the probability of being in class 1 as the prediction result; denote the predicted result as Y, the baseline as Y0, and the degree of influence of the explanatory variable Xi on the predicted result as Yi, then according to Non-Patent Document 1, the degree of influence Yi is calculated to satisfy equation (1); the degree of influence Yi is a positive or negative real number, where positive values indicate the impact on predictions that will be classified as "Class 1," while negative values indicate the impact on predictions that will not be classified as "Class 1"; a larger absolute value of the influence indicates the magnitude of the explanatory variable's impact on the prediction; if the explanatory variables include those with correlations or other relationships, that is, if there are multiple explanatory variables related to a single factor, the influence of that factor on the prediction results is calculated as the influence of the multiple explanatory variables; analyzes the relationships between explanatory variables, identifies combinations of explanatory variables that are related, and aggregates the degree of influence based on those combinations of explanatory variables; this allows us to present predictive evidence that is easy for users to understand; ¶¶ [0045]-[0050] with FIGS. 2-3: the history information 200 stores one or more records consisting of an ID 201 and a value 202; each record corresponds to one data point to be predicted; value 202 is a group of fields that store the values of each of the multiple explanatory variables included in the data to be predicted; the history information 300 stores one or more records consisting of an ID 301 and an impact level 302; each record corresponds to one predictive data point; influence 302 is a group of fields that stores the influence of each explanatory variable on the prediction results; ¶¶ [0051]-[0055] with FIG. 4: the analysis information 400 stores one or more records consisting of a first explanatory variable 401, a second explanatory variable 402, a data-related indicator 403, an influence-related indicator 404, an input data-related indicator 405, and an input influence-related indicator 406; for each related indicator data point, one analysis piece of information 400 is generated; in the analysis information 400, there is one record for each pair of explanatory variables; the data-related indicator 403 shows the relationship between the values of the paired explanatory variables (i.e., the first explanatory variable 401 and the second explanatory variable 402); the influence-related indicator 404 shows the relationship between the influence levels of the paired explanatory variables; the input data-related indicator 405 and the input influence-related indicator 406 are set to values included in the operation data; ¶¶ [0056]-[0060] with FIGS. 5: the aggregated explanatory variable information 500 stores one or more records consisting of a group 501 and element explanatory variables 502; in the aggregate explanatory variable information 500, there is one record for each group; group 501 is the identification information of the group; the element explanatory variable 502 is identification information for the explanatory variables included in the group; ¶¶ [0062]-[0073] with FIGS. 1-3 and 5-6: the data receiving unit 101 receives the data to be predicted from the terminal 150 (step S601); when the prediction execution unit 103 receives data to be predicted, it obtains model information, performs model-based processing on the data to be predicted, and outputs the prediction result (step S602); when the prediction basis output unit 104 receives the data to be predicted, it calculates the degree of influence of the explanatory variables on the prediction result (step S603); the prediction basis output unit 104 acquires model information and calculates the degree of influence of each explanatory variable of the data to be predicted on the prediction result using mathematical methods; the result output unit 108 refers to the aggregate explanatory variable information 500 and selects a target record (step S604); the result output unit 108 aggregates the influence of the explanatory variables to be aggregated based on the target record (step S605); the result output unit 108 refers to the prediction basis data, obtains the influence of the explanatory variables corresponding to the element explanatory variables 502 of the target record, and sums up the obtained influence values; the result output unit 108 removes the field indicating the influence of the explanatory variable corresponding to the element explanatory variable 502 of the target record from the prediction basis data, and adds the group field to the prediction basis data; the result output unit 108 sets the total influence value for the group's field; by performing the same process for all groups, the predictive basis data is transformed into aggregated predictive basis data; the result output unit 108 determines whether processing has been completed for all records of the aggregated explanatory variable information 500 (step S606); if it is determined that processing is not complete for all records of the aggregated explanatory variable information 500, the result output unit 108 returns to step S604 and performs the same processing; ¶¶ [0083]-[0100] with FIGS. 1-5 and 8: the related indicator calculation unit 106 generates a list of combinations of explanatory variables (step S801); the related indicator calculation unit 106 selects a target pair from the list (step S802); the related indicator calculation unit 106 obtains the values of the explanatory variables that form the target pair from each record of the historical information 200 (step S803); the related indicator calculation unit 106 calculates data-related indicators by performing statistical analysis using the first group of temporary records (step S804); e.g., the related indicator calculation unit 106 calculates a correlation coefficient, which shows the correlation between the values of each explanatory variable, as a data-related indicator; the related indicator calculation unit 106 obtains the influence of the explanatory variables forming the target pair from each record of the historical information 300 (step S805); the related indicator calculation unit 106 calculates the influence-related indicator by performing a statistical analysis using the second set of temporary records (step S806); e.g., the related indicator calculation unit 106 calculates correlation coefficients, which show the correlation of the degree of influence of each explanatory variable, as influence-related indicators; the related indicator calculation unit 106 determines whether processing has been completed for all pairs registered in the list (step S807); if it is determined that processing is not complete for all pairs registered in the list, the related indicator calculation unit 106 returns to step S802 and performs the same processing; if it is determined that processing has been completed for all pairs registered in the list, the related indicator calculation unit 106 sends the registration record for each pair to the related indicator storage unit 113 (step S808); ¶¶ [0105]-[0106] with FIG. 10A: the field name in the analysis information manipulation section 1010 is provided with a sort button for rearranging records; the input data-related indicator fields and input influence-related indicator fields of the records in the analysis information manipulation field 1010 are controlled to accept operations from the user; the user sets a value in the field to control whether pairs with high correlation indices due to spurious correlation are aggregated, or whether pairs with low correlation indices are aggregated; ¶¶ [0139]-[0151] with FIGS. 1, 4-5, and 11: steps S1104 and S1105 are processes for determining whether or not there is a relationship between the values of the paired explanatory variables; if the value of data-related indicator 403 is greater than the first threshold, or if the value of input data related indicator 405 is greater than the second threshold, it is determined that a correlation exists between the values of the paired explanatory variables; steps S1106 and S1108 are processes for determining whether or not there is a relationship between the influence levels of the paired explanatory variables; if the value of the influence-related index 404 is greater than the third threshold, or if the value of the input influence-related index 406 is greater than the fourth threshold, it is determined that a relationship exists between the influences of the paired explanatory variables; if there is a correlation between the values of the paired explanatory variables, and there is also a correlation between the influence levels of the paired explanatory variables, the aggregation variable determination unit 107 determines that aggregation is possible because there is a correlation between the paired explanatory variables; if the result of step S1110 is YES, the aggregate variable determination unit 107 may refer to the aggregate explanatory variable information 500 and aggregate multiple pairs based on the transitive property; this allows us to generate groups consisting of three or more explanatory variables; when presenting information that shows the basis for predicting an event using an arbitrary model, it is possible to determine multiple explanatory variables that can be aggregated, generate groups from the multiple explanatory variables, and present the aggregated influence for each group; this makes it easier to understand the basis for predictions by comparing the degree of impact; determined whether or not there was a relationship between explanatory variables based on the comparison results of related indicators and thresholds; however, this is not the only way to determine whether or not there is a relationship between explanatory variables; e.g., the presence or absence of a relationship between explanatory variables may be determined based on the presence or absence of a function that shows the relationship between the values of the explanatory variables and the relationship between the influence of the explanatory variables; in this case, the analysis information 400 includes a field for a function instead of a field for related indicators; in the related indicator calculation process, the related indicator calculation unit 106 performs statistical analysis in step S804 to generate a function that shows the relationship between the values of the explanatory variables, and in step S806 to generate a function that shows the relationship between the degree of influence of the explanatory variables; in the aggregate variable determination process, the aggregate variable determination unit 107 registers groups of explanatory variables in the aggregate explanatory variable information 500 that have functions indicating the relationships between the values of the explanatory variables and functions indicating the relationships between the influences of the explanatory variables; in addition, during the aggregation variable determination process, the relationship between the values of the explanatory variables and the degree of influence of the explanatory variables may be determined based on the degree of the function), and
outputting information on the identified first explanatory variable (DAISUKE, ABSTRACT: a result output unit that outputs display information for displaying the prediction basis data; converts the prediction basis data into aggregated prediction basis data, and generates display information based on the aggregated prediction basis data; ¶ [0024]: generates display information to present the prediction results and aggregated prediction basis data to the user, and transmits the display information to the terminal 150; ¶¶ [0033]-[0034] with FIG. 1: the prediction execution unit 103 outputs a prediction result by performing model-based processing on the data to be predicted, and transmits the prediction result to the result output unit 108; the prediction basis output unit 104 transmits the prediction basis data to the result output unit 108; ¶ [0038]: generates display information based on the prediction results and aggregated prediction basis data, and transmits the display information to the terminal 150; e.g., display information is generated to show the prediction results and bar graphs representing the influence of the explanatory variables and the influence of the groups; ¶¶ [0074]-[0081] with FIGS. 1, 5-6, and 7A-B: if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the prediction basis information 700 in Figure 7A includes one or more records consisting of an explanatory variable 701, an influence level 702, and a value 703; each record corresponds to one explanatory variable or one group; the 700 records of the prediction basis information are sorted in descending order of their absolute impact; explanatory variable 701 is identification information for an explanatory variable or group: influence 702 is the influence of the explanatory variable or group; the value 703 is the value of the explanatory variable, or the value of the explanatory variable included in the group; the prediction basis information 700 in Figure 7B includes one or more records consisting of explanatory variables 711, influence 712, value 713, element explanatory variables 714, element value 715, and element influence 716; each record corresponds to one explanatory variable or one aggregate explanatory variable; the element explanatory variable 714 is identification information for the explanatory variables included in the group; element value 715 is the values of the explanatory variables included in the group; element Influence 716 is the influence of the explanatory variables included in the group; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results).
DAISUKE further discloses an information processing method described above and a program for causing a computer to perform operations described above (DAISUKE, ¶¶ [0153]-[0155]: each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, e.g., by designing them as integrated circuits, which can also be realized by software program code that implements the functions of the embodiments; a storage medium containing program code is provided to a computer, and the computer's processor reads the program code stored on the storage medium; the program code that implements the functions can be implemented in a wide range of programming or scripting languages; the program code for the software that implements the functions may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium).
DAISUKE fails to explicitly disclose selecting, as a first explanatory variable, an input variable that affects a prediction result, and outputting information on the selected first explanatory variable.
Jorstad teaches systems and methods for explaining results obtained from predictive processes and/or predictive models (Jorstad), wherein selecting, as a first explanatory variable, an input variable that affects a prediction result, and outputting information on the selected first explanatory variable (Jorstad, ABSTRACT: obtaining a plurality of values each corresponding to one of a plurality of variables; the plurality of variables include variables of interest; obtaining a prediction for the values from a model, determining metric(s) for each of the variables of interest, and determining one or more of the variables of interest to be one or more influential variables based on the metric(s) determined for each of the variables of interest; the variables include one or more non-influential variables that is/are different from the influential variable(s); the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); displaying in a graphical user interface or printing in a report an explanation identifying the influential variable(s) and/or a justification of the determination that the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); ¶¶ [0025]-[0057] with FIGS. 1 and 3-5: determine why the model 100 outputs a particular prediction; one or more of the input variables 102 may have a greater influence on the prediction 104 than the other input variables 102; the explanation procedure 300 identifies a number "i" of the most influential of the input variables 102 on the prediction 104; the model 100 may be a black box to the explanation procedure 300; the explanation procedure 300 may be used to provide explanations for on-line streaming data in addition to off-line batch data; the prediction surface is re-sampled for each input variable separately to evaluate the impact of changes to that input variable on the resulting prediction; in other words, all of the input variables 102 are held constant except one, which is sampled at different values; in first block 305, execute the model 100 on the original unmodified input record 106 to obtain the actual prediction 104; in block 310, identify one or more of the input variables 102 as being of interest; in block 315, select one of the input variables of interest; the explanation procedure 300 operates on a locality principle and modifies one of the input variables 102 at a time; in block 320, obtain sample values of the input variable selected in block 315 (e.g., the input variable 121); in block 325, execute the model 100 once for each of the sample values but uses the original value of each of the other input variables included in the input record 106; i.e., the value of the input variable selected in block 315 (e.g., the input variable 121) is changed but the values of all other input variables are left unchanged; in block 325, obtain sample predictions that are each associated with a different one of the sample values; in block 330, generates one or more metrics for the input variable selected in block 315 (e.g., the input variable 121) by comparing the sample predictions with the actual prediction 104; the metric(s) generated in block 330 may include one or more of the following: (a) a minimum ("Min") metric, which is the smallest predicted value expected from modifying the input variable; (b) a maximum ("Max") metric, which the largest predicted value expected from modifying the input variable; (c) a range, which is equal to a difference between the Max metric and the Min metric; (d) an upside metric, which equals the Max metric minus the Actual with values less than zero being truncated to zero and represents an amount of potential increase in predicted values expected by changing the input variable; (e) a downside metric, which equals the Actual minus the Min metric with values less than zero being truncated to zero and represents an amount of potential decrease in predicted values expected by changing the input variable; (f) an ExpectedUpside metric, which is equal to sum(probability(bin)*UpDifference), where the Up Difference equals (sampled(bin)-Actual) for all the bins where sampled(bin)>Actual and zero for all the bins where sampled(bin)≤Actual; and (g) an ExpectedDownside metric, which is equal to sum(probability(bin)*DownDifference) where the DownDifference equals (Actual-sampled(bin)) for all the bins where sampled(bin)<Actual and zero for all the bins where sampled(bin)≥Actual; the actual prediction 104 is referred to above as "Actual" above, the term "bin" identifies the sample value selected from one of the sample bins 322 and the term "sampled(bin)" is the sample prediction obtained for the sample value; the term "probability (bin)" is the prior probability associated with the sample value identified by the term "bin"; the ExpectedUpside and ExpectedDownside metrics use the prior probabilities to adjust the expected values, treating each of the input variables of interest as a discrete random variable; those sample values that are unlikely based on the prior distribution of an original dataset are penalized; in block 330, assign the metric(s) to or associates the metric(s) with the input variable selected in block 315 (e.g., the input variable 121); then, in decision block 335, determine whether it has evaluated all of the input variables of interest; the decision in decision block 335 is "YES" when the explanation computing device 302 has evaluated all of the input variables of interest; otherwise, the decision in decision block 335 is "NO"; when the decision in decision block 335 is "NO," return to block 315 and selects another one of the input variables of interest; on the other hand, when the decision in decision block 335 is "YES," the metric(s) for each of the input variables of interest has been collected and advances to block 340; in block 340, uses the metric(s) assigned to each of the input variables of interest in block 330 to identify the number "i" of the most influential input variables; e.g., identify the number "i" (e.g., three) of the input variables of interest having the largest ExpectedDownside metrics as being the most influential variables; at least a portion of those of the input variables of interest that are not identified as being most influential variables may be identified as or considered to be non-influential variables; in block 340, weight or rank the input variables of interest based on the metric(s); e.g., when the input variables of interest rank the input variables of interest based on the ExpectedDownside metric calculated for each of the input variables of interest are ranked, each input variable of interest appears only once in the ranking; independently of the characteristics and values of the input variable, assign a single rank to the input variable and include the input variable only once in the ranking; these ranks allow meaningful comparisons between continuous and categorical variables, with or without missing and special values; in optional block 345, identify one or more changes to the input variables of interest that would result in a more desirable prediction; i.e., identify one or more corrective actions that can be taken; in optional block 347, identify text descriptions 500 for each of the most influential input variables identified in block 340; the text descriptions 500 may include or be associated with reason codes; a reason code may indicate a negative condition or reason for rejection; in block 350, display a graphical user interface 352 including the explanation 360 to the user (e.g., a consumer, a loan applicant, and the like) on a display device; in optional block 347, the text descriptions 500 may be identified for each of the most influential input variables; ¶¶ [0076]-[0080] with FIGS. 1, 3-4, and 9-11: a useful property of explanation generation when applied to practical problems is that it can become unnecessary to compute sample values and corresponding sample predictions for all of the input variables; e.g., to satisfy certain regulatory requirements, it may be necessary to report only the top five most influential input variables in the explanation for each record; in block 310, identify one or more of the input variables 102 as being of interest; thus, in block 310, select only those of the input variables 102 having a value for the global measure of variable importance that exceeds a threshold value; alternatively, select only a predetermined number of the input variables 102 with the largest values for the global measure of variable importance; variable importance measures the impact of each input variable across the entire dataset and is often used to decide which input variables to include in a model during the model development process; explanations rank the input variables of interest based on their impact to individual predictions, which lead to decisions; explanations can be used to provide feedback to individual users or consumers during live processes; individual explanations can also be aggregated to form global measures of importance or to provide measures of importance for different partitions of the population; e.g., the data can be partitioned into different groupings of rejected populations, from those that are rejected most strongly to those that are rejected less strongly; these groupings can show systematic patterns as to what factors are causing these groups of individuals to be rejected; this information can be useful for the purpose of accountability in decision making as well as providing a greater understanding of model behavior).
DAISUKE and Jorstad are analogous art because they are from the same field of endeavor, systems and methods for explaining results obtained from predictive processes and/or predictive models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Jorstad to DAISUKE. Motivation for doing so would reduce computational complexity by computing explanations for only a fraction of the overall input variables (Jorstad, ¶ [0076]).
Claim 17
DAISUKE in view of Jorstad discloses all the elements as stated in Claim 1 and further discloses wherein the input variable includes information about an age or a history of a person (DAISUKE, ¶ [0003]: in loan assessments to predict the risk of default for loan applicants, a model is used in which the age and annual income of the loan applicant are used as explanatory variables, and the risk of default is used as the dependent variable).
Claim 19
DAISUKE in view of Jorstad discloses all the elements as stated in Claim 1 and further discloses wherein the control unit selects the input variable that does not affect the prediction result as a second explanatory variable based on the causal model, and outputs information on the second explanatory variable while distinguishing the information on the second explanatory variable from the information on the first explanatory variable (DAISUKE, ¶ [0106] with FIG. 10A: the user sets a value in the field to control whether pairs with high correlation indices due to spurious correlation are aggregated, or whether pairs with low correlation indices are aggregated; ¶¶ [0023]-[0024]: data is generated as prediction basis data, which consists of values that evaluate the magnitude of the influence of each explanatory variable on the prediction result, i.e., the degree of influence of each explanatory variable; identifies explanatory variables to be aggregated by analyzing the relationships between explanatory variables, and aggregates the degree of influence of the identified explanatory variables; converts the prediction basis data into aggregated prediction basis data consisting of unaggregated and aggregated impact values; generates display information to present the prediction results and aggregated prediction basis data to the user, and transmits the display information to the terminal 150; ¶ [0034]: calculates the degree of influence of each explanatory variable on the prediction result and generates prediction basis data that includes the degrees of influence of multiple explanatory variables; ¶¶ [0036]-[0038]: calculates related indicators using the data to be predicted and the data on which the prediction is based; calculates the absolute value of the correlation coefficient calculated from the regression analysis as a related indicator; determines the explanatory variables to be aggregated based on the related indicator data; generates group data for the groups generated from the determined explanatory variables; generates aggregated prediction basis data by aggregating the influence of the explanatory variables to be aggregated based on the prediction basis data and group data; generates display information based on the prediction results and aggregated prediction basis data, and transmits the display information to the terminal 150; e.g., display information is generated to show the prediction results and bar graphs representing the influence of the explanatory variables and the influence of the groups; ¶¶ [0040]-[0044]: consider a model that performs classification into two classes (Class 0 and Class 1); the model will output a value indicating the probability of being in class 1 as the prediction result; denote the predicted result as Y, the baseline as Y0, and the degree of influence of the explanatory variable Xi on the predicted result as Yi, then according to Non-Patent Document 1, the degree of influence Yi is calculated to satisfy equation (1); the degree of influence Yi is a positive or negative real number, where positive values indicate the impact on predictions that will be classified as "Class 1," while negative values indicate the impact on predictions that will not be classified as "Class 1"; a larger absolute value of the influence indicates the magnitude of the explanatory variable's impact on the prediction; if the explanatory variables include those with correlations or other relationships, that is, if there are multiple explanatory variables related to a single factor, the influence of that factor on the prediction results is calculated as the influence of the multiple explanatory variables; analyzes the relationships between explanatory variables, identifies combinations of explanatory variables that are related, and aggregates the degree of influence based on those combinations of explanatory variables; this allows us to present predictive evidence that is easy for users to understand; ¶¶ [0065]-[0081] with FIGS. 1, 5-6, and 7A-B: when the prediction basis output unit 104 receives the data to be predicted, it calculates the degree of influence of the explanatory variables on the prediction result (step S603); the prediction basis output unit 104 acquires model information and calculates the degree of influence of each explanatory variable of the data to be predicted on the prediction result using mathematical methods; the result output unit 108 refers to the aggregate explanatory variable information 500 and selects a target record (step S604); the result output unit 108 aggregates the influence of the explanatory variables to be aggregated based on the target record (step S605); the result output unit 108 refers to the prediction basis data, obtains the influence of the explanatory variables corresponding to the element explanatory variables 502 of the target record, and sums up the obtained influence values; the result output unit 108 removes the field indicating the influence of the explanatory variable corresponding to the element explanatory variable 502 of the target record from the prediction basis data, and adds the group field to the prediction basis data; the result output unit 108 sets the total influence value for the group's field; by performing the same process for all groups, the predictive basis data is transformed into aggregated predictive basis data; the result output unit 108 determines whether processing has been completed for all records of the aggregated explanatory variable information 500 (step S606); if it is determined that processing is not complete for all records of the aggregated explanatory variable information 500, the result output unit 108 returns to step S604 and performs the same processing; if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the 700 records of the prediction basis information are sorted in descending order of their absolute impact; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results) (Jorstad, ABSTRACT: obtaining a plurality of values each corresponding to one of a plurality of variables; the plurality of variables include variables of interest; obtaining a prediction for the values from a model, determining metric(s) for each of the variables of interest, and determining one or more of the variables of interest to be one or more influential variables based on the metric(s) determined for each of the variables of interest; the variables include one or more non-influential variables that is/are different from the influential variable(s); the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); displaying in a graphical user interface or printing in a report an explanation identifying the influential variable(s) and/or a justification of the determination that the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); ¶¶ [0025]-[0057] with FIGS. 1 and 3-5: determine why the model 100 outputs a particular prediction; one or more of the input variables 102 may have a greater influence on the prediction 104 than the other input variables 102; the explanation procedure 300 identifies a number "i" of the most influential of the input variables 102 on the prediction 104; in block 310, identify one or more of the input variables 102 as being of interest; in block 315, select one of the input variables of interest; in block 330, generates one or more metrics for the input variable selected in block 315 (e.g., the input variable 121) by comparing the sample predictions with the actual prediction 104; in block 340, uses the metric(s) assigned to each of the input variables of interest in block 330 to identify the number "i" of the most influential input variables; e.g., identify the number "i" (e.g., three) of the input variables of interest having the largest ExpectedDownside metrics as being the most influential variables; at least a portion of those of the input variables of interest that are not identified as being most influential variables may be identified as or considered to be non-influential variables; in block 340, weight or rank the input variables of interest based on the metric(s); e.g., when the input variables of interest rank the input variables of interest based on the ExpectedDownside metric calculated for each of the input variables of interest are ranked, each input variable of interest appears only once in the ranking; independently of the characteristics and values of the input variable, assign a single rank to the input variable and include the input variable only once in the ranking; these ranks allow meaningful comparisons between continuous and categorical variables, with or without missing and special values; in block 350, display a graphical user interface 352 including the explanation 360 to the user (e.g., a consumer, a loan applicant, and the like) on a display device; in optional block 347, the text descriptions 500 may be identified for each of the most influential input variables; ¶¶ [0076] and [0078] with FIGS. 1 and 3: a useful property of explanation generation when applied to practical problems is that it can become unnecessary to compute sample values and corresponding sample predictions for all of the input variables; e.g., to satisfy certain regulatory requirements, it may be necessary to report only the top five most influential input variables in the explanation for each record; in block 310, identify one or more of the input variables 102 as being of interest; thus, in block 310, select only those of the input variables 102 having a value for the global measure of variable importance that exceeds a threshold value; alternatively, select only a predetermined number of the input variables 102 with the largest values for the global measure of variable importance; variable importance measures the impact of each input variable across the entire dataset and is often used to decide which input variables to include in a model during the model development process; explanations rank the input variables of interest based on their impact to individual predictions, which lead to decisions; explanations can be used to provide feedback to individual users or consumers during live processes; individual explanations can also be aggregated to form global measures of importance or to provide measures of importance for different partitions of the population; e.g., the data can be partitioned into different groupings of rejected populations, from those that are rejected most strongly to those that are rejected less strongly; these groupings can show systematic patterns as to what factors are causing these groups of individuals to be rejected; this information can be useful for the purpose of accountability in decision making as well as providing a greater understanding of model behavior).
Claims 2-10 and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over DAISUKE in view of Jorstad as applied to Claim 1 and 19 respectively above, and further in view of Wei et al. (US 2022/0004910 A1, priority date on 07/01/2020), hereinafter Wei.
Claim 2
DAISUKE in view of Jorstad discloses all the elements as stated in Claim 1 and further discloses wherein the control unit selects the first explanatory variable as a reason for the prediction result from among the plurality of input variables based on information as to whether the input variable and the prediction result are (DAISUKE, ¶ [0106] with FIG. 10A: the user sets a value in the field to control whether pairs with high correlation indices due to spurious correlation are aggregated, or whether pairs with low correlation indices are aggregated; ¶¶ [0023]-[0024]: data is generated as prediction basis data, which consists of values that evaluate the magnitude of the influence of each explanatory variable on the prediction result, i.e., the degree of influence of each explanatory variable; identifies explanatory variables to be aggregated by analyzing the relationships between explanatory variables, and aggregates the degree of influence of the identified explanatory variables; converts the prediction basis data into aggregated prediction basis data consisting of unaggregated and aggregated impact values; generates display information to present the prediction results and aggregated prediction basis data to the user, and transmits the display information to the terminal 150; ¶ [0034]: calculates the degree of influence of each explanatory variable on the prediction result and generates prediction basis data that includes the degrees of influence of multiple explanatory variables; ¶¶ [0036]-[0038]: calculates related indicators using the data to be predicted and the data on which the prediction is based; calculates the absolute value of the correlation coefficient calculated from the regression analysis as a related indicator; determines the explanatory variables to be aggregated based on the related indicator data; generates group data for the groups generated from the determined explanatory variables; generates aggregated prediction basis data by aggregating the influence of the explanatory variables to be aggregated based on the prediction basis data and group data; generates display information based on the prediction results and aggregated prediction basis data, and transmits the display information to the terminal 150; e.g., display information is generated to show the prediction results and bar graphs representing the influence of the explanatory variables and the influence of the groups; ¶¶ [0040]-[0044]: consider a model that performs classification into two classes (Class 0 and Class 1); the model will output a value indicating the probability of being in class 1 as the prediction result; denote the predicted result as Y, the baseline as Y0, and the degree of influence of the explanatory variable Xi on the predicted result as Yi, then according to Non-Patent Document 1, the degree of influence Yi is calculated to satisfy equation (1); the degree of influence Yi is a positive or negative real number, where positive values indicate the impact on predictions that will be classified as "Class 1," while negative values indicate the impact on predictions that will not be classified as "Class 1"; a larger absolute value of the influence indicates the magnitude of the explanatory variable's impact on the prediction; if the explanatory variables include those with correlations or other relationships, that is, if there are multiple explanatory variables related to a single factor, the influence of that factor on the prediction results is calculated as the influence of the multiple explanatory variables; analyzes the relationships between explanatory variables, identifies combinations of explanatory variables that are related, and aggregates the degree of influence based on those combinations of explanatory variables; this allows us to present predictive evidence that is easy for users to understand; ¶¶ [0065]-[0081] with FIGS. 1, 5-6, and 7A-B: when the prediction basis output unit 104 receives the data to be predicted, it calculates the degree of influence of the explanatory variables on the prediction result (step S603); the prediction basis output unit 104 acquires model information and calculates the degree of influence of each explanatory variable of the data to be predicted on the prediction result using mathematical methods; the result output unit 108 refers to the aggregate explanatory variable information 500 and selects a target record (step S604); the result output unit 108 aggregates the influence of the explanatory variables to be aggregated based on the target record (step S605); the result output unit 108 refers to the prediction basis data, obtains the influence of the explanatory variables corresponding to the element explanatory variables 502 of the target record, and sums up the obtained influence values; the result output unit 108 removes the field indicating the influence of the explanatory variable corresponding to the element explanatory variable 502 of the target record from the prediction basis data, and adds the group field to the prediction basis data; the result output unit 108 sets the total influence value for the group's field; by performing the same process for all groups, the predictive basis data is transformed into aggregated predictive basis data; the result output unit 108 determines whether processing has been completed for all records of the aggregated explanatory variable information 500 (step S606); if it is determined that processing is not complete for all records of the aggregated explanatory variable information 500, the result output unit 108 returns to step S604 and performs the same processing; if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the 700 records of the prediction basis information are sorted in descending order of their absolute impact; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results) (Jorstad, ABSTRACT: obtaining a plurality of values each corresponding to one of a plurality of variables; the plurality of variables include variables of interest; obtaining a prediction for the values from a model, determining metric(s) for each of the variables of interest, and determining one or more of the variables of interest to be one or more influential variables based on the metric(s) determined for each of the variables of interest; the variables include one or more non-influential variables that is/are different from the influential variable(s); the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); displaying in a graphical user interface or printing in a report an explanation identifying the influential variable(s) and/or a justification of the determination that the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); ¶¶ [0025]-[0057] with FIGS. 1 and 3-5: determine why the model 100 outputs a particular prediction; one or more of the input variables 102 may have a greater influence on the prediction 104 than the other input variables 102; the explanation procedure 300 identifies a number "i" of the most influential of the input variables 102 on the prediction 104; in block 310, identify one or more of the input variables 102 as being of interest; in block 315, select one of the input variables of interest; in block 330, generates one or more metrics for the input variable selected in block 315 (e.g., the input variable 121) by comparing the sample predictions with the actual prediction 104; in block 340, uses the metric(s) assigned to each of the input variables of interest in block 330 to identify the number "i" of the most influential input variables; e.g., identify the number "i" (e.g., three) of the input variables of interest having the largest ExpectedDownside metrics as being the most influential variables; at least a portion of those of the input variables of interest that are not identified as being most influential variables may be identified as or considered to be non-influential variables; in block 340, weight or rank the input variables of interest based on the metric(s); e.g., when the input variables of interest rank the input variables of interest based on the ExpectedDownside metric calculated for each of the input variables of interest are ranked, each input variable of interest appears only once in the ranking; independently of the characteristics and values of the input variable, assign a single rank to the input variable and include the input variable only once in the ranking; these ranks allow meaningful comparisons between continuous and categorical variables, with or without missing and special values; in block 350, display a graphical user interface 352 including the explanation 360 to the user (e.g., a consumer, a loan applicant, and the like) on a display device; in optional block 347, the text descriptions 500 may be identified for each of the most influential input variables; ¶¶ [0076] and [0078] with FIGS. 1 and 3: a useful property of explanation generation when applied to practical problems is that it can become unnecessary to compute sample values and corresponding sample predictions for all of the input variables; e.g., to satisfy certain regulatory requirements, it may be necessary to report only the top five most influential input variables in the explanation for each record; in block 310, identify one or more of the input variables 102 as being of interest; thus, in block 310, select only those of the input variables 102 having a value for the global measure of variable importance that exceeds a threshold value; alternatively, select only a predetermined number of the input variables 102 with the largest values for the global measure of variable importance; variable importance measures the impact of each input variable across the entire dataset and is often used to decide which input variables to include in a model during the model development process; explanations rank the input variables of interest based on their impact to individual predictions, which lead to decisions; explanations can be used to provide feedback to individual users or consumers during live processes; individual explanations can also be aggregated to form global measures of importance or to provide measures of importance for different partitions of the population; e.g., the data can be partitioned into different groupings of rejected populations, from those that are rejected most strongly to those that are rejected less strongly; these groupings can show systematic patterns as to what factors are causing these groups of individuals to be rejected; this information can be useful for the purpose of accountability in decision making as well as providing a greater understanding of model behavior).
DAISUKE in view of Jorstad fails to explicitly disclose wherein selects the first explanatory variable based on whether the input variable and the prediction result are pseudo correlations.
Wei teaches a system and a method relating to determining causality (Wei, ¶ [0001]), wherein selects the first explanatory variable based on whether the input variable and the prediction result are pseudo correlations (Wei, ¶¶ [0020]-[0023] with FIG. 1: with traditional methods for causality determining, by setting Markov conditions and loyalty, a method based on conditional independence may identify the causal skeleton from the joint distribution, e.g., through a computer usage statistical test (conditional independent test), and direct edges to Markov equivalence classes through a series of rules (e.g., identifying v-shaped structures or colliders, avoiding loops, etc.); on the contrary, the causal mechanism and data distribution are described through a specific model category (an identifiable functional model or structural equation model (SEM)); if the data generating process belongs to such a model category, a complete causal diagram may be identified; constraint-based methods and score-based methods have been proposed for recovering causal structures from mixed data; propose a model for using mixed data types of data to determine causality among variables; the model may take variables and their set of parent variables as input, determine the causal order between variables and optionally determine an association, and further determine the causal relationship between variables, and then output the determined causality in the form of a directed acyclic graph; establish and use a functional model for mixed types of variables to determine the causality 130 through the observed data 110; ¶¶ [0026]-[0038] with FIG. 2: at block 202, obtains a group of variables; at block 204, obtains a causal model; the group of variables obtained at block 202 and information related to the group of variables, e.g., type of variables, other subset of variables and the number of variables, may be used as input of the causal model; the causal model obtained at block 204 is a mixed non-linear causal model, wherein linearity means that uniformity and superposition need to be satisfied; the causal model obtained at block 204 may determine, from the group of variables and information related to the group of variables obtained at block 202, possible causality among these variables, take a further operation to screen the determined causality and finally obtain accurate causality; with respect to each variable in the group of variables obtained at block 202, determine a set of parent variables of the variable and determine the causality among variables based on the type of the variable and the set of parent variables of the variable; the set of parent variables of a variable is a set of variables on which a value of the variable relies, i.e., the variable has causality with a variable in the set of parent variables of the variable; use the causal model obtained at block 204 to determine a causal sequence between variables in the group of variables obtained at block 202, and determine the causality based on the determined causal sequence; only when there is a causal sequence between two variables, the two variables might have causality; use the causal model obtained at block 204 to determine the possible causal sequence between variables by a method like greedy search; first use the causal model obtained at block 204 to obtain an initial causal sequence between variables in the group of variables obtained at block 202; then, determine fitness of the initial causal sequence, wherein the fitness indicates a probability that the initial causal sequence correctly represents the causal sequence between variables; finally, determine the causal sequence between variables based on the fitness and the initial causal sequence; first generate a parent relationship graph of each variable in the group of variables based on the determined set of parent variables of each variable for variables in the group of variables obtained at block 202; then, determine the causal sequence between variables by using, e.g., a graph theory method based on the parent relationship graphs; use the causal model obtained at block 204 to further determine association between variables by a method such as greedy search, and determine the causality based on the determined causal sequence and association; first use the causal model obtained at block 204 to determine initial causality among variables in the group of variables obtained at block 202 based on the determined causal sequence between variables; then, conduct a conditional independence test on the initial causality; finally, determine the causality among variables based on a result of the conditional independence test and the initial causality; additionally, also first determine association between variables, and determine the initial causality based on the determined causal sequence between variables and the determined association between variables; since both the causal sequence and the association are used, the initial causality determined at this point will become more accurate; first obtain causal information about the group of variables, the causal information indicating partial causality among a part of variables in the group of variables and comprising expert knowledge integration; then, use the causal model obtained at block 204 to determine the causality among variables in the group of variables based on the determined causal sequence between variables and the causal information; it should be understood that the causal information is related to the group of variables as the observed data 110 and thus may be obtained in the operation shown at any of blocks 202, 204 and 206; additionally, in the operation, also first determine association between variables, and determine the causality among variables in the group of variables based on the determined causal sequence, association and causal information between variables, thereby determining the causality more accurately; determine the causality by at least one of a constraint-based solution and a search-based solution; typical constraint-based technical solutions mainly comprise a PC (PeterClark) algorithm and an inductive causation algorithm, etc., which may comprise an undirected graph learning stage and a direction learning stage; search-based solutions comprise, e.g., a greedy equivalence search (GES) solution; the determined causality among variables in the group of variables may take the form of a directed acyclic graph, the directed acyclic graph comprising nodes and edges, a node representing a variable in the group of variables, and an edge representing causality among variables; ¶¶ [0042]-[0078] with FIG. 3: propose a structural equation model for mixed data types, the model allowing the causal mechanism to be non-linear and thus supporting a wider range of practical applications; the causal structure may be identified from a data distribution that follows the model, and the identified causal structure may be displayed through a directed acyclic graph; a maximum likelihood estimator is further proposed, the maximum likelihood estimator being used to select the causal sequence between variables rather than the causal structure, and results of the maximum likelihood estimator having sequential consistency; the sequential space is much smaller than the directed acyclic graph space and is easy to search; moreover, if the sequence between variables is known, the causal structure learning may be attributed to variable selection; this may be solved by sparse regression or (conditional) independence test, so that the causal structure learning may be performed with less computational overhead; further proposes an efficient sequential search method that benefits from a novel sequential space cutting method; with a maximum likelihood estimator, the method constructs a factor optimization problem, in which the causal sequence may be recovered using greedy search; to further accelerate sequential search, a graph lasso equipped with a kernel alignment method is first used to learn the sparse skeleton between variables, and the skeleton is projected into a series of topological ordering constraints to reduce the search space; with the method, the causality among variables may be accurately determined; in the initial modeling stage 310, a mixed non-linear causal model will be built, the model describing a non-linear relationship between mixed discrete variables and continuous variables, and the identifiability of the model being proved with further content; the mixed non-linear causal model may be defined through Equation (1); after the initial modeling stage 310, the flow proceeds to a causal sequence determining stage 320; a real causal structure may be identified from the joint distribution that follows the mixed non-linear causal model; the structural learning problem with directed acyclic graph constraints may be converted into the problem of learning the best sequence between variables, which seems easier because the sequential space is much smaller than the directed acyclic graph space; once the sequence is determined, acyclic constraints may be enforced by constraining the parent of a variable to be a subset of a variable before the variable; causal structure learning may be attributed to variable selection, which may be solved by sparse regression or (conditional) independence test; the real causal sequence may be identified from the joint distribution that follows the mixed non-linear causal model, and then the fitting of the sequence of mixed binary variables and continuous variables may be evaluated by the Mixed Nonlinear Information Criterion (MNIC), wherein MNIC scores are sequentially consistent; the operation in the causal sequence determining stage 320 may be performed based on the MNIC estimator, so that possible causal sequences between variables may be determined for model inputs comprising different types of variables; an additional sequence determining method 330 may be applied to the causal sequence determining stage 320 to achieve the effect of accelerating causality inference, the additional sequence determining method 330 comprising spatial clipping based on variable group sequence constraints; after the causal sequence determining stage 320, the flow proceeds to a causal structure learning stage 350, and then the causality 130 may be generated as output; a three-stage algorithm may be used to estimate the causal structure of observed data, i.e., to perform the operations in the causal sequence determining stage 320 and the causal structure learning stage 350; first, a graph lasso equipped with a kernel alignment method is used to learn the sparse skeleton between variables, and then the skeleton is projected to a series of topological ordering constraints so as to reduce the search space; next, under sequence constraints, greedy search is used in feasible space to estimate E in Equation (4); finally, a kernel-based conditional independence (KCI) test is used to trim edges so as to recover
G
full,min from
G
full; the whole algorithm is briefed in Algorithm 1; Stage 1: Generate Topological Ordering Constraints: (a) use Equations (9) and (10) to build a precision matrix Θ; (b) extract M SCC (i.e., Strong Connection Components) from Θ; (c) assign a random group sequence, and build a sequential constraint set C; Stage 2: Estimate the Causal Sequence (Corresponding to the Causal Sequence Determining Stage 320); Stage 3: Remove Extra Edges (Corresponding to the Causal Structure Learning Stage 350); in stage 3 of Algorithm 1, not only a method based on independence judgment may be used to remove edges, but also a feature selecting method may be used; the causal sequence determining stage 320 may comprise search spatial cutting: kernel alignment may be used to measure the similarity between two kernel functions, and also may be used to generate a pseudo-correlation matrix between random variables; Equation (9) is used to generate the pseudo-correlation matrix A on the observed data X=(X1, …, Xn), wherein each element A(i,j) is kernel alignment between Xi and Xj; the RBF kernel is used for continuous variables, and the delta kernel is used for binary variables; then, A is introduced to the graph lasso so as to learn the precision matrix Θ shown in Equation (10), wherein Θij=0 represents that there is no direct edge between Xi and Xj; Strong Connection Components (SCC) are generated from Θ; since no edge connects different SCCs, topological sequences between SCCs may be assigned at random; these sequential constraints will be used to reduce the search space in the second stage; the causal structure learning stage 350 may comprise trimming: using a conditional independence test to trim pseudo-edges from the super directed acyclic graph; In the conditional independence test, there exist some hyper-parameters: the kernel width and regularization parameters used to construct the kernel matrix; for an unconditional independence test, since the continuous variable has been regularized to unit variance, the median of the paired distances of these points is used as the kernel width; for a conditional independence test, when the conditional set is small (i.e., s2), the median of the paired distances of these points is used as the kernel width; for the regularization parameter, an empirical value (10-3) is used, which shows good effect; when the conditional set is large, the extended multi-output Gaussian process regression is used to learn hyper-parameters by maximizing the total marginal likelihood).
DAISUKE in view of Jorstad, and Wei are analogous art because they are from the same field of endeavor, a system and a method relating to determining causality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Wei to DAISUKE in view of Jorstad. Motivation for doing so would determine causality accurately, efficiently, and effectively.
Claim 3
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 2 and further discloses wherein the control unit selects the input variable that is not in a pseudo correlation relationship with the prediction result as the first explanatory variable (Wei, ¶¶ [0042]-[0078] with FIG. 3: propose a structural equation model for mixed data types, the model allowing the causal mechanism to be non-linear and thus supporting a wider range of practical applications; the causal structure may be identified from a data distribution that follows the model, and the identified causal structure may be displayed through a directed acyclic graph; a maximum likelihood estimator is further proposed, the maximum likelihood estimator being used to select the causal sequence between variables rather than the causal structure, and results of the maximum likelihood estimator having sequential consistency; the sequential space is much smaller than the directed acyclic graph space and is easy to search; moreover, if the sequence between variables is known, the causal structure learning may be attributed to variable selection; this may be solved by sparse regression or (conditional) independence test, so that the causal structure learning may be performed with less computational overhead; further proposes an efficient sequential search method that benefits from a novel sequential space cutting method; with a maximum likelihood estimator, the method constructs a factor optimization problem, in which the causal sequence may be recovered using greedy search; to further accelerate sequential search, a graph lasso equipped with a kernel alignment method is first used to learn the sparse skeleton between variables, and the skeleton is projected into a series of topological ordering constraints to reduce the search space; with the method, the causality among variables may be accurately determined; in the initial modeling stage 310, a mixed non-linear causal model will be built, the model describing a non-linear relationship between mixed discrete variables and continuous variables, and the identifiability of the model being proved with further content; the mixed non-linear causal model may be defined through Equation (1); after the initial modeling stage 310, the flow proceeds to a causal sequence determining stage 320; a real causal structure may be identified from the joint distribution that follows the mixed non-linear causal model; the structural learning problem with directed acyclic graph constraints may be converted into the problem of learning the best sequence between variables, which seems easier because the sequential space is much smaller than the directed acyclic graph space; once the sequence is determined, acyclic constraints may be enforced by constraining the parent of a variable to be a subset of a variable before the variable; causal structure learning may be attributed to variable selection, which may be solved by sparse regression or (conditional) independence test; the real causal sequence may be identified from the joint distribution that follows the mixed non-linear causal model, and then the fitting of the sequence of mixed binary variables and continuous variables may be evaluated by the Mixed Nonlinear Information Criterion (MNIC), wherein MNIC scores are sequentially consistent; the operation in the causal sequence determining stage 320 may be performed based on the MNIC estimator, so that possible causal sequences between variables may be determined for model inputs comprising different types of variables; an additional sequence determining method 330 may be applied to the causal sequence determining stage 320 to achieve the effect of accelerating causality inference, the additional sequence determining method 330 comprising spatial clipping based on variable group sequence constraints; after the causal sequence determining stage 320, the flow proceeds to a causal structure learning stage 350, and then the causality 130 may be generated as output; a three-stage algorithm may be used to estimate the causal structure of observed data, i.e., to perform the operations in the causal sequence determining stage 320 and the causal structure learning stage 350; first, a graph lasso equipped with a kernel alignment method is used to learn the sparse skeleton between variables, and then the skeleton is projected to a series of topological ordering constraints so as to reduce the search space; next, under sequence constraints, greedy search is used in feasible space to estimate E in Equation (4); finally, a kernel-based conditional independence (KCI) test is used to trim edges so as to recover
G
full,min from
G
full; the whole algorithm is briefed in Algorithm 1; Stage 1: Generate Topological Ordering Constraints: (a) use Equations (9) and (10) to build a precision matrix Θ; (b) extract M SCC (i.e., Strong Connection Components) from Θ; (c) assign a random group sequence, and build a sequential constraint set C; Stage 2: Estimate the Causal Sequence (Corresponding to the Causal Sequence Determining Stage 320); Stage 3: Remove Extra Edges (Corresponding to the Causal Structure Learning Stage 350); in stage 3 of Algorithm 1, not only a method based on independence judgment may be used to remove edges, but also a feature selecting method may be used; the causal sequence determining stage 320 may comprise search spatial cutting: kernel alignment may be used to measure the similarity between two kernel functions, and also may be used to generate a pseudo-correlation matrix between random variables; Equation (9) is used to generate the pseudo-correlation matrix A on the observed data X=(X1, …, Xn), wherein each element A(i,j) is kernel alignment between Xi and Xj; the RBF kernel is used for continuous variables, and the delta kernel is used for binary variables; then, A is introduced to the graph lasso so as to learn the precision matrix Θ shown in Equation (10), wherein Θij=0 represents that there is no direct edge between Xi and Xj; Strong Connection Components (SCC) are generated from Θ; since no edge connects different SCCs, topological sequences between SCCs may be assigned at random; these sequential constraints will be used to reduce the search space in the second stage; the causal structure learning stage 350 may comprise trimming: using a conditional independence test to trim pseudo-edges from the super directed acyclic graph; In the conditional independence test, there exist some hyper-parameters: the kernel width and regularization parameters used to construct the kernel matrix; for an unconditional independence test, since the continuous variable has been regularized to unit variance, the median of the paired distances of these points is used as the kernel width; for a conditional independence test, when the conditional set is small (i.e., s2), the median of the paired distances of these points is used as the kernel width; for the regularization parameter, an empirical value (10-3) is used, which shows good effect; when the conditional set is large, the extended multi-output Gaussian process regression is used to learn hyper-parameters by maximizing the total marginal likelihood).
Claim 4
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 2 and further discloses wherein the control unit selects the input variable that is not conditionally independent of the prediction result as the first explanatory variable (Jorstad, ABSTRACT: obtaining a plurality of values each corresponding to one of a plurality of variables; the plurality of variables include variables of interest; obtaining a prediction for the values from a model, determining metric(s) for each of the variables of interest, and determining one or more of the variables of interest to be one or more influential variables based on the metric(s) determined for each of the variables of interest; the variables include one or more non-influential variables that is/are different from the influential variable(s); the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); displaying in a graphical user interface or printing in a report an explanation identifying the influential variable(s) and/or a justification of the determination that the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); ¶¶ [0025]-[0057] with FIGS. 1 and 3-5: determine why the model 100 outputs a particular prediction; one or more of the input variables 102 may have a greater influence on the prediction 104 than the other input variables 102; the explanation procedure 300 identifies a number "i" of the most influential of the input variables 102 on the prediction 104; in block 310, identify one or more of the input variables 102 as being of interest; in block 315, select one of the input variables of interest; in block 330, generates one or more metrics for the input variable selected in block 315 (e.g., the input variable 121) by comparing the sample predictions with the actual prediction 104; in block 340, uses the metric(s) assigned to each of the input variables of interest in block 330 to identify the number "i" of the most influential input variables; e.g., identify the number "i" (e.g., three) of the input variables of interest having the largest ExpectedDownside metrics as being the most influential variables; at least a portion of those of the input variables of interest that are not identified as being most influential variables may be identified as or considered to be non-influential variables; in block 340, weight or rank the input variables of interest based on the metric(s); e.g., when the input variables of interest rank the input variables of interest based on the ExpectedDownside metric calculated for each of the input variables of interest are ranked, each input variable of interest appears only once in the ranking; independently of the characteristics and values of the input variable, assign a single rank to the input variable and include the input variable only once in the ranking; these ranks allow meaningful comparisons between continuous and categorical variables, with or without missing and special values; in block 350, display a graphical user interface 352 including the explanation 360 to the user (e.g., a consumer, a loan applicant, and the like) on a display device; in optional block 347, the text descriptions 500 may be identified for each of the most influential input variables; ¶¶ [0076] and [0078] with FIGS. 1 and 3: a useful property of explanation generation when applied to practical problems is that it can become unnecessary to compute sample values and corresponding sample predictions for all of the input variables; e.g., to satisfy certain regulatory requirements, it may be necessary to report only the top five most influential input variables in the explanation for each record; in block 310, identify one or more of the input variables 102 as being of interest; thus, in block 310, select only those of the input variables 102 having a value for the global measure of variable importance that exceeds a threshold value; alternatively, select only a predetermined number of the input variables 102 with the largest values for the global measure of variable importance; variable importance measures the impact of each input variable across the entire dataset and is often used to decide which input variables to include in a model during the model development process; explanations rank the input variables of interest based on their impact to individual predictions, which lead to decisions; explanations can be used to provide feedback to individual users or consumers during live processes; individual explanations can also be aggregated to form global measures of importance or to provide measures of importance for different partitions of the population; e.g., the data can be partitioned into different groupings of rejected populations, from those that are rejected most strongly to those that are rejected less strongly; these groupings can show systematic patterns as to what factors are causing these groups of individuals to be rejected; this information can be useful for the purpose of accountability in decision making as well as providing a greater understanding of model behavior) (Wei, ¶¶ [0020]-[0023] with FIG. 1: with traditional methods for causality determining, by setting Markov conditions and loyalty, a method based on conditional independence may identify the causal skeleton from the joint distribution, e.g., through a computer usage statistical test (conditional independent test), and direct edges to Markov equivalence classes through a series of rules (e.g., identifying v-shaped structures or colliders, avoiding loops, etc.); on the contrary, the causal mechanism and data distribution are described through a specific model category (an identifiable functional model or structural equation model (SEM)); if the data generating process belongs to such a model category, a complete causal diagram may be identified; constraint-based methods and score-based methods have been proposed for recovering causal structures from mixed data; propose a model for using mixed data types of data to determine causality among variables; the model may take variables and their set of parent variables as input, determine the causal order between variables and optionally determine an association, and further determine the causal relationship between variables, and then output the determined causality in the form of a directed acyclic graph; establish and use a functional model for mixed types of variables to determine the causality 130 through the observed data 110; ¶¶ [0026]-[0038] with FIG. 2: at block 202, obtains a group of variables; at block 204, obtains a causal model; the group of variables obtained at block 202 and information related to the group of variables, e.g., type of variables, other subset of variables and the number of variables, may be used as input of the causal model; the causal model obtained at block 204 is a mixed non-linear causal model, wherein linearity means that uniformity and superposition need to be satisfied; the causal model obtained at block 204 may determine, from the group of variables and information related to the group of variables obtained at block 202, possible causality among these variables, take a further operation to screen the determined causality and finally obtain accurate causality; with respect to each variable in the group of variables obtained at block 202, determine a set of parent variables of the variable and determine the causality among variables based on the type of the variable and the set of parent variables of the variable; the set of parent variables of a variable is a set of variables on which a value of the variable relies, i.e., the variable has causality with a variable in the set of parent variables of the variable; use the causal model obtained at block 204 to determine a causal sequence between variables in the group of variables obtained at block 202, and determine the causality based on the determined causal sequence; only when there is a causal sequence between two variables, the two variables might have causality; use the causal model obtained at block 204 to determine the possible causal sequence between variables by a method like greedy search; first use the causal model obtained at block 204 to obtain an initial causal sequence between variables in the group of variables obtained at block 202; then, determine fitness of the initial causal sequence, wherein the fitness indicates a probability that the initial causal sequence correctly represents the causal sequence between variables; finally, determine the causal sequence between variables based on the fitness and the initial causal sequence; first generate a parent relationship graph of each variable in the group of variables based on the determined set of parent variables of each variable for variables in the group of variables obtained at block 202; then, determine the causal sequence between variables by using, e.g., a graph theory method based on the parent relationship graphs; use the causal model obtained at block 204 to further determine association between variables by a method such as greedy search, and determine the causality based on the determined causal sequence and association; first use the causal model obtained at block 204 to determine initial causality among variables in the group of variables obtained at block 202 based on the determined causal sequence between variables; then, conduct a conditional independence test on the initial causality; finally, determine the causality among variables based on a result of the conditional independence test and the initial causality; additionally, also first determine association between variables, and determine the initial causality based on the determined causal sequence between variables and the determined association between variables; since both the causal sequence and the association are used, the initial causality determined at this point will become more accurate; first obtain causal information about the group of variables, the causal information indicating partial causality among a part of variables in the group of variables and comprising expert knowledge integration; then, use the causal model obtained at block 204 to determine the causality among variables in the group of variables based on the determined causal sequence between variables and the causal information; it should be understood that the causal information is related to the group of variables as the observed data 110 and thus may be obtained in the operation shown at any of blocks 202, 204 and 206; additionally, in the operation, also first determine association between variables, and determine the causality among variables in the group of variables based on the determined causal sequence, association and causal information between variables, thereby determining the causality more accurately; determine the causality by at least one of a constraint-based solution and a search-based solution; typical constraint-based technical solutions mainly comprise a PC (PeterClark) algorithm and an inductive causation algorithm, etc., which may comprise an undirected graph learning stage and a direction learning stage; search-based solutions comprise, e.g., a greedy equivalence search (GES) solution; the determined causality among variables in the group of variables may take the form of a directed acyclic graph, the directed acyclic graph comprising nodes and edges, a node representing a variable in the group of variables, and an edge representing causality among variables; ¶¶ [0042]-[0078] with FIG. 3: propose a structural equation model for mixed data types, the model allowing the causal mechanism to be non-linear and thus supporting a wider range of practical applications; the causal structure may be identified from a data distribution that follows the model, and the identified causal structure may be displayed through a directed acyclic graph; a maximum likelihood estimator is further proposed, the maximum likelihood estimator being used to select the causal sequence between variables rather than the causal structure, and results of the maximum likelihood estimator having sequential consistency; the sequential space is much smaller than the directed acyclic graph space and is easy to search; moreover, if the sequence between variables is known, the causal structure learning may be attributed to variable selection; this may be solved by sparse regression or (conditional) independence test, so that the causal structure learning may be performed with less computational overhead; further proposes an efficient sequential search method that benefits from a novel sequential space cutting method; with a maximum likelihood estimator, the method constructs a factor optimization problem, in which the causal sequence may be recovered using greedy search; to further accelerate sequential search, a graph lasso equipped with a kernel alignment method is first used to learn the sparse skeleton between variables, and the skeleton is projected into a series of topological ordering constraints to reduce the search space; with the method, the causality among variables may be accurately determined; in the initial modeling stage 310, a mixed non-linear causal model will be built, the model describing a non-linear relationship between mixed discrete variables and continuous variables, and the identifiability of the model being proved with further content; the mixed non-linear causal model may be defined through Equation (1); after the initial modeling stage 310, the flow proceeds to a causal sequence determining stage 320; a real causal structure may be identified from the joint distribution that follows the mixed non-linear causal model; the structural learning problem with directed acyclic graph constraints may be converted into the problem of learning the best sequence between variables, which seems easier because the sequential space is much smaller than the directed acyclic graph space; once the sequence is determined, acyclic constraints may be enforced by constraining the parent of a variable to be a subset of a variable before the variable; causal structure learning may be attributed to variable selection, which may be solved by sparse regression or (conditional) independence test; the real causal sequence may be identified from the joint distribution that follows the mixed non-linear causal model, and then the fitting of the sequence of mixed binary variables and continuous variables may be evaluated by the Mixed Nonlinear Information Criterion (MNIC), wherein MNIC scores are sequentially consistent; the operation in the causal sequence determining stage 320 may be performed based on the MNIC estimator, so that possible causal sequences between variables may be determined for model inputs comprising different types of variables; an additional sequence determining method 330 may be applied to the causal sequence determining stage 320 to achieve the effect of accelerating causality inference, the additional sequence determining method 330 comprising spatial clipping based on variable group sequence constraints; after the causal sequence determining stage 320, the flow proceeds to a causal structure learning stage 350, and then the causality 130 may be generated as output; a three-stage algorithm may be used to estimate the causal structure of observed data, i.e., to perform the operations in the causal sequence determining stage 320 and the causal structure learning stage 350; first, a graph lasso equipped with a kernel alignment method is used to learn the sparse skeleton between variables, and then the skeleton is projected to a series of topological ordering constraints so as to reduce the search space; next, under sequence constraints, greedy search is used in feasible space to estimate E in Equation (4); finally, a kernel-based conditional independence (KCI) test is used to trim edges so as to recover
G
full,min from
G
full; the whole algorithm is briefed in Algorithm 1; Stage 1: Generate Topological Ordering Constraints: (a) use Equations (9) and (10) to build a precision matrix Θ; (b) extract M SCC (i.e., Strong Connection Components) from Θ; (c) assign a random group sequence, and build a sequential constraint set C; Stage 2: Estimate the Causal Sequence (Corresponding to the Causal Sequence Determining Stage 320); Stage 3: Remove Extra Edges (Corresponding to the Causal Structure Learning Stage 350); in stage 3 of Algorithm 1, not only a method based on independence judgment may be used to remove edges, but also a feature selecting method may be used; the causal sequence determining stage 320 may comprise search spatial cutting: kernel alignment may be used to measure the similarity between two kernel functions, and also may be used to generate a pseudo-correlation matrix between random variables; Equation (9) is used to generate the pseudo-correlation matrix A on the observed data X=(X1, …, Xn), wherein each element A(i,j) is kernel alignment between Xi and Xj; the RBF kernel is used for continuous variables, and the delta kernel is used for binary variables; then, A is introduced to the graph lasso so as to learn the precision matrix Θ shown in Equation (10), wherein Θij=0 represents that there is no direct edge between Xi and Xj; Strong Connection Components (SCC) are generated from Θ; since no edge connects different SCCs, topological sequences between SCCs may be assigned at random; these sequential constraints will be used to reduce the search space in the second stage; the causal structure learning stage 350 may comprise trimming: using a conditional independence test to trim pseudo-edges from the super directed acyclic graph; In the conditional independence test, there exist some hyper-parameters: the kernel width and regularization parameters used to construct the kernel matrix; for an unconditional independence test, since the continuous variable has been regularized to unit variance, the median of the paired distances of these points is used as the kernel width; for a conditional independence test, when the conditional set is small (i.e., s2), the median of the paired distances of these points is used as the kernel width; for the regularization parameter, an empirical value (10-3) is used, which shows good effect; when the conditional set is large, the extended multi-output Gaussian process regression is used to learn hyper-parameters by maximizing the total marginal likelihood).
Claim 5
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 2 and further discloses wherein the control unit outputs strength information indicating strength of a relationship between the first explanatory variable selected as the reason and the prediction result (DAISUKE, ¶¶ [0074]-[0081] with FIGS. 1, 5-6, and 7A-B: if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the prediction basis information 700 in Figure 7A includes one or more records consisting of an explanatory variable 701, an influence level 702, and a value 703; each record corresponds to one explanatory variable or one group; the 700 records of the prediction basis information are sorted in descending order of their absolute impact; explanatory variable 701 is identification information for an explanatory variable or group: influence 702 is the influence of the explanatory variable or group; the value 703 is the value of the explanatory variable, or the value of the explanatory variable included in the group; the prediction basis information 700 in Figure 7B includes one or more records consisting of explanatory variables 711, influence 712, value 713, element explanatory variables 714, element value 715, and element influence 716; each record corresponds to one explanatory variable or one aggregate explanatory variable; the element explanatory variable 714 is identification information for the explanatory variables included in the group; element value 715 is the values of the explanatory variables included in the group; element Influence 716 is the influence of the explanatory variables included in the group; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results).
Claim 6
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 2 and further discloses wherein the control unit selects a combination of at least two of the input variables as the reason for the prediction result (DAISUKE, ¶ [0054] with FIG. 4: the data-related indicator 403 shows the relationship between the values of the paired explanatory variables (i.e., the first explanatory variable 401 and the second explanatory variable 402); the influence-related indicator 404 shows the relationship between the influence levels of the paired explanatory variables; ¶ [0065]-[0081] with FIGS. 1, 3, 5-6, and 7A-B: when the prediction basis output unit 104 receives the data to be predicted, it calculates the degree of influence of the explanatory variables on the prediction result (step S603); the prediction basis output unit 104 acquires model information and calculates the degree of influence of each explanatory variable of the data to be predicted on the prediction result using mathematical methods; the result output unit 108 refers to the aggregate explanatory variable information 500 and selects a target record (step S604); the result output unit 108 aggregates the influence of the explanatory variables to be aggregated based on the target record (step S605); the result output unit 108 refers to the prediction basis data, obtains the influence of the explanatory variables corresponding to the element explanatory variables 502 of the target record, and sums up the obtained influence values; the result output unit 108 removes the field indicating the influence of the explanatory variable corresponding to the element explanatory variable 502 of the target record from the prediction basis data, and adds the group field to the prediction basis data; the result output unit 108 sets the total influence value for the group's field; by performing the same process for all groups, the predictive basis data is transformed into aggregated predictive basis data; if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the prediction basis information 700 in Figure 7A includes one or more records consisting of an explanatory variable 701, an influence level 702, and a value 703; each record corresponds to one explanatory variable or one group; the 700 records of the prediction basis information are sorted in descending order of their absolute impact; explanatory variable 701 is identification information for an explanatory variable or group: influence 702 is the influence of the explanatory variable or group; the value 703 is the value of the explanatory variable, or the value of the explanatory variable included in the group; the prediction basis information 700 in Figure 7B includes one or more records consisting of explanatory variables 711, influence 712, value 713, element explanatory variables 714, element value 715, and element influence 716; each record corresponds to one explanatory variable or one aggregate explanatory variable; the element explanatory variable 714 is identification information for the explanatory variables included in the group; element value 715 is the values of the explanatory variables included in the group; element Influence 716 is the influence of the explanatory variables included in the group; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results; ¶¶[0085]-[0093] with FIG. 1-3 and 8: the related indicator calculation unit 106 generates a list of combinations of explanatory variables (step S801); the related indicator calculation unit 106 selects a target pair from the list (step S802); the related indicator calculation unit 106 obtains the values of the explanatory variables that form the target pair from each record of the historical information 200 (step S803); the related indicator calculation unit 106 calculates data-related indicators by performing statistical analysis using the first group of temporary records (step S804); e.g., the related indicator calculation unit 106 calculates a correlation coefficient, which shows the correlation between the values of each explanatory variable, as a data-related indicator; the related indicator calculation unit 106 obtains the influence of the explanatory variables forming the target pair from each record of the historical information 300 (step S805); the related indicator calculation unit 106 calculates the influence-related indicator by performing a statistical analysis using the second set of temporary records (step S806); e.g., the related indicator calculation unit 106 calculates correlation coefficients, which show the correlation of the degree of influence of each explanatory variable, as influence-related indicators; the related indicator calculation unit 106 determines whether processing has been completed for all pairs registered in the list (step S807); if it is determined that processing is not complete for all pairs registered in the list, the related indicator calculation unit 106 returns to step S802 and performs the same processing; if it is determined that processing has been completed for all pairs registered in the list, the related indicator calculation unit 106 sends the registration record for each pair to the related indicator storage unit 113 (step S808); ¶¶ [0139]-[0146] with FIGS. 1, 4-5, and 11: steps S1104 and S1105 are processes for determining whether or not there is a relationship between the values of the paired explanatory variables; if the value of data-related indicator 403 is greater than the first threshold, or if the value of input data related indicator 405 is greater than the second threshold, it is determined that a correlation exists between the values of the paired explanatory variables; steps S1106 and S1108 are processes for determining whether or not there is a relationship between the influence levels of the paired explanatory variables; if the value of the influence-related index 404 is greater than the third threshold, or if the value of the input influence-related index 406 is greater than the fourth threshold, it is determined that a relationship exists between the influences of the paired explanatory variables; if there is a correlation between the values of the paired explanatory variables, and there is also a correlation between the influence levels of the paired explanatory variables, the aggregation variable determination unit 107 determines that aggregation is possible because there is a correlation between the paired explanatory variables; if the result of step S1110 is YES, the aggregate variable determination unit 107 may refer to the aggregate explanatory variable information 500 and aggregate multiple pairs based on the transitive property; this allows us to generate groups consisting of three or more explanatory variables; when presenting information that shows the basis for predicting an event using an arbitrary model, it is possible to determine multiple explanatory variables that can be aggregated, generate groups from the multiple explanatory variables, and present the aggregated influence for each group; this makes it easier to understand the basis for predictions by comparing the degree of impact).
Claim 7
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 6 and further discloses wherein the control unit outputs strength information indicating strength of a relationship between the at least two input variables included in the combination and the prediction result in association with information regarding the combination (DAISUKE, ¶ [0054] with FIG. 4: the data-related indicator 403 shows the relationship between the values of the paired explanatory variables (i.e., the first explanatory variable 401 and the second explanatory variable 402); the influence-related indicator 404 shows the relationship between the influence levels of the paired explanatory variables; ¶ [0065]-[0081] with FIGS. 1, 3, 5-6, and 7A-B: when the prediction basis output unit 104 receives the data to be predicted, it calculates the degree of influence of the explanatory variables on the prediction result (step S603); the prediction basis output unit 104 acquires model information and calculates the degree of influence of each explanatory variable of the data to be predicted on the prediction result using mathematical methods; the result output unit 108 refers to the aggregate explanatory variable information 500 and selects a target record (step S604); the result output unit 108 aggregates the influence of the explanatory variables to be aggregated based on the target record (step S605); the result output unit 108 refers to the prediction basis data, obtains the influence of the explanatory variables corresponding to the element explanatory variables 502 of the target record, and sums up the obtained influence values; the result output unit 108 removes the field indicating the influence of the explanatory variable corresponding to the element explanatory variable 502 of the target record from the prediction basis data, and adds the group field to the prediction basis data; the result output unit 108 sets the total influence value for the group's field; by performing the same process for all groups, the predictive basis data is transformed into aggregated predictive basis data; if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the prediction basis information 700 in Figure 7A includes one or more records consisting of an explanatory variable 701, an influence level 702, and a value 703; each record corresponds to one explanatory variable or one group; the 700 records of the prediction basis information are sorted in descending order of their absolute impact; explanatory variable 701 is identification information for an explanatory variable or group: influence 702 is the influence of the explanatory variable or group; the value 703 is the value of the explanatory variable, or the value of the explanatory variable included in the group; the prediction basis information 700 in Figure 7B includes one or more records consisting of explanatory variables 711, influence 712, value 713, element explanatory variables 714, element value 715, and element influence 716; each record corresponds to one explanatory variable or one aggregate explanatory variable; the element explanatory variable 714 is identification information for the explanatory variables included in the group; element value 715 is the values of the explanatory variables included in the group; element Influence 716 is the influence of the explanatory variables included in the group; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results; ¶¶[0085]-[0093] with FIG. 1-3 and 8: the related indicator calculation unit 106 generates a list of combinations of explanatory variables (step S801); the related indicator calculation unit 106 selects a target pair from the list (step S802); the related indicator calculation unit 106 obtains the values of the explanatory variables that form the target pair from each record of the historical information 200 (step S803); the related indicator calculation unit 106 calculates data-related indicators by performing statistical analysis using the first group of temporary records (step S804); e.g., the related indicator calculation unit 106 calculates a correlation coefficient, which shows the correlation between the values of each explanatory variable, as a data-related indicator; the related indicator calculation unit 106 obtains the influence of the explanatory variables forming the target pair from each record of the historical information 300 (step S805); the related indicator calculation unit 106 calculates the influence-related indicator by performing a statistical analysis using the second set of temporary records (step S806); e.g., the related indicator calculation unit 106 calculates correlation coefficients, which show the correlation of the degree of influence of each explanatory variable, as influence-related indicators; the related indicator calculation unit 106 determines whether processing has been completed for all pairs registered in the list (step S807); if it is determined that processing is not complete for all pairs registered in the list, the related indicator calculation unit 106 returns to step S802 and performs the same processing; if it is determined that processing has been completed for all pairs registered in the list, the related indicator calculation unit 106 sends the registration record for each pair to the related indicator storage unit 113 (step S808); ¶¶ [0139]-[0146] with FIGS. 1, 4-5, and 11: steps S1104 and S1105 are processes for determining whether or not there is a relationship between the values of the paired explanatory variables; if the value of data-related indicator 403 is greater than the first threshold, or if the value of input data related indicator 405 is greater than the second threshold, it is determined that a correlation exists between the values of the paired explanatory variables; steps S1106 and S1108 are processes for determining whether or not there is a relationship between the influence levels of the paired explanatory variables; if the value of the influence-related index 404 is greater than the third threshold, or if the value of the input influence-related index 406 is greater than the fourth threshold, it is determined that a relationship exists between the influences of the paired explanatory variables; if there is a correlation between the values of the paired explanatory variables, and there is also a correlation between the influence levels of the paired explanatory variables, the aggregation variable determination unit 107 determines that aggregation is possible because there is a correlation between the paired explanatory variables; if the result of step S1110 is YES, the aggregate variable determination unit 107 may refer to the aggregate explanatory variable information 500 and aggregate multiple pairs based on the transitive property; this allows us to generate groups consisting of three or more explanatory variables; when presenting information that shows the basis for predicting an event using an arbitrary model, it is possible to determine multiple explanatory variables that can be aggregated, generate groups from the multiple explanatory variables, and present the aggregated influence for each group; this makes it easier to understand the basis for predictions by comparing the degree of impact).
Claim 8
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 5 and further discloses wherein the control unit determines an order or a color on a display screen corresponding to the first explanatory variable based on the strength information, and outputs the display screen (DAISUKE, ¶¶ [0074]-[0081] with FIGS. 1, 5-6, and 7A-B: if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the prediction basis information 700 in Figure 7A includes one or more records consisting of an explanatory variable 701, an influence level 702, and a value 703; each record corresponds to one explanatory variable or one group; the 700 records of the prediction basis information are sorted in descending order of their absolute impact; explanatory variable 701 is identification information for an explanatory variable or group: influence 702 is the influence of the explanatory variable or group; the value 703 is the value of the explanatory variable, or the value of the explanatory variable included in the group; the prediction basis information 700 in Figure 7B includes one or more records consisting of explanatory variables 711, influence 712, value 713, element explanatory variables 714, element value 715, and element influence 716; each record corresponds to one explanatory variable or one aggregate explanatory variable; the element explanatory variable 714 is identification information for the explanatory variables included in the group; element value 715 is the values of the explanatory variables included in the group; element Influence 716 is the influence of the explanatory variables included in the group; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results; ¶¶ [0101]-[0106] and [0052]-[0054] with FIGS. 1, 4, and 10A-B: Figures 10A and 10B show an example of an operation screen 1000 presented to the user via the terminal 150; when the related indicator receiving unit 102 receives a request to set related indicators from the terminal 150, it displays the operation screen 1000 shown in Figure 10A via the terminal 150; the analysis information manipulation field 1010 is a field that displays a table with the same data structure as the analysis information 400, wherein the analysis information 400 stores one or more records consisting of a first explanatory variable 401, a second explanatory variable 402, a data-related indicator 403, an influence-related indicator 404, an input data-related indicator 405, and an input influence-related indicator 406, wherein the influence-related indicator 404 is a field that stores an indicator (influence-related indicator) that shows the relationship between the influence levels of the paired explanatory variables; the field name in the analysis information manipulation section 1010 is provided with a sort button for rearranging records; the user sets a value in the field to control whether pairs with high correlation indices due to spurious correlation are aggregated, or whether pairs with low correlation indices are aggregated).
Claim 9
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 6 and further discloses wherein the control unit outputs an interface for determining the combination of the input variables, and determines a combination of the input variables based on an operation corresponding to the interface (DAISUKE, ¶¶ [0074]-[0081] with FIGS. 1, 5-6, and 7A-B: if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the prediction basis information 700 in Figure 7A includes one or more records consisting of an explanatory variable 701, an influence level 702, and a value 703; each record corresponds to one explanatory variable or one group; the 700 records of the prediction basis information are sorted in descending order of their absolute impact; explanatory variable 701 is identification information for an explanatory variable or group: influence 702 is the influence of the explanatory variable or group; the value 703 is the value of the explanatory variable, or the value of the explanatory variable included in the group; the prediction basis information 700 in Figure 7B includes one or more records consisting of explanatory variables 711, influence 712, value 713, element explanatory variables 714, element value 715, and element influence 716; each record corresponds to one explanatory variable or one aggregate explanatory variable; the element explanatory variable 714 is identification information for the explanatory variables included in the group; element value 715 is the values of the explanatory variables included in the group; element Influence 716 is the influence of the explanatory variables included in the group; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results; ¶¶ [0101]-[0106] and [0052]-[0054] with FIGS. 1, 4, and 10A-B: Figures 10A and 10B show an example of an operation screen 1000 presented to the user via the terminal 150; when the related indicator receiving unit 102 receives a request to set related indicators from the terminal 150, it displays the operation screen 1000 shown in Figure 10A via the terminal 150; the analysis information manipulation field 1010 is a field that displays a table with the same data structure as the analysis information 400, wherein the analysis information 400 stores one or more records consisting of a first explanatory variable 401, a second explanatory variable 402, a data-related indicator 403, an influence-related indicator 404, an input data-related indicator 405, and an input influence-related indicator 406, wherein the influence-related indicator 404 is a field that stores an indicator (influence-related indicator) that shows the relationship between the influence levels of the paired explanatory variables; the field name in the analysis information manipulation section 1010 is provided with a sort button for rearranging records; the user sets a value in the field to control whether pairs with high correlation indices due to spurious correlation are aggregated, or whether pairs with low correlation indices are aggregated; ¶¶ [0119]-[0146] with FIG. 1, 4-5, and 11: initializes the aggregate explanatory variable information 500 (step S1101); selects a target record from the analysis information 400 (step S1102); determines whether the input data-related indicator 405 of the target record is blank or not (step S1103); if it is determined that the input data-related indicator 405 of the target record is blank, determines whether the value of the data-related indicator 403 of the target record is greater than the first threshold (step S1104); if the value of the data-related indicator 403 of the target record is determined to be less than or equal to the first threshold, proceeds to step S1110; otherwise, proceeds to step S1106; in step S1103, if it is determined that the input data-related index 405 of the target record is not blank, determines whether the value of the input data-related index 405 of the target record is greater than the second threshold (step S1105); if the value of the input data-related indicator 405 of the target record is determined to be less than or equal to the second threshold, proceeds to step S1110; otherwise, proceeds to step S1106; if the result of step S1104 or step S1105 is YES, determines whether the input influence-related index 406 of the target record is blank (step S1106); if it is determined that the input influence-related index 406 for the target record is blank, determines whether the value of the influence-related index 404 for the target record is greater than the third threshold (step S1107); if the value of the influence-related indicator 404 for the target record is determined to be below the third threshold, proceeds to step S1110; otherwise, proceeds to step S1109; in step S1106, if it is determined that the input influence-related index 406 for the target record is not blank, determines whether the value of the input influence-related index 406 for the target record is greater than the fourth threshold (step S1108); if the value of the input influence-related indicator 406 for the target record is determined to be less than or equal to the fourth threshold, proceeds to step S1110; otherwise, proceeds to step S1109; if the result of step S1107 or step S1108 is YES, generates group data for the pair of explanatory variables corresponding to the target record and transmits the group data to the aggregate variable storage unit 114 (step S1109); if the result of step S1104, step S1105, step S1107, or step S1108 is NO, or after the processing in step S1109 has been executed, determines whether or not the processing of all records of the analysis information 400 has been completed (step S1110); if it is determined that processing of all records in the analysis information 400 has not been completed, the aggregate variable determination unit 107 returns to step S1102 and performs the same processing; if it is determined that processing of all records in the analysis information 400 has been completed, the aggregate variable determination unit 107 terminates the aggregate variable determination process; steps S1104 and S1105 are processes for determining whether or not there is a relationship between the values of the paired explanatory variables; if the value of data-related indicator 403 is greater than the first threshold, or if the value of input data related indicator 405 is greater than the second threshold, it is determined that a correlation exists between the values of the paired explanatory variables; steps S1106 and S1108 are processes for determining whether or not there is a relationship between the influence levels of the paired explanatory variables; if the value of the influence-related index 404 is greater than the third threshold, or if the value of the input influence-related index 406 is greater than the fourth threshold, it is determined that a relationship exists between the influences of the paired explanatory variables; if there is a correlation between the values of the paired explanatory variables, and there is also a correlation between the influence levels of the paired explanatory variables, determines that aggregation is possible because there is a correlation between the paired explanatory variables; if the result of step S1110 is YES, refer to the aggregate explanatory variable information 500 and aggregate multiple pairs based on the transitive property; this allows us to generate groups consisting of three or more explanatory variables; when presenting information that shows the basis for predicting an event using an arbitrary model, it is possible to determine multiple explanatory variables that can be aggregated, generate groups from the multiple explanatory variables, and present the aggregated influence for each group; this makes it easier to understand the basis for predictions by comparing the degree of impact).
Claim 10
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 2 and further discloses wherein the control unit estimates a causal graph with an output variable indicating the prediction result as an objective variable for the plurality of the input variables, and selects the first explanatory variable as the reason from the input variables having a direct causal relationship with the objective variable (DAISUKE, ¶¶ [0023]-[0024]: data is generated as prediction basis data, which consists of values that evaluate the magnitude of the influence of each explanatory variable on the prediction result, i.e., the degree of influence of each explanatory variable; identifies explanatory variables to be aggregated by analyzing the relationships between explanatory variables, and aggregates the degree of influence of the identified explanatory variables; converts the prediction basis data into aggregated prediction basis data consisting of unaggregated and aggregated impact values; generates display information to present the prediction results and aggregated prediction basis data to the user, and transmits the display information to the terminal 150; ¶ [0034]: calculates the degree of influence of each explanatory variable on the prediction result and generates prediction basis data that includes the degrees of influence of multiple explanatory variables; ¶¶ [0036]-[0038]: calculates related indicators using the data to be predicted and the data on which the prediction is based; calculates the absolute value of the correlation coefficient calculated from the regression analysis as a related indicator; determines the explanatory variables to be aggregated based on the related indicator data; generates group data for the groups generated from the determined explanatory variables; generates aggregated prediction basis data by aggregating the influence of the explanatory variables to be aggregated based on the prediction basis data and group data; generates display information based on the prediction results and aggregated prediction basis data, and transmits the display information to the terminal 150; e.g., display information is generated to show the prediction results and bar graphs representing the influence of the explanatory variables and the influence of the groups; ¶¶ [0040]-[0044]: consider a model that performs classification into two classes (Class 0 and Class 1); the model will output a value indicating the probability of being in class 1 as the prediction result; denote the predicted result as Y, the baseline as Y0, and the degree of influence of the explanatory variable Xi on the predicted result as Yi, then according to Non-Patent Document 1, the degree of influence Yi is calculated to satisfy equation (1); the degree of influence Yi is a positive or negative real number, where positive values indicate the impact on predictions that will be classified as "Class 1," while negative values indicate the impact on predictions that will not be classified as "Class 1"; a larger absolute value of the influence indicates the magnitude of the explanatory variable's impact on the prediction; if the explanatory variables include those with correlations or other relationships, that is, if there are multiple explanatory variables related to a single factor, the influence of that factor on the prediction results is calculated as the influence of the multiple explanatory variables; analyzes the relationships between explanatory variables, identifies combinations of explanatory variables that are related, and aggregates the degree of influence based on those combinations of explanatory variables; this allows us to present predictive evidence that is easy for users to understand; ¶¶ [0065]-[0081] with FIGS. 1, 5-6, and 7A-B: when the prediction basis output unit 104 receives the data to be predicted, it calculates the degree of influence of the explanatory variables on the prediction result (step S603); the prediction basis output unit 104 acquires model information and calculates the degree of influence of each explanatory variable of the data to be predicted on the prediction result using mathematical methods; the result output unit 108 refers to the aggregate explanatory variable information 500 and selects a target record (step S604); the result output unit 108 aggregates the influence of the explanatory variables to be aggregated based on the target record (step S605); the result output unit 108 refers to the prediction basis data, obtains the influence of the explanatory variables corresponding to the element explanatory variables 502 of the target record, and sums up the obtained influence values; the result output unit 108 removes the field indicating the influence of the explanatory variable corresponding to the element explanatory variable 502 of the target record from the prediction basis data, and adds the group field to the prediction basis data; the result output unit 108 sets the total influence value for the group's field; by performing the same process for all groups, the predictive basis data is transformed into aggregated predictive basis data; the result output unit 108 determines whether processing has been completed for all records of the aggregated explanatory variable information 500 (step S606); if it is determined that processing is not complete for all records of the aggregated explanatory variable information 500, the result output unit 108 returns to step S604 and performs the same processing; if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the 700 records of the prediction basis information are sorted in descending order of their absolute impact; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results) (Jorstad, ABSTRACT: obtaining a plurality of values each corresponding to one of a plurality of variables; the plurality of variables include variables of interest; obtaining a prediction for the values from a model, determining metric(s) for each of the variables of interest, and determining one or more of the variables of interest to be one or more influential variables based on the metric(s) determined for each of the variables of interest; the variables include one or more non-influential variables that is/are different from the influential variable(s); the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); displaying in a graphical user interface or printing in a report an explanation identifying the influential variable(s) and/or a justification of the determination that the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); ¶¶ [0025]-[0057] with FIGS. 1 and 3-5: determine why the model 100 outputs a particular prediction; one or more of the input variables 102 may have a greater influence on the prediction 104 than the other input variables 102; the explanation procedure 300 identifies a number "i" of the most influential of the input variables 102 on the prediction 104; in block 310, identify one or more of the input variables 102 as being of interest; in block 315, select one of the input variables of interest; in block 330, generates one or more metrics for the input variable selected in block 315 (e.g., the input variable 121) by comparing the sample predictions with the actual prediction 104; in block 340, uses the metric(s) assigned to each of the input variables of interest in block 330 to identify the number "i" of the most influential input variables; e.g., identify the number "i" (e.g., three) of the input variables of interest having the largest ExpectedDownside metrics as being the most influential variables; at least a portion of those of the input variables of interest that are not identified as being most influential variables may be identified as or considered to be non-influential variables; in block 340, weight or rank the input variables of interest based on the metric(s); e.g., when the input variables of interest rank the input variables of interest based on the ExpectedDownside metric calculated for each of the input variables of interest are ranked, each input variable of interest appears only once in the ranking; independently of the characteristics and values of the input variable, assign a single rank to the input variable and include the input variable only once in the ranking; these ranks allow meaningful comparisons between continuous and categorical variables, with or without missing and special values; in block 350, display a graphical user interface 352 including the explanation 360 to the user (e.g., a consumer, a loan applicant, and the like) on a display device; in optional block 347, the text descriptions 500 may be identified for each of the most influential input variables; ¶¶ [0076] and [0078] with FIGS. 1 and 3: a useful property of explanation generation when applied to practical problems is that it can become unnecessary to compute sample values and corresponding sample predictions for all of the input variables; e.g., to satisfy certain regulatory requirements, it may be necessary to report only the top five most influential input variables in the explanation for each record; in block 310, identify one or more of the input variables 102 as being of interest; thus, in block 310, select only those of the input variables 102 having a value for the global measure of variable importance that exceeds a threshold value; alternatively, select only a predetermined number of the input variables 102 with the largest values for the global measure of variable importance; variable importance measures the impact of each input variable across the entire dataset and is often used to decide which input variables to include in a model during the model development process; explanations rank the input variables of interest based on their impact to individual predictions, which lead to decisions; explanations can be used to provide feedback to individual users or consumers during live processes; individual explanations can also be aggregated to form global measures of importance or to provide measures of importance for different partitions of the population; e.g., the data can be partitioned into different groupings of rejected populations, from those that are rejected most strongly to those that are rejected less strongly; these groupings can show systematic patterns as to what factors are causing these groups of individuals to be rejected; this information can be useful for the purpose of accountability in decision making as well as providing a greater understanding of model behavior) (Wei, ¶¶ [0020]-[0023] with FIG. 1: with traditional methods for causality determining, by setting Markov conditions and loyalty, a method based on conditional independence may identify the causal skeleton from the joint distribution, e.g., through a computer usage statistical test (conditional independent test), and direct edges to Markov equivalence classes through a series of rules (e.g., identifying v-shaped structures or colliders, avoiding loops, etc.); on the contrary, the causal mechanism and data distribution are described through a specific model category (an identifiable functional model or structural equation model (SEM)); if the data generating process belongs to such a model category, a complete causal diagram may be identified; constraint-based methods and score-based methods have been proposed for recovering causal structures from mixed data; propose a model for using mixed data types of data to determine causality among variables; the model may take variables and their set of parent variables as input, determine the causal order between variables and optionally determine an association, and further determine the causal relationship between variables, and then output the determined causality in the form of a directed acyclic graph; establish and use a functional model for mixed types of variables to determine the causality 130 through the observed data 110; ¶¶ [0026]-[0038] with FIG. 2: at block 202, obtains a group of variables; at block 204, obtains a causal model; the group of variables obtained at block 202 and information related to the group of variables, e.g., type of variables, other subset of variables and the number of variables, may be used as input of the causal model; the causal model obtained at block 204 is a mixed non-linear causal model, wherein linearity means that uniformity and superposition need to be satisfied; the causal model obtained at block 204 may determine, from the group of variables and information related to the group of variables obtained at block 202, possible causality among these variables, take a further operation to screen the determined causality and finally obtain accurate causality; with respect to each variable in the group of variables obtained at block 202, determine a set of parent variables of the variable and determine the causality among variables based on the type of the variable and the set of parent variables of the variable; the set of parent variables of a variable is a set of variables on which a value of the variable relies, i.e., the variable has causality with a variable in the set of parent variables of the variable; use the causal model obtained at block 204 to determine a causal sequence between variables in the group of variables obtained at block 202, and determine the causality based on the determined causal sequence; only when there is a causal sequence between two variables, the two variables might have causality; use the causal model obtained at block 204 to determine the possible causal sequence between variables by a method like greedy search; first use the causal model obtained at block 204 to obtain an initial causal sequence between variables in the group of variables obtained at block 202; then, determine fitness of the initial causal sequence, wherein the fitness indicates a probability that the initial causal sequence correctly represents the causal sequence between variables; finally, determine the causal sequence between variables based on the fitness and the initial causal sequence; first generate a parent relationship graph of each variable in the group of variables based on the determined set of parent variables of each variable for variables in the group of variables obtained at block 202; then, determine the causal sequence between variables by using, e.g., a graph theory method based on the parent relationship graphs; use the causal model obtained at block 204 to further determine association between variables by a method such as greedy search, and determine the causality based on the determined causal sequence and association; first use the causal model obtained at block 204 to determine initial causality among variables in the group of variables obtained at block 202 based on the determined causal sequence between variables; then, conduct a conditional independence test on the initial causality; finally, determine the causality among variables based on a result of the conditional independence test and the initial causality; additionally, also first determine association between variables, and determine the initial causality based on the determined causal sequence between variables and the determined association between variables; since both the causal sequence and the association are used, the initial causality determined at this point will become more accurate; first obtain causal information about the group of variables, the causal information indicating partial causality among a part of variables in the group of variables and comprising expert knowledge integration; then, use the causal model obtained at block 204 to determine the causality among variables in the group of variables based on the determined causal sequence between variables and the causal information; it should be understood that the causal information is related to the group of variables as the observed data 110 and thus may be obtained in the operation shown at any of blocks 202, 204 and 206; additionally, in the operation, also first determine association between variables, and determine the causality among variables in the group of variables based on the determined causal sequence, association and causal information between variables, thereby determining the causality more accurately; determine the causality by at least one of a constraint-based solution and a search-based solution; typical constraint-based technical solutions mainly comprise a PC (PeterClark) algorithm and an inductive causation algorithm, etc., which may comprise an undirected graph learning stage and a direction learning stage; search-based solutions comprise, e.g., a greedy equivalence search (GES) solution; the determined causality among variables in the group of variables may take the form of a directed acyclic graph, the directed acyclic graph comprising nodes and edges, a node representing a variable in the group of variables, and an edge representing causality among variables; ¶¶ [0042]-[0078] with FIG. 3: propose a structural equation model for mixed data types, the model allowing the causal mechanism to be non-linear and thus supporting a wider range of practical applications; the causal structure may be identified from a data distribution that follows the model, and the identified causal structure may be displayed through a directed acyclic graph; a maximum likelihood estimator is further proposed, the maximum likelihood estimator being used to select the causal sequence between variables rather than the causal structure, and results of the maximum likelihood estimator having sequential consistency; the sequential space is much smaller than the directed acyclic graph space and is easy to search; moreover, if the sequence between variables is known, the causal structure learning may be attributed to variable selection; this may be solved by sparse regression or (conditional) independence test, so that the causal structure learning may be performed with less computational overhead; further proposes an efficient sequential search method that benefits from a novel sequential space cutting method; with a maximum likelihood estimator, the method constructs a factor optimization problem, in which the causal sequence may be recovered using greedy search; to further accelerate sequential search, a graph lasso equipped with a kernel alignment method is first used to learn the sparse skeleton between variables, and the skeleton is projected into a series of topological ordering constraints to reduce the search space; with the method, the causality among variables may be accurately determined; in the initial modeling stage 310, a mixed non-linear causal model will be built, the model describing a non-linear relationship between mixed discrete variables and continuous variables, and the identifiability of the model being proved with further content; the mixed non-linear causal model may be defined through Equation (1); after the initial modeling stage 310, the flow proceeds to a causal sequence determining stage 320; a real causal structure may be identified from the joint distribution that follows the mixed non-linear causal model; the structural learning problem with directed acyclic graph constraints may be converted into the problem of learning the best sequence between variables, which seems easier because the sequential space is much smaller than the directed acyclic graph space; once the sequence is determined, acyclic constraints may be enforced by constraining the parent of a variable to be a subset of a variable before the variable; causal structure learning may be attributed to variable selection, which may be solved by sparse regression or (conditional) independence test; the real causal sequence may be identified from the joint distribution that follows the mixed non-linear causal model, and then the fitting of the sequence of mixed binary variables and continuous variables may be evaluated by the Mixed Nonlinear Information Criterion (MNIC), wherein MNIC scores are sequentially consistent; the operation in the causal sequence determining stage 320 may be performed based on the MNIC estimator, so that possible causal sequences between variables may be determined for model inputs comprising different types of variables; an additional sequence determining method 330 may be applied to the causal sequence determining stage 320 to achieve the effect of accelerating causality inference, the additional sequence determining method 330 comprising spatial clipping based on variable group sequence constraints; after the causal sequence determining stage 320, the flow proceeds to a causal structure learning stage 350, and then the causality 130 may be generated as output; a three-stage algorithm may be used to estimate the causal structure of observed data, i.e., to perform the operations in the causal sequence determining stage 320 and the causal structure learning stage 350; first, a graph lasso equipped with a kernel alignment method is used to learn the sparse skeleton between variables, and then the skeleton is projected to a series of topological ordering constraints so as to reduce the search space; next, under sequence constraints, greedy search is used in feasible space to estimate E in Equation (4); finally, a kernel-based conditional independence (KCI) test is used to trim edges so as to recover
G
full,min from
G
full; the whole algorithm is briefed in Algorithm 1; Stage 1: Generate Topological Ordering Constraints: (a) use Equations (9) and (10) to build a precision matrix Θ; (b) extract M SCC (i.e., Strong Connection Components) from Θ; (c) assign a random group sequence, and build a sequential constraint set C; Stage 2: Estimate the Causal Sequence (Corresponding to the Causal Sequence Determining Stage 320); Stage 3: Remove Extra Edges (Corresponding to the Causal Structure Learning Stage 350); in stage 3 of Algorithm 1, not only a method based on independence judgment may be used to remove edges, but also a feature selecting method may be used; the causal sequence determining stage 320 may comprise search spatial cutting: kernel alignment may be used to measure the similarity between two kernel functions, and also may be used to generate a pseudo-correlation matrix between random variables; Equation (9) is used to generate the pseudo-correlation matrix A on the observed data X=(X1, …, Xn), wherein each element A(i,j) is kernel alignment between Xi and Xj; the RBF kernel is used for continuous variables, and the delta kernel is used for binary variables; then, A is introduced to the graph lasso so as to learn the precision matrix Θ shown in Equation (10), wherein Θij=0 represents that there is no direct edge between Xi and Xj; Strong Connection Components (SCC) are generated from Θ; since no edge connects different SCCs, topological sequences between SCCs may be assigned at random; these sequential constraints will be used to reduce the search space in the second stage; the causal structure learning stage 350 may comprise trimming: using a conditional independence test to trim pseudo-edges from the super directed acyclic graph; In the conditional independence test, there exist some hyper-parameters: the kernel width and regularization parameters used to construct the kernel matrix; for an unconditional independence test, since the continuous variable has been regularized to unit variance, the median of the paired distances of these points is used as the kernel width; for a conditional independence test, when the conditional set is small (i.e., s2), the median of the paired distances of these points is used as the kernel width; for the regularization parameter, an empirical value (10-3) is used, which shows good effect; when the conditional set is large, the extended multi-output Gaussian process regression is used to learn hyper-parameters by maximizing the total marginal likelihood).
Claim 20
DAISUKE in view of Jorstad discloses all the elements as stated in Claim 19 and further discloses wherein the control unit selects the second explanatory variable as a reason for the prediction result from among the plurality of input variables based on information as to whether the input variable and the prediction result are (DAISUKE, ¶ [0106] with FIG. 10A: the user sets a value in the field to control whether pairs with high correlation indices due to spurious correlation are aggregated, or whether pairs with low correlation indices are aggregated; ¶¶ [0023]-[0024]: data is generated as prediction basis data, which consists of values that evaluate the magnitude of the influence of each explanatory variable on the prediction result, i.e., the degree of influence of each explanatory variable; identifies explanatory variables to be aggregated by analyzing the relationships between explanatory variables, and aggregates the degree of influence of the identified explanatory variables; converts the prediction basis data into aggregated prediction basis data consisting of unaggregated and aggregated impact values; generates display information to present the prediction results and aggregated prediction basis data to the user, and transmits the display information to the terminal 150; ¶ [0034]: calculates the degree of influence of each explanatory variable on the prediction result and generates prediction basis data that includes the degrees of influence of multiple explanatory variables; ¶¶ [0036]-[0038]: calculates related indicators using the data to be predicted and the data on which the prediction is based; calculates the absolute value of the correlation coefficient calculated from the regression analysis as a related indicator; determines the explanatory variables to be aggregated based on the related indicator data; generates group data for the groups generated from the determined explanatory variables; generates aggregated prediction basis data by aggregating the influence of the explanatory variables to be aggregated based on the prediction basis data and group data; generates display information based on the prediction results and aggregated prediction basis data, and transmits the display information to the terminal 150; e.g., display information is generated to show the prediction results and bar graphs representing the influence of the explanatory variables and the influence of the groups; ¶¶ [0040]-[0044]: consider a model that performs classification into two classes (Class 0 and Class 1); the model will output a value indicating the probability of being in class 1 as the prediction result; denote the predicted result as Y, the baseline as Y0, and the degree of influence of the explanatory variable Xi on the predicted result as Yi, then according to Non-Patent Document 1, the degree of influence Yi is calculated to satisfy equation (1); the degree of influence Yi is a positive or negative real number, where positive values indicate the impact on predictions that will be classified as "Class 1," while negative values indicate the impact on predictions that will not be classified as "Class 1"; a larger absolute value of the influence indicates the magnitude of the explanatory variable's impact on the prediction; if the explanatory variables include those with correlations or other relationships, that is, if there are multiple explanatory variables related to a single factor, the influence of that factor on the prediction results is calculated as the influence of the multiple explanatory variables; analyzes the relationships between explanatory variables, identifies combinations of explanatory variables that are related, and aggregates the degree of influence based on those combinations of explanatory variables; this allows us to present predictive evidence that is easy for users to understand; ¶¶ [0065]-[0081] with FIGS. 1, 5-6, and 7A-B: when the prediction basis output unit 104 receives the data to be predicted, it calculates the degree of influence of the explanatory variables on the prediction result (step S603); the prediction basis output unit 104 acquires model information and calculates the degree of influence of each explanatory variable of the data to be predicted on the prediction result using mathematical methods; the result output unit 108 refers to the aggregate explanatory variable information 500 and selects a target record (step S604); the result output unit 108 aggregates the influence of the explanatory variables to be aggregated based on the target record (step S605); the result output unit 108 refers to the prediction basis data, obtains the influence of the explanatory variables corresponding to the element explanatory variables 502 of the target record, and sums up the obtained influence values; the result output unit 108 removes the field indicating the influence of the explanatory variable corresponding to the element explanatory variable 502 of the target record from the prediction basis data, and adds the group field to the prediction basis data; the result output unit 108 sets the total influence value for the group's field; by performing the same process for all groups, the predictive basis data is transformed into aggregated predictive basis data; the result output unit 108 determines whether processing has been completed for all records of the aggregated explanatory variable information 500 (step S606); if it is determined that processing is not complete for all records of the aggregated explanatory variable information 500, the result output unit 108 returns to step S604 and performs the same processing; if it is determined that processing has been completed for all records of the aggregated explanatory variable information 500, the result output unit 108 generates display information to present the prediction basis information 700, 710 as shown in Figure 7A or Figure 7B, and transmits the display information to the terminal 150 (step S607); the 700 records of the prediction basis information are sorted in descending order of their absolute impact; by presenting the influence of groups generated by aggregating related explanatory variables, it becomes easier to compare influences; by presenting the influence of each group, it is possible to evaluate the factors that affect the prediction results) (Jorstad, ABSTRACT: obtaining a plurality of values each corresponding to one of a plurality of variables; the plurality of variables include variables of interest; obtaining a prediction for the values from a model, determining metric(s) for each of the variables of interest, and determining one or more of the variables of interest to be one or more influential variables based on the metric(s) determined for each of the variables of interest; the variables include one or more non-influential variables that is/are different from the influential variable(s); the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); displaying in a graphical user interface or printing in a report an explanation identifying the influential variable(s) and/or a justification of the determination that the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); ¶¶ [0025]-[0057] with FIGS. 1 and 3-5: determine why the model 100 outputs a particular prediction; one or more of the input variables 102 may have a greater influence on the prediction 104 than the other input variables 102; the explanation procedure 300 identifies a number "i" of the most influential of the input variables 102 on the prediction 104; in block 310, identify one or more of the input variables 102 as being of interest; in block 315, select one of the input variables of interest; in block 330, generates one or more metrics for the input variable selected in block 315 (e.g., the input variable 121) by comparing the sample predictions with the actual prediction 104; in block 340, uses the metric(s) assigned to each of the input variables of interest in block 330 to identify the number "i" of the most influential input variables; e.g., identify the number "i" (e.g., three) of the input variables of interest having the largest ExpectedDownside metrics as being the most influential variables; at least a portion of those of the input variables of interest that are not identified as being most influential variables may be identified as or considered to be non-influential variables; in block 340, weight or rank the input variables of interest based on the metric(s); e.g., when the input variables of interest rank the input variables of interest based on the ExpectedDownside metric calculated for each of the input variables of interest are ranked, each input variable of interest appears only once in the ranking; independently of the characteristics and values of the input variable, assign a single rank to the input variable and include the input variable only once in the ranking; these ranks allow meaningful comparisons between continuous and categorical variables, with or without missing and special values; in block 350, display a graphical user interface 352 including the explanation 360 to the user (e.g., a consumer, a loan applicant, and the like) on a display device; in optional block 347, the text descriptions 500 may be identified for each of the most influential input variables; ¶¶ [0076] and [0078] with FIGS. 1 and 3: a useful property of explanation generation when applied to practical problems is that it can become unnecessary to compute sample values and corresponding sample predictions for all of the input variables; e.g., to satisfy certain regulatory requirements, it may be necessary to report only the top five most influential input variables in the explanation for each record; in block 310, identify one or more of the input variables 102 as being of interest; thus, in block 310, select only those of the input variables 102 having a value for the global measure of variable importance that exceeds a threshold value; alternatively, select only a predetermined number of the input variables 102 with the largest values for the global measure of variable importance; variable importance measures the impact of each input variable across the entire dataset and is often used to decide which input variables to include in a model during the model development process; explanations rank the input variables of interest based on their impact to individual predictions, which lead to decisions; explanations can be used to provide feedback to individual users or consumers during live processes; individual explanations can also be aggregated to form global measures of importance or to provide measures of importance for different partitions of the population; e.g., the data can be partitioned into different groupings of rejected populations, from those that are rejected most strongly to those that are rejected less strongly; these groupings can show systematic patterns as to what factors are causing these groups of individuals to be rejected; this information can be useful for the purpose of accountability in decision making as well as providing a greater understanding of model behavior).
DAISUKE in view of Jorstad fails to explicitly disclose wherein selects the second explanatory variable based on whether the input variable and the prediction result are pseudo correlations.
Wei teaches a system and a method relating to determining causality (Wei, ¶ [0001]), wherein selects the first explanatory variable based on whether the input variable and the prediction result are pseudo correlations (Wei, ¶¶ [0020]-[0023] with FIG. 1: with traditional methods for causality determining, by setting Markov conditions and loyalty, a method based on conditional independence may identify the causal skeleton from the joint distribution, e.g., through a computer usage statistical test (conditional independent test), and direct edges to Markov equivalence classes through a series of rules (e.g., identifying v-shaped structures or colliders, avoiding loops, etc.); on the contrary, the causal mechanism and data distribution are described through a specific model category (an identifiable functional model or structural equation model (SEM)); if the data generating process belongs to such a model category, a complete causal diagram may be identified; constraint-based methods and score-based methods have been proposed for recovering causal structures from mixed data; propose a model for using mixed data types of data to determine causality among variables; the model may take variables and their set of parent variables as input, determine the causal order between variables and optionally determine an association, and further determine the causal relationship between variables, and then output the determined causality in the form of a directed acyclic graph; establish and use a functional model for mixed types of variables to determine the causality 130 through the observed data 110; ¶¶ [0026]-[0038] with FIG. 2: at block 202, obtains a group of variables; at block 204, obtains a causal model; the group of variables obtained at block 202 and information related to the group of variables, e.g., type of variables, other subset of variables and the number of variables, may be used as input of the causal model; the causal model obtained at block 204 is a mixed non-linear causal model, wherein linearity means that uniformity and superposition need to be satisfied; the causal model obtained at block 204 may determine, from the group of variables and information related to the group of variables obtained at block 202, possible causality among these variables, take a further operation to screen the determined causality and finally obtain accurate causality; with respect to each variable in the group of variables obtained at block 202, determine a set of parent variables of the variable and determine the causality among variables based on the type of the variable and the set of parent variables of the variable; the set of parent variables of a variable is a set of variables on which a value of the variable relies, i.e., the variable has causality with a variable in the set of parent variables of the variable; use the causal model obtained at block 204 to determine a causal sequence between variables in the group of variables obtained at block 202, and determine the causality based on the determined causal sequence; only when there is a causal sequence between two variables, the two variables might have causality; use the causal model obtained at block 204 to determine the possible causal sequence between variables by a method like greedy search; first use the causal model obtained at block 204 to obtain an initial causal sequence between variables in the group of variables obtained at block 202; then, determine fitness of the initial causal sequence, wherein the fitness indicates a probability that the initial causal sequence correctly represents the causal sequence between variables; finally, determine the causal sequence between variables based on the fitness and the initial causal sequence; first generate a parent relationship graph of each variable in the group of variables based on the determined set of parent variables of each variable for variables in the group of variables obtained at block 202; then, determine the causal sequence between variables by using, e.g., a graph theory method based on the parent relationship graphs; use the causal model obtained at block 204 to further determine association between variables by a method such as greedy search, and determine the causality based on the determined causal sequence and association; first use the causal model obtained at block 204 to determine initial causality among variables in the group of variables obtained at block 202 based on the determined causal sequence between variables; then, conduct a conditional independence test on the initial causality; finally, determine the causality among variables based on a result of the conditional independence test and the initial causality; additionally, also first determine association between variables, and determine the initial causality based on the determined causal sequence between variables and the determined association between variables; since both the causal sequence and the association are used, the initial causality determined at this point will become more accurate; first obtain causal information about the group of variables, the causal information indicating partial causality among a part of variables in the group of variables and comprising expert knowledge integration; then, use the causal model obtained at block 204 to determine the causality among variables in the group of variables based on the determined causal sequence between variables and the causal information; it should be understood that the causal information is related to the group of variables as the observed data 110 and thus may be obtained in the operation shown at any of blocks 202, 204 and 206; additionally, in the operation, also first determine association between variables, and determine the causality among variables in the group of variables based on the determined causal sequence, association and causal information between variables, thereby determining the causality more accurately; determine the causality by at least one of a constraint-based solution and a search-based solution; typical constraint-based technical solutions mainly comprise a PC (PeterClark) algorithm and an inductive causation algorithm, etc., which may comprise an undirected graph learning stage and a direction learning stage; search-based solutions comprise, e.g., a greedy equivalence search (GES) solution; the determined causality among variables in the group of variables may take the form of a directed acyclic graph, the directed acyclic graph comprising nodes and edges, a node representing a variable in the group of variables, and an edge representing causality among variables; ¶¶ [0042]-[0078] with FIG. 3: propose a structural equation model for mixed data types, the model allowing the causal mechanism to be non-linear and thus supporting a wider range of practical applications; the causal structure may be identified from a data distribution that follows the model, and the identified causal structure may be displayed through a directed acyclic graph; a maximum likelihood estimator is further proposed, the maximum likelihood estimator being used to select the causal sequence between variables rather than the causal structure, and results of the maximum likelihood estimator having sequential consistency; the sequential space is much smaller than the directed acyclic graph space and is easy to search; moreover, if the sequence between variables is known, the causal structure learning may be attributed to variable selection; this may be solved by sparse regression or (conditional) independence test, so that the causal structure learning may be performed with less computational overhead; further proposes an efficient sequential search method that benefits from a novel sequential space cutting method; with a maximum likelihood estimator, the method constructs a factor optimization problem, in which the causal sequence may be recovered using greedy search; to further accelerate sequential search, a graph lasso equipped with a kernel alignment method is first used to learn the sparse skeleton between variables, and the skeleton is projected into a series of topological ordering constraints to reduce the search space; with the method, the causality among variables may be accurately determined; in the initial modeling stage 310, a mixed non-linear causal model will be built, the model describing a non-linear relationship between mixed discrete variables and continuous variables, and the identifiability of the model being proved with further content; the mixed non-linear causal model may be defined through Equation (1); after the initial modeling stage 310, the flow proceeds to a causal sequence determining stage 320; a real causal structure may be identified from the joint distribution that follows the mixed non-linear causal model; the structural learning problem with directed acyclic graph constraints may be converted into the problem of learning the best sequence between variables, which seems easier because the sequential space is much smaller than the directed acyclic graph space; once the sequence is determined, acyclic constraints may be enforced by constraining the parent of a variable to be a subset of a variable before the variable; causal structure learning may be attributed to variable selection, which may be solved by sparse regression or (conditional) independence test; the real causal sequence may be identified from the joint distribution that follows the mixed non-linear causal model, and then the fitting of the sequence of mixed binary variables and continuous variables may be evaluated by the Mixed Nonlinear Information Criterion (MNIC), wherein MNIC scores are sequentially consistent; the operation in the causal sequence determining stage 320 may be performed based on the MNIC estimator, so that possible causal sequences between variables may be determined for model inputs comprising different types of variables; an additional sequence determining method 330 may be applied to the causal sequence determining stage 320 to achieve the effect of accelerating causality inference, the additional sequence determining method 330 comprising spatial clipping based on variable group sequence constraints; after the causal sequence determining stage 320, the flow proceeds to a causal structure learning stage 350, and then the causality 130 may be generated as output; a three-stage algorithm may be used to estimate the causal structure of observed data, i.e., to perform the operations in the causal sequence determining stage 320 and the causal structure learning stage 350; first, a graph lasso equipped with a kernel alignment method is used to learn the sparse skeleton between variables, and then the skeleton is projected to a series of topological ordering constraints so as to reduce the search space; next, under sequence constraints, greedy search is used in feasible space to estimate E in Equation (4); finally, a kernel-based conditional independence (KCI) test is used to trim edges so as to recover
G
full,min from
G
full; the whole algorithm is briefed in Algorithm 1; Stage 1: Generate Topological Ordering Constraints: (a) use Equations (9) and (10) to build a precision matrix Θ; (b) extract M SCC (i.e., Strong Connection Components) from Θ; (c) assign a random group sequence, and build a sequential constraint set C; Stage 2: Estimate the Causal Sequence (Corresponding to the Causal Sequence Determining Stage 320); Stage 3: Remove Extra Edges (Corresponding to the Causal Structure Learning Stage 350); in stage 3 of Algorithm 1, not only a method based on independence judgment may be used to remove edges, but also a feature selecting method may be used; the causal sequence determining stage 320 may comprise search spatial cutting: kernel alignment may be used to measure the similarity between two kernel functions, and also may be used to generate a pseudo-correlation matrix between random variables; Equation (9) is used to generate the pseudo-correlation matrix A on the observed data X=(X1, …, Xn), wherein each element A(i,j) is kernel alignment between Xi and Xj; the RBF kernel is used for continuous variables, and the delta kernel is used for binary variables; then, A is introduced to the graph lasso so as to learn the precision matrix Θ shown in Equation (10), wherein Θij=0 represents that there is no direct edge between Xi and Xj; Strong Connection Components (SCC) are generated from Θ; since no edge connects different SCCs, topological sequences between SCCs may be assigned at random; these sequential constraints will be used to reduce the search space in the second stage; the causal structure learning stage 350 may comprise trimming: using a conditional independence test to trim pseudo-edges from the super directed acyclic graph; In the conditional independence test, there exist some hyper-parameters: the kernel width and regularization parameters used to construct the kernel matrix; for an unconditional independence test, since the continuous variable has been regularized to unit variance, the median of the paired distances of these points is used as the kernel width; for a conditional independence test, when the conditional set is small (i.e., s2), the median of the paired distances of these points is used as the kernel width; for the regularization parameter, an empirical value (10-3) is used, which shows good effect; when the conditional set is large, the extended multi-output Gaussian process regression is used to learn hyper-parameters by maximizing the total marginal likelihood).
DAISUKE in view of Jorstad, and Wei are analogous art because they are from the same field of endeavor, a system and a method relating to determining causality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Wei to DAISUKE in view of Jorstad. Motivation for doing so would determine causality accurately, efficiently, and effectively.
Claim 21
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 20 and further discloses wherein the control unit selects the input variable having a pseudo correlation with the prediction result or the input variable that becomes conditionally independent as the second explanatory variable (Wei, ¶¶ [0042]-[0078] with FIG. 3: propose a structural equation model for mixed data types, the model allowing the causal mechanism to be non-linear and thus supporting a wider range of practical applications; the causal structure may be identified from a data distribution that follows the model, and the identified causal structure may be displayed through a directed acyclic graph; a maximum likelihood estimator is further proposed, the maximum likelihood estimator being used to select the causal sequence between variables rather than the causal structure, and results of the maximum likelihood estimator having sequential consistency; the sequential space is much smaller than the directed acyclic graph space and is easy to search; moreover, if the sequence between variables is known, the causal structure learning may be attributed to variable selection; this may be solved by sparse regression or (conditional) independence test, so that the causal structure learning may be performed with less computational overhead; further proposes an efficient sequential search method that benefits from a novel sequential space cutting method; with a maximum likelihood estimator, the method constructs a factor optimization problem, in which the causal sequence may be recovered using greedy search; to further accelerate sequential search, a graph lasso equipped with a kernel alignment method is first used to learn the sparse skeleton between variables, and the skeleton is projected into a series of topological ordering constraints to reduce the search space; with the method, the causality among variables may be accurately determined; in the initial modeling stage 310, a mixed non-linear causal model will be built, the model describing a non-linear relationship between mixed discrete variables and continuous variables, and the identifiability of the model being proved with further content; the mixed non-linear causal model may be defined through Equation (1); after the initial modeling stage 310, the flow proceeds to a causal sequence determining stage 320; a real causal structure may be identified from the joint distribution that follows the mixed non-linear causal model; the structural learning problem with directed acyclic graph constraints may be converted into the problem of learning the best sequence between variables, which seems easier because the sequential space is much smaller than the directed acyclic graph space; once the sequence is determined, acyclic constraints may be enforced by constraining the parent of a variable to be a subset of a variable before the variable; causal structure learning may be attributed to variable selection, which may be solved by sparse regression or (conditional) independence test; the real causal sequence may be identified from the joint distribution that follows the mixed non-linear causal model, and then the fitting of the sequence of mixed binary variables and continuous variables may be evaluated by the Mixed Nonlinear Information Criterion (MNIC), wherein MNIC scores are sequentially consistent; the operation in the causal sequence determining stage 320 may be performed based on the MNIC estimator, so that possible causal sequences between variables may be determined for model inputs comprising different types of variables; an additional sequence determining method 330 may be applied to the causal sequence determining stage 320 to achieve the effect of accelerating causality inference, the additional sequence determining method 330 comprising spatial clipping based on variable group sequence constraints; after the causal sequence determining stage 320, the flow proceeds to a causal structure learning stage 350, and then the causality 130 may be generated as output; a three-stage algorithm may be used to estimate the causal structure of observed data, i.e., to perform the operations in the causal sequence determining stage 320 and the causal structure learning stage 350; first, a graph lasso equipped with a kernel alignment method is used to learn the sparse skeleton between variables, and then the skeleton is projected to a series of topological ordering constraints so as to reduce the search space; next, under sequence constraints, greedy search is used in feasible space to estimate E in Equation (4); finally, a kernel-based conditional independence (KCI) test is used to trim edges so as to recover
G
full,min from
G
full; the whole algorithm is briefed in Algorithm 1; Stage 1: Generate Topological Ordering Constraints: (a) use Equations (9) and (10) to build a precision matrix Θ; (b) extract M SCC (i.e., Strong Connection Components) from Θ; (c) assign a random group sequence, and build a sequential constraint set C; Stage 2: Estimate the Causal Sequence (Corresponding to the Causal Sequence Determining Stage 320); Stage 3: Remove Extra Edges (Corresponding to the Causal Structure Learning Stage 350); in stage 3 of Algorithm 1, not only a method based on independence judgment may be used to remove edges, but also a feature selecting method may be used; the causal sequence determining stage 320 may comprise search spatial cutting: kernel alignment may be used to measure the similarity between two kernel functions, and also may be used to generate a pseudo-correlation matrix between random variables; Equation (9) is used to generate the pseudo-correlation matrix A on the observed data X=(X1, …, Xn), wherein each element A(i,j) is kernel alignment between Xi and Xj; the RBF kernel is used for continuous variables, and the delta kernel is used for binary variables; then, A is introduced to the graph lasso so as to learn the precision matrix Θ shown in Equation (10), wherein Θij=0 represents that there is no direct edge between Xi and Xj; Strong Connection Components (SCC) are generated from Θ; since no edge connects different SCCs, topological sequences between SCCs may be assigned at random; these sequential constraints will be used to reduce the search space in the second stage; the causal structure learning stage 350 may comprise trimming: using a conditional independence test to trim pseudo-edges from the super directed acyclic graph; In the conditional independence test, there exist some hyper-parameters: the kernel width and regularization parameters used to construct the kernel matrix; for an unconditional independence test, since the continuous variable has been regularized to unit variance, the median of the paired distances of these points is used as the kernel width; for a conditional independence test, when the conditional set is small (i.e., s2), the median of the paired distances of these points is used as the kernel width; for the regularization parameter, an empirical value (10-3) is used, which shows good effect; when the conditional set is large, the extended multi-output Gaussian process regression is used to learn hyper-parameters by maximizing the total marginal likelihood).
Claims 13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over DAISUKE in view of Jorstad and Wei as applied to Claim 2 above, and further in view of Blöbaum et al. ("ESTIMATION OF INTERVENTIONAL EFFECTS OF FEATURES ON PREDICTION", 2017 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 25–28, 2017, TOKYO, JAPAN, pp. 1-6), hereinafter Blöbaum .
Claim 13
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claim 2 and further discloses wherein the control unit calculates an (Jorstad, ABSTRACT: obtaining a plurality of values each corresponding to one of a plurality of variables; the plurality of variables include variables of interest; obtaining a prediction for the values from a model, determining metric(s) for each of the variables of interest, and determining one or more of the variables of interest to be one or more influential variables based on the metric(s) determined for each of the variables of interest; the variables include one or more non-influential variables that is/are different from the influential variable(s); the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); displaying in a graphical user interface or printing in a report an explanation identifying the influential variable(s) and/or a justification of the determination that the influential variable(s) has/have a greater influence on the prediction than the non-influential variable(s); ¶¶ [0025]-[0057] with FIGS. 1 and 3-5: determine why the model 100 outputs a particular prediction; one or more of the input variables 102 may have a greater influence on the prediction 104 than the other input variables 102; the explanation procedure 300 identifies a number "i" of the most influential of the input variables 102 on the prediction 104; the model 100 may be a black box to the explanation procedure 300; the explanation procedure 300 may be used to provide explanations for on-line streaming data in addition to off-line batch data; the prediction surface is re-sampled for each input variable separately to evaluate the impact of changes to that input variable on the resulting prediction; in other words, all of the input variables 102 are held constant except one, which is sampled at different values; in first block 305, execute the model 100 on the original unmodified input record 106 to obtain the actual prediction 104; in block 310, identify one or more of the input variables 102 as being of interest; in block 315, select one of the input variables of interest; the explanation procedure 300 operates on a locality principle and modifies one of the input variables 102 at a time; in block 320, obtain sample values of the input variable selected in block 315 (e.g., the input variable 121); in block 325, execute the model 100 once for each of the sample values but uses the original value of each of the other input variables included in the input record 106; i.e., the value of the input variable selected in block 315 (e.g., the input variable 121) is changed but the values of all other input variables are left unchanged; in block 325, obtain sample predictions that are each associated with a different one of the sample values; in block 330, generates one or more metrics for the input variable selected in block 315 (e.g., the input variable 121) by comparing the sample predictions with the actual prediction 104; the metric(s) generated in block 330 may include one or more of the following: (a) a minimum ("Min") metric, which is the smallest predicted value expected from modifying the input variable; (b) a maximum ("Max") metric, which the largest predicted value expected from modifying the input variable; (c) a range, which is equal to a difference between the Max metric and the Min metric; (d) an upside metric, which equals the Max metric minus the Actual with values less than zero being truncated to zero and represents an amount of potential increase in predicted values expected by changing the input variable; (e) a downside metric, which equals the Actual minus the Min metric with values less than zero being truncated to zero and represents an amount of potential decrease in predicted values expected by changing the input variable; (f) an ExpectedUpside metric, which is equal to sum(probability(bin)*UpDifference), where the Up Difference equals (sampled(bin)-Actual) for all the bins where sampled(bin)>Actual and zero for all the bins where sampled(bin)≤Actual; and (g) an ExpectedDownside metric, which is equal to sum(probability(bin)*DownDifference) where the DownDifference equals (Actual-sampled(bin)) for all the bins where sampled(bin)<Actual and zero for all the bins where sampled(bin)≥Actual; the actual prediction 104 is referred to above as "Actual" above, the term "bin" identifies the sample value selected from one of the sample bins 322 and the term "sampled(bin)" is the sample prediction obtained for the sample value; the term "probability (bin)" is the prior probability associated with the sample value identified by the term "bin"; the ExpectedUpside and ExpectedDownside metrics use the prior probabilities to adjust the expected values, treating each of the input variables of interest as a discrete random variable; those sample values that are unlikely based on the prior distribution of an original dataset are penalized; in block 330, assign the metric(s) to or associates the metric(s) with the input variable selected in block 315 (e.g., the input variable 121); then, in decision block 335, determine whether it has evaluated all of the input variables of interest; the decision in decision block 335 is "YES" when the explanation computing device 302 has evaluated all of the input variables of interest; otherwise, the decision in decision block 335 is "NO"; when the decision in decision block 335 is "NO," return to block 315 and selects another one of the input variables of interest; on the other hand, when the decision in decision block 335 is "YES," the metric(s) for each of the input variables of interest has been collected and advances to block 340; in block 340, uses the metric(s) assigned to each of the input variables of interest in block 330 to identify the number "i" of the most influential input variables; e.g., identify the number "i" (e.g., three) of the input variables of interest having the largest ExpectedDownside metrics as being the most influential variables; at least a portion of those of the input variables of interest that are not identified as being most influential variables may be identified as or considered to be non-influential variables; in block 340, weight or rank the input variables of interest based on the metric(s); e.g., when the input variables of interest rank the input variables of interest based on the ExpectedDownside metric calculated for each of the input variables of interest are ranked, each input variable of interest appears only once in the ranking; independently of the characteristics and values of the input variable, assign a single rank to the input variable and include the input variable only once in the ranking; these ranks allow meaningful comparisons between continuous and categorical variables, with or without missing and special values; in optional block 345, identify one or more changes to the input variables of interest that would result in a more desirable prediction; i.e., identify one or more corrective actions that can be taken; in optional block 347, identify text descriptions 500 for each of the most influential input variables identified in block 340; the text descriptions 500 may include or be associated with reason codes; a reason code may indicate a negative condition or reason for rejection; in block 350, display a graphical user interface 352 including the explanation 360 to the user (e.g., a consumer, a loan applicant, and the like) on a display device; in optional block 347, the text descriptions 500 may be identified for each of the most influential input variables; ¶¶ [0076]-[0080] with FIGS. 1, 3-4, and 9-11: a useful property of explanation generation when applied to practical problems is that it can become unnecessary to compute sample values and corresponding sample predictions for all of the input variables; e.g., to satisfy certain regulatory requirements, it may be necessary to report only the top five most influential input variables in the explanation for each record; in block 310, identify one or more of the input variables 102 as being of interest; thus, in block 310, select only those of the input variables 102 having a value for the global measure of variable importance that exceeds a threshold value; alternatively, select only a predetermined number of the input variables 102 with the largest values for the global measure of variable importance; variable importance measures the impact of each input variable across the entire dataset and is often used to decide which input variables to include in a model during the model development process; explanations rank the input variables of interest based on their impact to individual predictions, which lead to decisions; explanations can be used to provide feedback to individual users or consumers during live processes; individual explanations can also be aggregated to form global measures of importance or to provide measures of importance for different partitions of the population; e.g., the data can be partitioned into different groupings of rejected populations, from those that are rejected most strongly to those that are rejected less strongly; these groupings can show systematic patterns as to what factors are causing these groups of individuals to be rejected; this information can be useful for the purpose of accountability in decision making as well as providing a greater understanding of model behavior) (Wei, ¶¶ [0039]-[0041]: the performance of the application system may be improved based on the causality determined at block 206. Specifically, causal variables affecting the causality in the application system may be adjusted or monitored, so that the performance of the application system may be improved; the variable that most affects the power loss may be adjusted first based on the found causality; in this way, the performance of the power transmission system may be improved; the running of the application system may be adjusted based on the causality determined at block 206, e.g., regarding the machining system, if the causality among various attributes and the fact whether the product is qualified has been determined, then the attribute that most affects unqualified products may be adjusted first based on the found causality).
DAISUKE in view of Jorstad and Wei fails to explicitly disclose wherein calculates an intervention effect in a case of intervening in the first explanatory variable selected as the reason.
Blöbaum teaches a system and a method for the interpretability of prediction mechanisms (ABSTRACT), wherein calculates an intervention effect in a case of intervening in the first explanatory variable selected as the reason (Blöbaum, ABSTRACT of Page 1: propose a framework that identifies the feature with the greatest causal influence on the prediction and estimates the necessary causal intervention of a feature such that a desired prediction is obtained; Section 1 of Pages 1-2: consider how to manipulate a data generation process such that a certain prediction; e.g., consider a classifier that classifies a patient as healthy or sick based on several observations such as a certain medicine’s dose, blood pressure, and heart rate, which feature must we change so that the patient is classified as healthy; a feature with a high coefficient in the prediction model may not necessarily be the best choice for manipulation in order to heal the patient; instead, we must consider the causal relationship between the features; draw a connection between a given prediction mechanism and the causal relationship of the predictors and target; propose a framework consisting of three parts: 1) integration of the causal structure of the prediction model into the causal structure of the underlying data generation process; 2) identification of the feature with the greatest causal influence on the prediction; 3) estimation of the necessary manipulation of a specific feature to achieve a certain prediction; Section 2 with FIG. 1(a) of Pages 1-3: the causal structure of a set of variables
V
can be represented by a directed acyclic graph (DAG), where each variable is a vertex; the direct causal influence of a variable Xi on a variable Xj is indicated by an arrow between these two vertices; each connection has a certain strength that specifies the strength of the influence of Xi on Xj; the entire causal structure can, therefore, be represented by a matrix B ∈ Rn×n, where bij represents the connection strength of Xj to Xi; if bij = 0, no direct causal relationship exists; the sets of parent and child variables of Xi are denoted by pa(Xi) and ch(Xi), respectively, and include all directly connected ancestors and descendants of Xi, respectively; a variable with no ancestors is called a root variable; given a causal graph with variables
V
, causal graphical models allow description of the manner in which the variables in
V
change if a variable Xi ∈
V
is fixed to a certain value c; e.g., in Figure 1(a), if we fix the value of X1 to c, the changes in the remaining variables can be described by the causal connections defined by B; in order to analyze this behavior, we first define the term intervention as do(Xi = c), which represents the fixing of Xi to a constant value c; note that, in the general context of causality, it is assumed that the causal structure and parameters remain unchanged under any kind of intervention; the expected intervention effect of Xi on another variable Xj is called the total effect and is expressed as equation (4), which is the conditional expectation of Xj given the intervention do(Xi = c); if we are only interested in the extent to which the intervention on Xi affects Xj , we can estimate this influence via the derivative ∂/∂Xi E[Xj |do(Xi=c)]; these regression coefficients can be obtained by minimizing the squared error; the coefficient ωji is also called the causal effect or the intervention effect of Xi on Xi; take this causal influence into account in order to find the optimal c for the intervention; Section 4 of Pages 3-5 with FIG. 1(b) of Page 3: proposed framework consists of three steps: (a) training of a prediction model f with predictors X and target Y and integration of this model into the causal structure of the data generation process, which is defined by the set of variables
V
and causal connection matrix B; (b) identification of the feature Xi ∈
X
having the greatest intervention effect on the prediction
Y
^
= f(X); (c) estimation of the intervention do(Xi = c) such that the expectation of the prediction of the post-intervention observations is equal or close to a specified value; let Y ∈
V
denote the target variable we want to predict; if we consider the underlying causal structure of the discriminative prediction mechanisms defined by (1) and (2), a clear causal relationship exists, where all variables in
X
are parents of the prediction
Y
^
;
Y
^
may have a very different causal relationship with its predictors compared to the actual target Y in the data generation process; if we wish to combine the causal structure of
V
defined by B with the causal structure of the prediction mechanism, we must introduce a new variable
Y
^
with pa(
Y
^
) =
X
and connection strengths defined by the coefficients of the prediction model, as illustrated in Figure 1(b); investigate the manner in which the intervention of a variable affects the prediction; determine what kind of intervention is necessary for a specific prediction; propagate through the causal graph and estimate the total effects of the intervention on all variables; formalize this problem as a minimization problem in order to acquire a more general formulation of the problem that facilitates the easy addition of further constraints; the intervention is optimal with respect to
Y
^
in the original population and not with respect to Y; however, to find the optimal intervention with respect to Y , it is simply necessary to set
X
= pa(Y) as predictors; the general concept behind the algorithm is to propagate through the causal network while monitoring the extent by which each variable would be affected by an intervention on variable Xi; therefore, the optimal intervention may yield values outside the domains of some variables).
DAISUKE in view of Jorstad and Wei, and Blöbaum are analogous art because they are from the same field of endeavor, a system and a method for the interpretability of prediction mechanisms. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Blöbaum to DAISUKE in view of Jorstad and Wei, Motivation for doing so would provide new insights toward improved understanding of the relationship between a prediction problem and the corresponding prediction mechanism, and a further and deeper understanding of prediction mechanisms from a causal perspective (Blöbaum, Section 3 of Page 3 and Section 6 of Page 6).
Claim 18
DAISUKE in view of Jorstad, Wei, and Blöbaum discloses all the elements as stated in Claim 13 and further discloses wherein the control unit acquires a selection operation for the output first explanatory variable, and calculates an intervention effect for the first explanatory variable selected by the selection operation (Blöbaum, ABSTRACT of Page 1: propose a framework that identifies the feature with the greatest causal influence on the prediction and estimates the necessary causal intervention of a feature such that a desired prediction is obtained; Section 1 of Pages 1-2: consider how to manipulate a data generation process such that a certain prediction; e.g., consider a classifier that classifies a patient as healthy or sick based on several observations such as a certain medicine’s dose, blood pressure, and heart rate, which feature must we change so that the patient is classified as healthy; a feature with a high coefficient in the prediction model may not necessarily be the best choice for manipulation in order to heal the patient; instead, we must consider the causal relationship between the features; draw a connection between a given prediction mechanism and the causal relationship of the predictors and target; propose a framework consisting of three parts: 1) integration of the causal structure of the prediction model into the causal structure of the underlying data generation process; 2) identification of the feature with the greatest causal influence on the prediction; 3) estimation of the necessary manipulation of a specific feature to achieve a certain prediction; Section 2 with FIG. 1(a) of Pages 1-3: the causal structure of a set of variables
V
can be represented by a directed acyclic graph (DAG), where each variable is a vertex; the direct causal influence of a variable Xi on a variable Xj is indicated by an arrow between these two vertices; each connection has a certain strength that specifies the strength of the influence of Xi on Xj; the entire causal structure can, therefore, be represented by a matrix B ∈ Rn×n, where bij represents the connection strength of Xj to Xi; if bij = 0, no direct causal relationship exists; the sets of parent and child variables of Xi are denoted by pa(Xi) and ch(Xi), respectively, and include all directly connected ancestors and descendants of Xi, respectively; a variable with no ancestors is called a root variable; given a causal graph with variables
V
, causal graphical models allow description of the manner in which the variables in
V
change if a variable Xi ∈
V
is fixed to a certain value c; e.g., in Figure 1(a), if we fix the value of X1 to c, the changes in the remaining variables can be described by the causal connections defined by B; in order to analyze this behavior, we first define the term intervention as do(Xi = c), which represents the fixing of Xi to a constant value c; note that, in the general context of causality, it is assumed that the causal structure and parameters remain unchanged under any kind of intervention; the expected intervention effect of Xi on another variable Xj is called the total effect and is expressed as equation (4), which is the conditional expectation of Xj given the intervention do(Xi = c); if we are only interested in the extent to which the intervention on Xi affects Xj , we can estimate this influence via the derivative ∂/∂Xi E[Xj |do(Xi=c)]; these regression coefficients can be obtained by minimizing the squared error; the coefficient ωji is also called the causal effect or the intervention effect of Xi on Xi; take this causal influence into account in order to find the optimal c for the intervention; Section 4 of Pages 3-5 with FIG. 1(b) of Page 3: proposed framework consists of three steps: (a) training of a prediction model f with predictors X and target Y and integration of this model into the causal structure of the data generation process, which is defined by the set of variables
V
and causal connection matrix B; (b) identification of the feature Xi ∈
X
having the greatest intervention effect on the prediction
Y
^
= f(X); (c) estimation of the intervention do(Xi = c) such that the expectation of the prediction of the post-intervention observations is equal or close to a specified value; let Y ∈
V
denote the target variable we want to predict; if we consider the underlying causal structure of the discriminative prediction mechanisms defined by (1) and (2), a clear causal relationship exists, where all variables in
X
are parents of the prediction
Y
^
;
Y
^
may have a very different causal relationship with its predictors compared to the actual target Y in the data generation process; if we wish to combine the causal structure of
V
defined by B with the causal structure of the prediction mechanism, we must introduce a new variable
Y
^
with pa(
Y
^
) =
X
and connection strengths defined by the coefficients of the prediction model, as illustrated in Figure 1(b); investigate the manner in which the intervention of a variable affects the prediction; determine what kind of intervention is necessary for a specific prediction; propagate through the causal graph and estimate the total effects of the intervention on all variables; formalize this problem as a minimization problem in order to acquire a more general formulation of the problem that facilitates the easy addition of further constraints; the intervention is optimal with respect to
Y
^
in the original population and not with respect to Y; however, to find the optimal intervention with respect to Y , it is simply necessary to set
X
= pa(Y) as predictors; the general concept behind the algorithm is to propagate through the causal network while monitoring the extent by which each variable would be affected by an intervention on variable Xi; therefore, the optimal intervention may yield values outside the domains of some variables).
Claims 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over DAISUKE in view of Jorstad as applied to Claim 1 above, and further in view of WANG et al. (US 2021/0240174 A1, filed on 02/08/2019 ), hereinafter WANG.
Claim 14
DAISUKE in view of Jorstad discloses all the elements as stated in Claim 1 and except failing to explicitly disclose wherein the input variable includes information acquired by a sensor.
WANG teaches a system and a method relating to estimating causal relations (WANG, ¶¶ [0005] and [0045]), wherein the input variable includes information acquired by a sensor (WANG, ¶¶ [0045]-[0049] with FIG. 1: estimates a causal structure model 215 from training data 150 (S11); the causal structure model 215 is a model having a directional graph structure indicating causal relations between pieces of data regarding a monitoring-target system; the causal structure model 215 can be represented by a causal loop diagram (Causal Loop Diagram: CLD); the causal structure model 215 is also called a CLD model; in the causal structure model 215, the start-point node of an edge is a cause node (explanatory-variable node), and the end-point node of the edge is a result node (response-variable node); the result node represents a response variable, and the cause node represents an explanatory variable; a variable of the result node is represented by a regression formula of a variable of the cause node; the causal structure model 215 has a hierarchical structure; a cause node (explanatory variable) of an upper-layer result node (response variable) is a result node (response variable) of a still lower-layer cause node (explanatory variable); in the causal structure model 215, a top node 216 that is reached by tracing nodes according to directional edges indicates a monitoring-target response variable; inputs test data 170 to the estimated (generated) causal structure model 215 and acquires an estimated value of the monitoring-target response variable (S12); e.g., the test data 170 is real time data, and includes data indicating the current status and situation of the monitoring target system; compares the estimated value of the monitoring-target response variable generated by the causal structure model 215 with a measurement value of the monitoring-target response variable included in the test data 170, and determines the degree of deviation therebetween; on the basis of the computed degree of deviation, decides whether an abnormality has occurred in the monitoring target system (S13); in a case in which it is decided that an abnormality has occurred in the monitoring-target system, a node which is deemed to be an abnormality cause is searched for in the causal structure model 215 to which the test data 170 has been input; the causal structure model 215 has a directional graph structure indicating relations between causes and results, and the remote monitoring system can estimate a potential abnormality cause efficiently with high accuracy; ¶¶ [0062]-[0066] with FIGS. 1 and 4-7: estimates a monitoring-target response variable value from test data according to a causal structure model; FIG. 7 illustrates one test data record 171 in the test data database 233; the remote monitoring system keeps monitoring the monitoring-target system, and regularly adds test data records 171 to the test data database 233; the test data record 171 includes information of statuses of the monitoring-target system and values (measurement values) obtained from data measured by sensors; the test data database 233 includes time fields 172, fields 173 indicating statuses (operational statuses) of the monitoring-target system, and fields 174 of measurement data; e.g., it is supposed here that the monitoring-target system is an air conditioner; the time fields 172 indicate times at which data of test data records are acquired; each of the fields 173 indicating operational statuses indicates an item of the setting of the air conditioner; e.g., the fields 173 indicating operational statuses include fields of ON/OFF of the power supply, fields of the operation mode such as cooling or heating, fields of the fan speed, fields of the vertical swing, and the like; the fields 174 of measurement data include, e.g., fields of the room temperature of a room where the air conditioner is installed, fields of the temperature of air taken in by the air conditioner, fields of the temperature of air discharged from the air conditioner, and the like; each of nodes in a causal structure model indicates a value of measurement data regarding the monitoring-target system; the causal structure model 215 indicates causal relations between pieces of the measurement data regarding the monitoring-target system; ¶¶ [0087]-[0088] and [0091] with FIGS. 10 and 12: the monitoring image 400 includes an acquisition time 401 of the test data record, a measurement value 402 of a monitoring-target response variable, an estimated value 403 of the response variable, and a normality/abnormality decision result 404; the value of the response variable is measured by a sensor A; the monitoring image 400 further includes a model check button 405, a button 406 for displaying a measurement value history and an estimated-value history, and a button 407 for displaying information of other sensor data (response variables); an example image 420 to be displayed in a case in which the button 406 is selected by the user on the monitoring image 400; the image 420 indicates a measurement value history, an estimated-value history, and an estimation-accuracy history of the monitoring-target response variable by sensor A; the image 420 includes a table 421 indicating the measurement value history, the estimated-value history, and the estimation- accuracy history, a graph 422 indicating the measurement value history (temporal changes), and a graph 423 indicating the estimated-value history (temporal changes); ¶ [0137] with FIG. 24: FIG. 24 illustrates a configuration example of the training data database 212; the training data database 212 includes a plurality of training data records 351 which includes information regarding statuses of the monitoring-target system, values obtained from data measured by sensors, and a maintenance record; the training data database 212 includes time fields 352, fields 353 indicating statuses (operational statuses) of the monitoring-target system, fields 354 of measurement data, and fields 355 of maintenance records).
DAISUKE in view of Jorstad, and WANG are analogous art because they are from the same field of endeavor, a system and a method relating to estimating causal relations. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of WANG to DAISUKE in view of Jorstad. Motivation for doing so would expand capability of identifying prediction causes to estimate causes of system abnormalities promptly with high accuracy (WANG, ¶ [0011]).
Claim 15
DAISUKE in view of Jorstad and WANG discloses all the elements as stated in Claim 14 and further discloses wherein the input variable includes information on an operating environment or an operating state of a device acquired by a sensor (WANG, ¶¶ [0045]-[0049] with FIG. 1: estimates a causal structure model 215 from training data 150 (S11); the causal structure model 215 is a model having a directional graph structure indicating causal relations between pieces of data regarding a monitoring-target system; the causal structure model 215 can be represented by a causal loop diagram (Causal Loop Diagram: CLD); the causal structure model 215 is also called a CLD model; in the causal structure model 215, the start-point node of an edge is a cause node (explanatory-variable node), and the end-point node of the edge is a result node (response-variable node); the result node represents a response variable, and the cause node represents an explanatory variable; a variable of the result node is represented by a regression formula of a variable of the cause node; the causal structure model 215 has a hierarchical structure; a cause node (explanatory variable) of an upper-layer result node (response variable) is a result node (response variable) of a still lower-layer cause node (explanatory variable); in the causal structure model 215, a top node 216 that is reached by tracing nodes according to directional edges indicates a monitoring-target response variable; inputs test data 170 to the estimated (generated) causal structure model 215 and acquires an estimated value of the monitoring-target response variable (S12); e.g., the test data 170 is real time data, and includes data indicating the current status and situation of the monitoring target system; compares the estimated value of the monitoring-target response variable generated by the causal structure model 215 with a measurement value of the monitoring-target response variable included in the test data 170, and determines the degree of deviation therebetween; on the basis of the computed degree of deviation, decides whether an abnormality has occurred in the monitoring target system (S13); in a case in which it is decided that an abnormality has occurred in the monitoring-target system, a node which is deemed to be an abnormality cause is searched for in the causal structure model 215 to which the test data 170 has been input; the causal structure model 215 has a directional graph structure indicating relations between causes and results, and the remote monitoring system can estimate a potential abnormality cause efficiently with high accuracy; ¶¶ [0062]-[0066] with FIGS. 1 and 4-7: estimates a monitoring-target response variable value from test data according to a causal structure model; FIG. 7 illustrates one test data record 171 in the test data database 233; the remote monitoring system keeps monitoring the monitoring-target system, and regularly adds test data records 171 to the test data database 233; the test data record 171 includes information of statuses of the monitoring-target system and values (measurement values) obtained from data measured by sensors; the test data database 233 includes time fields 172, fields 173 indicating statuses (operational statuses) of the monitoring-target system, and fields 174 of measurement data; e.g., it is supposed here that the monitoring-target system is an air conditioner; the time fields 172 indicate times at which data of test data records are acquired; each of the fields 173 indicating operational statuses indicates an item of the setting of the air conditioner; e.g., the fields 173 indicating operational statuses include fields of ON/OFF of the power supply, fields of the operation mode such as cooling or heating, fields of the fan speed, fields of the vertical swing, and the like; the fields 174 of measurement data include, e.g., fields of the room temperature of a room where the air conditioner is installed, fields of the temperature of air taken in by the air conditioner, fields of the temperature of air discharged from the air conditioner, and the like; each of nodes in a causal structure model indicates a value of measurement data regarding the monitoring-target system; the causal structure model 215 indicates causal relations between pieces of the measurement data regarding the monitoring-target system; ¶¶ [0087]-[0088] and [0091] with FIGS. 10 and 12: the monitoring image 400 includes an acquisition time 401 of the test data record, a measurement value 402 of a monitoring-target response variable, an estimated value 403 of the response variable, and a normality/abnormality decision result 404; the value of the response variable is measured by a sensor A; the monitoring image 400 further includes a model check button 405, a button 406 for displaying a measurement value history and an estimated-value history, and a button 407 for displaying information of other sensor data (response variables); an example image 420 to be displayed in a case in which the button 406 is selected by the user on the monitoring image 400; the image 420 indicates a measurement value history, an estimated-value history, and an estimation-accuracy history of the monitoring-target response variable by sensor A; the image 420 includes a table 421 indicating the measurement value history, the estimated-value history, and the estimation- accuracy history, a graph 422 indicating the measurement value history (temporal changes), and a graph 423 indicating the estimated-value history (temporal changes); ¶ [0137] with FIG. 24: FIG. 24 illustrates a configuration example of the training data database 212; the training data database 212 includes a plurality of training data records 351 which includes information regarding statuses of the monitoring-target system, values obtained from data measured by sensors, and a maintenance record; the training data database 212 includes time fields 352, fields 353 indicating statuses (operational statuses) of the monitoring-target system, fields 354 of measurement data, and fields 355 of maintenance records).
Claim 16
DAISUKE in view of Jorstad and WANG discloses all the elements as stated in Claim 15 and further discloses wherein the input variable includes information regarding temperature, humidity, voltage, current, electric power, or vibration acquired by a sensor, and the control unit selects at least one of the information regarding temperature, humidity, voltage, current, electric power, or vibration acquired by the sensor as the first explanatory variable (WANG, ¶¶ [0045]-[0049] with FIG. 1: estimates a causal structure model 215 from training data 150 (S11); the causal structure model 215 is a model having a directional graph structure indicating causal relations between pieces of data regarding a monitoring-target system; the causal structure model 215 can be represented by a causal loop diagram (Causal Loop Diagram: CLD); the causal structure model 215 is also called a CLD model; in the causal structure model 215, the start-point node of an edge is a cause node (explanatory-variable node), and the end-point node of the edge is a result node (response-variable node); the result node represents a response variable, and the cause node represents an explanatory variable; a variable of the result node is represented by a regression formula of a variable of the cause node; the causal structure model 215 has a hierarchical structure; a cause node (explanatory variable) of an upper-layer result node (response variable) is a result node (response variable) of a still lower-layer cause node (explanatory variable); in the causal structure model 215, a top node 216 that is reached by tracing nodes according to directional edges indicates a monitoring-target response variable; inputs test data 170 to the estimated (generated) causal structure model 215 and acquires an estimated value of the monitoring-target response variable (S12); e.g., the test data 170 is real time data, and includes data indicating the current status and situation of the monitoring target system; compares the estimated value of the monitoring-target response variable generated by the causal structure model 215 with a measurement value of the monitoring-target response variable included in the test data 170, and determines the degree of deviation therebetween; on the basis of the computed degree of deviation, decides whether an abnormality has occurred in the monitoring target system (S13); in a case in which it is decided that an abnormality has occurred in the monitoring-target system, a node which is deemed to be an abnormality cause is searched for in the causal structure model 215 to which the test data 170 has been input; the causal structure model 215 has a directional graph structure indicating relations between causes and results, and the remote monitoring system can estimate a potential abnormality cause efficiently with high accuracy; ¶¶ [0062]-[0066] with FIGS. 1 and 4-7: estimates a monitoring-target response variable value from test data according to a causal structure model; FIG. 7 illustrates one test data record 171 in the test data database 233; the remote monitoring system keeps monitoring the monitoring-target system, and regularly adds test data records 171 to the test data database 233; the test data record 171 includes information of statuses of the monitoring-target system and values (measurement values) obtained from data measured by sensors; the test data database 233 includes time fields 172, fields 173 indicating statuses (operational statuses) of the monitoring-target system, and fields 174 of measurement data; e.g., it is supposed here that the monitoring-target system is an air conditioner; the time fields 172 indicate times at which data of test data records are acquired; each of the fields 173 indicating operational statuses indicates an item of the setting of the air conditioner; e.g., the fields 173 indicating operational statuses include fields of ON/OFF of the power supply, fields of the operation mode such as cooling or heating, fields of the fan speed, fields of the vertical swing, and the like; the fields 174 of measurement data include, e.g., fields of the room temperature of a room where the air conditioner is installed, fields of the temperature of air taken in by the air conditioner, fields of the temperature of air discharged from the air conditioner, and the like; each of nodes in a causal structure model indicates a value of measurement data regarding the monitoring-target system; the causal structure model 215 indicates causal relations between pieces of the measurement data regarding the monitoring-target system; ¶¶ [0087]-[0088] and [0091] with FIGS. 10 and 12: the monitoring image 400 includes an acquisition time 401 of the test data record, a measurement value 402 of a monitoring-target response variable, an estimated value 403 of the response variable, and a normality/abnormality decision result 404; the value of the response variable is measured by a sensor A; the monitoring image 400 further includes a model check button 405, a button 406 for displaying a measurement value history and an estimated-value history, and a button 407 for displaying information of other sensor data (response variables); an example image 420 to be displayed in a case in which the button 406 is selected by the user on the monitoring image 400; the image 420 indicates a measurement value history, an estimated-value history, and an estimation-accuracy history of the monitoring-target response variable by sensor A; the image 420 includes a table 421 indicating the measurement value history, the estimated-value history, and the estimation- accuracy history, a graph 422 indicating the measurement value history (temporal changes), and a graph 423 indicating the estimated-value history (temporal changes); ¶ [0137] with FIG. 24: FIG. 24 illustrates a configuration example of the training data database 212; the training data database 212 includes a plurality of training data records 351 which includes information regarding statuses of the monitoring-target system, values obtained from data measured by sensors, and a maintenance record; the training data database 212 includes time fields 352, fields 353 indicating statuses (operational statuses) of the monitoring-target system, fields 354 of measurement data, and fields 355 of maintenance records).
Allowable Subject Matter
Claims 11-12 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101 and 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 11
DAISUKE in view of Jorstad and Wei discloses all the elements as stated in Claims 1-10 and 17-23.
Blöbaum discloses all the elements as stated in Claims 13 and 18.
WANG discloses all the elements as stated in Claims 14-16.
SOGAWA et al. (US 2021/0056449 A1, filed on 07/25/2018) discloses in ¶¶ [0001]-[0014] with FIGS. 5-6 that (1) a causal relation estimating device, a causal relation estimating method, and a causal relation estimating program for estimating a causal relation; (2) the causal relation means that there is a cause-effect relation between two or more objects; (3) the correlation means that there is a relevance between two or more objects; (4) the effect of each cause is represented by the direction of each arrow between variables having a causal relation; (5) it is common practice to make a prediction in consideration of a correlation of two or more variables; (6) there are various problems in the world, which can be solved by grasping a causal relation and measuring the degree of impact; (6) the statistical causal inference is a technique for estimating, from data, a causal structure G between variables and a causal parameter θ; (7) the causal structure G is a graph in which an influential relation between variables x is represented by a directed edge, and the causal parameter θ is a parameter related to the strength of the influential relation between variables x; (8) the causal structure can be estimated by an intervention operation to assign a specific value to any variable; (9) the intervention operation is so performed that intervention data on a variable when its top impact is ignored can be acquired; (10) use of this data enables the causal structure to be uniquely estimated; (10) FIG. 6 is an explanatory diagram illustrating an example of an intervention operation; e.g., such an intervention operation as to assign value C to variable x2 illustrating in FIG. 6 is so performed that the causal structure can be estimated by the intervention data when the influence of variable x1 is ignored; (11) the degree of influence on a specific variable y when a variable q with a certain intervention operation possible is changed without knowing a causal structure G should be able to be grasped with as few intervention operations as possible; (12) NPL 1 and NPL 2 disclose intervention methods for efficiently estimating a structure or parameters for overall causes and effects; (13) However, there may be a case where it will be enough just to be able to observe a value of a specific variable y even if the overall causal relation cannot be necessarily estimated in the actual situation; i.e., there is a case where it is enough just to be able to observe only the influence on the specific variable y to which attention is paid, rather than the causal structure G among all variables; e.g., in the example illustrated in FIG. 5, if it is enough just to be able to observe the influence on y when x1 as an intervening variable is changed, it will be preferred to be able to make modeling without strict consideration of relationships between, x1 to x6 and y; and (14) provide a causal relation estimating device, a causal relation estimating method, and a causal relation estimating program, which can effectively estimate a causal relation to a variable to which attention is paid. SOGAWA further discloses in ABSTRACT and ¶¶ [0015]-[0017] that (1) a query specification process of specifying a query as a combination of a variable, on which an intervention operation is performed for the causal relation, and a value of the variable; (2) an intervention data generating process of generating intervention data including a value of a target variable, acquired with an intervention operation based on the query; and (3) a causal relation updating process of updating the causal relation using the generated intervention data, wherein a query that minimizes an expected loss by updating is specified from among queries specified based on the expected loss representing an estimation error of the target variable by the query in the query specification process. SOGAWA also discloses in ¶¶ [0026]-[0058] with FIG. 1 that (1) the input unit 10 reads the observational data D stored in the storage unit 70, and inputs the read observational data D to the causal relation estimating unit 20; (2) the causal relation estimating unit 20 uses the input observational data D to estimate a model representing a causal relation (hereinafter referred to as a causal model), wherein the causal model is represented by a joint distribution P(θ, G) of a causal structure G and each of parameters (causal parameters) θ of the causal model; (3) the method of causing the causal relation estimating unit 20 to estimate the causal model is optional; e.g., the causal relation estimating unit 20 may use the observational data D to do Bayesian updating of P(G) and P(θ|G) expressed in Equation 1 below in order to estimate the causal model; (4) since the causal relation estimating unit 20 estimates the causal relation based on the observational data D alone, the causal structure G and the causal parameter θ cannot be uniquely identified, and therefore, it can be said that the causal relation estimated by the causal relation estimating unit 20 is a causal relation having ambiguity; (5) the query specification unit 30 specifies a combination of a variable, on which an intervention operation is performed for the causal relation, and a value of the variable (hereinafter referred to as a query); i.e., the query specification unit 30 specifies the variable used in the intervention operation and the value of the variable; (6) the query specification unit 30 specifies a query by paying attention to the ambiguity between the intervention operation and a specific variable y (hereinafter referred to as a target variable y) (i.e., the proneness of an estimation error between the intervention operation and the target variable y) in order to be able to grasp the degree of influence on the target variable y with as few intervention operations as possible; (7) when a causal model is updated using a query at the time of performing a certain intervention operation and a returned target variable, the query specification unit 30 evaluates how vague the relation between the query and the target variable is; (8) specifically, the query specification unit 30 evaluates an expected loss as a result of an error in the estimation of the query and the target variable; (9) the definition of the expected loss is optional; e.g., expectation uncertainty (uncertainty) or statistical uncertainty (entropy) is used; e.g., the expected loss by the query is expressed in Equation 4; i.e., the query specification unit 30 evaluates the ambiguity when the causal model is updated with y and X returned when the query is executed; (10) further, it can be said that expected values of y and X to be returned are calculated from a parameter distribution in the current causal model; (11) when the model expressed in Equation 4 is evaluated, e.g., the query specification unit 30 may calculate the expected loss by using a relational equation exemplified in Equation 5; (12) among queries specified based on the expected loss, the query specification unit 30 specifies such a query as to minimize the expected loss; (13) the lager the expected loss, the more ambiguous the relation between the query and the target variable (i.e., the higher the estimation error between the query and the target variable y); (14) therefore, from among queries largest in expected loss, the query specification unit 30 specifies a query capable of minimizing the expected loss by updating; e.g., when the expectation uncertainty expressed in Equation 4 is used as the expected loss, the query specification unit 30 may specify a query using Equation 6 which is indicated that a query q is decided to be used to minimize the expected loss among queries likely to cause the largest expected loss when a certain intervention operation is performed; (15) the query specification unit 30 specifies a query that minimizes the expected loss from among queries specified based on the expected loss representing the estimation error of the target variable by the queries, which can make the causal relation related to the target variable y clearer; (16) when a query is specified based on the expected loss, it is more preferred that a query largest in expected loss by updating should be specified; i.e., an evaluation is performed by focusing on the target variable y, rather than by applying evaluation criteria to the overall causal relation; (17) since the loss described above focuses only on the relation between an intervening variable and the target variable y, the query to be specified is used to update the causal model so that the causal relation to the target variable y can be made clear with few intervention operations; (18) the intervention data generating unit 40 acquires a value of the target variable y with an intervention operation based on the specified query; (19) then, the intervention data generating unit 40 generates data (hereinafter referred to as intervention data) including the acquired target variable y and the query; e.g., the intervention data generating unit 40 has only to acquire, as the value of the target variable y, the result of performing the intervention operation on the system of the causal relation to be estimated; (20) the causal relation updating unit 50 updates the causal relation using the generated intervention data; (21) specifically, the causal relation updating unit 50 updates the distribution P(G0, θ0) of the causal model with P(θ0|G0)P(G0); (22) the update is done on condition that the target variable y is observed based on the query, i.e., that any other x is not observed; (23) the method for the causal relation updating unit 50 to update the causal model is optional; e.g., Bayesian updating between incomplete data may be used; (24) the causal relation updating unit 50 may determine whether the number of updates exceeds a preset number of updates or not, or determine whether the expected loss is below a threshold value provided for the expected loss (uncertainty) or not; (25) when it is determined that update processing of the causal relation is repeated (e.g., when the number of updates does not exceed the preset number of updates, or when the expected loss exceeds the threshold value), the query specification unit 30, the intervention data generating unit 40, and the causal relation updating unit 50 repeat the processing described above; (26) the output unit 60 outputs the update results of the causal relation; and (27) the causal model output here is obtained by encoding the structure and parameters of the causal relation between X by focusing on the relation between Q and y. SOGAWA further teaches in ¶¶ [0063]-[] with FIGS. 1-2 that (1) the input unit 10 inputs observational data D (step S11); (2) the causal relation estimating unit 20 uses the input observational data D to estimate a causal model to serve as the basis (step S12); (3) the query specification unit 30 specifies a query to perform an intervention operation (step S13); (4) specifically, the query specification unit 30 specifies a query capable of minimizing the expected loss by updating from among queries specified based on the expected loss; (5) the intervention data generating unit 40 generates intervention data including a value of the target variable, acquired by the specified query, and the query (step S14); (6) the causal relation updating unit 50 updates the causal model using the generated intervention data (step S15); (7) the causal relation updating unit 50 determines whether update processing of the causal model is repeated or not (step S16); (8) when repetition is determined (Yes in step S16), the processing in step S13 and later is repeated; (9) on the other hand, when no repetition is determined (No in step S16), the output unit 60 outputs the updated causal model (step S17); (10) the query specification unit 30 specifies a query as a combination of a variable, on which an intervention operation is performed for the causal relation, and a value of the variable, and the intervention data generating unit 40 generates intervention data including a value of the target variable, acquired with an intervention operation based on the query; (11) then, the causal relation updating unit 50 uses the generated intervention data to update the causal relation; (12) on this occasion, the query specification unit 30 specifies a query that minimizes the expected loss by updating from among queries specified based on the expected loss representing the estimation error of the target variable by the query; and (13) this enables the causal relation to the variable, to which attention is paid, to be estimated efficiently; i.e., since uncertainty can be reduced efficiently by performing an intervention operation on the most uncertain part in the relation between the query q and the target variable y, the accuracy of modeling to represent the causal relation can be improved efficiently.
Yamamoto et al. ("The Role of Work Engagement in Organizational Commitment", Proceedings of the 2017 Spring Conference of the Japan Industrial Management Association, May 26, 2017, pp. 196-197) discloses in Section 3 of Pages 196-197 that (1) to examine the role of Work Engagement (WE), we conducted an exploratory survey using a questionnaire that excluded "Job Resources" (BO) from the JD-R model but covered 37 other constructs — including personality, engagement, organizational citizenship behavior, organizational commitment, job satisfaction, turnover intention, job characteristics, presenteeism, supervisor relationships, organizational justice, job resources, job demands, and organizational climate; (2) factor analysis was performed on each questionnaire, and 117 explanatory variables were extracted; (3) in the factor analysis, the number of factors was determined so that the internal uniformity was Cronbach's α-value of 0.80; (4) however, for questionnaires where the academic evidence for the Japanese translation is lacking, the factors used are those derived from existing research; (5) finally, a correlation analysis was performed on the obtained explanatory variables; (6) specifically, to eliminate spurious/pseudo correlations, partial correlation coefficients were calculated, and only those with p < 0.05 were selected.
Horimoto et al. (US 2020/0265919 A1, pub. date: 08/202/2020) discloses in ¶¶ [0053] and [0058]-[0062] with FIGS. 1, 2A-B, 3A-B, and 4 that (1) the analyzer 1 determines consistency between a known pathway having a network structure of an undirected graph and data on the amount of reaction, wherein undirected graphs to be analyzed by the analyzer 1 are those without a closed circuit; (2) a pathway has a network structure comprising substances as nodes linked to one another, and therefore an evaluation of consistency between a pathway and data on the amount of reaction is made with the pathway being considered as a network, wherein a "pathway" is a connection between substances which is found by experiment, and a "network" is a connection between substances in computational biology, and a term "graph" is also used herein, and is a term for mathematically expressing a network structure; (3) FIG. 2A shows an example of a known pathway, wherein the network consistency determination unit 14 divides the known pathway into subgraphs each comprising two nodes linked to each other as shown in FIG. 2B; (4) determines the first-order partial correlation coefficient between nodes in each subgraph based on data on the amount of reaction; (5) FIG. 3A shows an example of data on the amount of reaction to be analyzed, and expresses data on the amount of reaction in matrices by imitating microarrays; (6) data on the amount of reaction is quantitative data representing the extent of reaction of each substance caused by a predetermined treatment; (7) he correlation between two substances A and B, e.g., is determined by plotting the amount of reaction of the substance A and that of the substance B in each specimen, so that the correlation coefficient between the two substances can be determined; (8) FIG. 4 is a table including examples of correlation coefficients between the substances determined in the manner shown in FIG. 3B; (9) first-order partial correlation coefficients between substances are used for evaluating networks; (10) partial correlation coefficients indicate true correlations excluding the influence of other variables than two targeted variables, and can be calculated by a known method; (11) the network consistency determination unit 14 uses the partial correlation coefficient between each pair of nodes determined as above to determine the probability of an independence test between each pair of nodes in the known pathway; (12) the network consistency determination unit 14 then uses Fisher's combined probability to integrate the probabilities of the independence tests between the nodes as shown in FIG. 2B, and thereby determines a probability that represents the known pathway's independence from the data on the amount of reaction; (13) the use of partial correlation coefficients allows for detection of spurious correlations, which cannot be detected by using correlation coefficients, and for a more accurate estimation of the whole consistency; (14) while Fisher's combined probability is used to integrate the probabilities of the independence tests between the nodes, another combined probability such as Brown's combined probability can be used for the integration; (15) the network consistency determination unit 14 then randomly generates many networks each having the same number of nodes and the same number of links as the known pathway; (16) while networks having both the same number of nodes and the same number of links as the known pathway are randomly generated, the number of links does not have to be the same; (17) the network consistency determination unit 14 may randomly generate networks each having the same number of nodes as the known pathway and links whose number is different from that of links in the known pathway; (18) FIGS. 5A to 5C show examples of randomly generated multiple network; (19) the network consistency determination unit also uses the method described with FIG. 2B to determine the combined probability of each of these networks; and (20) specifically, each network is divided into subgraphs; the probability of an independence test is determined from the partial correlation coefficient of each subgraph; and those probabilities are combined to determine the combined probability value of the network.
Saito (US 2007 /0203870 A1, pub. date: 08/30/2007) discloses in ¶¶ [0005]-[0022] with FIGS. 1-2 that (1) one data mining technique is to discover the relationships between observed items by reconstructing independent directed acyclic graphs; (2) in FIG. 1, Xi(i=1-5) are nodes representing observed variables quantitatively indicating a state relating to an observed item; (3) in this technique, the presence of edges indicating the relationships between the nodes as well as the types of edges and the directions of arrows are specified by applying numerical techniques to the observed variables; (4) when there is an arrow going from node Xi to node Xj, the observed item relating to the observed variable Xi is the cause of the observed item relating to the observed variable Xj; (5) once an independent directed acyclic graph is obtained, it becomes possible to determine the strengths of the relationships between the observed variable; (6) FIG. 2 is a drawing in which partial regression coefficients P indicating the relational strengths are appended to the independent directed acyclic graph shown in FIG. 1; (7) by analyzing the above multiple regression equations using the least squares method, it is possible to estimate the partial regression coefficient β and the error e; (8) the PC algorithm is performed by following the below-given steps: (a) step 1: a completely undirected graph constructed by connecting, with undirected edges, all pairs of nodes among the nodes corresponding to the variables contained in the set of all variables V is taken as the initial state of the independent directed acyclic graph C; (b) step 2: in order to perform the graph reconstruction in steps, a variable n is established to indicate each step, and additionally, n is given an initial value of 0; (c1) step 3: as an ordered pair of adjacent (connected by an edge) nodes (Xi, Xj) in graph C, a pair of nodes 1s selected in which the number of elements in Ad(C, Xi)¥{Xj} is n or more;. (c2) additionally, a partial set S of Ad(C, Xi)¥{Xj} with n elements is selected; (c3) additionally, if the variable Xi and variable Xj. are conditionally independent when given a partial set S, the edge Eij connecting the node Xi and node Xj is deleted, and the elements of S are registered as the elements of the Sepset(Xi, Xj); (c4) this is performed with respect to all ordered pairs of nodes (Xi, Xj) for which the number of elements in Ad(C, Xi)¥{Xj} is n or more; (9) a method of determining whether a variable Xi and a variable Xj are conditionally independent when given a partial set S shall be described; (10) assumed that the variable vector (X1, X2, …, XP) follows a p-dimensional multivariate normal distribution; (11) a variance-covariance matrix shall be denoted Σ=(σij), and the inverse matrix will be denoted Σ-1=(σij); and (12) in this case, "σij =0" is equivalent to saying "the variable Xi and the variable Xj are conditionally independent when given a partial set consisting of the (p-2) variables other than the variable Xi and the variable Xj".
Togawa (US 2019/0087249 A1, pub. date: 03/21/2019) discloses in ABSTRACT and ¶¶ [0013]-[0027] that (1) predict occurrence of a predetermined failure in a certain apparatus among the apparatuses; (2) collect data for predicting the occurrence of the failure, from the apparatuses; (3) analyze the collected data and obtain an important-feature amount for making a predetermined standard prediction model adapt to the certain apparatus; (4) transmit the obtained important feature amount to the certain apparatus; (5) receive the important-feature amount to adjust the standard prediction model with the received important-feature amount; and (6) predict the occurrence of the failure with application of the data of the certain apparatus to the adjusted prediction model. Togawa further discloses in ¶¶ [0064]-[0072] with FIGS. 1 and 4-7 that (1) a sensor that detects the operation status of each part in the apparatus, sensors that detect temperature and humidity inside and outside the apparatus, and a function of acquiring information regarding date and time; (2) collect various types of data, such as measured values of sensors including the front end detecting sensor 42 and the paired skew sensors 43 in the host apparatus, temperature and humidity inside and outside the apparatus, the operation state of the apparatus, and an installed location, and repeatedly transmit the various types of data to the server 50; (3) the important-feature amount receiver 32 functions to receive the important-feature amount from the server 50; (4) the adjuster 33 functions to make an adjustment with the important-feature amount received from the server 50 such that the prediction model 38 fits the host apparatus; (5) the predictor 34 substitutes, e.g., the measured values measured by the front end detecting sensor 42 and the paired skew sensors 43 of the host apparatus, into the prediction model after the adjustment with the important feature amount, to execute computation; (6) then, the predictor 34 predicts whether the failure of the unexpected displacement is to occur; (7) in a case where a predicted result of the predictor 34 indicates that the unexpected displacement is to occur, the failure avoider 35 takes a measure for preventing the unexpected displacement from occurring; e.g., the failure avoider 35 executes automatic correction of shifting the transferring position of an image in the width direction of a sheet such that the transferring position corresponds to the displacement of the sheet; (8) the cause verifier 36 verifies the cause of the occurrence of the unexpected displacement; e.g., the cause verifier 36 verifies whether the cause is skew generated in picking up the sheet from the paper feed tray, skew due to the damage of the front end of the sheet caused in setting the sheet, skew caused during conveying, or slipping of a conveying roller due to paper powder; (9) the cause verifier 36 executes the verification, on the basis of the important-feature amount and the measured values of the front end detecting sensor 42 and the paired skew sensors 43; and (10) in a case where it is predicted that the unexpected displacement is to occur, the notifier 37 issues a notification with an alarm to the user of the image forming apparatus 10 or to the computer apparatus 70 in the sales company.
OKAWA et al. (US 2020/0311771 A1, pub. date: 10/01/2020) discloses in ¶¶ [0063]-[0067] with FIGS. 3-4 that (1) acquire, from the management terminal 400, information (hereinafter, also referred to as "set information of a prediction model") relating to an objective variable and an explanatory variable for generating a prediction model; (2) the set information of a prediction model is information designated by a manager or a user (hereinafter, also simply referred to as a "manager") of the contents control system; (3) the objective variable is, e.g., an index representing an advertising effect of a content; (4) herein, an index representing an advertising effect of a content is, e.g., a viewing amount and an audience rating; (5) the viewing amount may be a number of persons viewing a content while the output device 300 is outputting the content, or may be a total of stay time in an imaging range of a person viewing the content; (6) the audience rating may be a ratio of persons viewing a content among persons located in an imaging range while the output device 300 is outputting the content; (7) alternatively, the audience rating may be a total of stay time of a person viewing the content among totals of stay time of persons located in an imaging range; (8) the explanatory variable is set to, e.g., an attribute of a person, and environment information; (9) in the example of FIG. 4, the objective variable is set to {audience rating}, and the explanatory variable is set to each of items {place}, {time}, {day of the week}, {sex}, {age group}, and {content identification (ID)}. Herein, the content ID is information with which a content is identified; (10) the acquisition pattern generation unit 111 generates an acquisition pattern on the basis of the set information of the prediction model; and (11) the acquisition pattern is a pattern of data that a manager desires to acquire, and is one or more patterns designated by at least one value among values to which environment information or information relating to a person belongs; e.g., the acquisition pattern may be a data set generated on the basis of a combination of items designated as explanatory variables. OKAWA further discloses in ¶¶ [0116]-[0117] with FIGS.3 and 17 that (1) the prediction model generation unit 141 reads, from the acquisition pattern storage unit 112, the set information of the prediction model, i.e., the objective variable and the explanatory variable (S901); (2) moreover, the prediction model generation unit 141 generates a prediction model by use of the set information of the prediction model and the viewing data (S902); (3) when the objective variable and the explanatory variable illustrated in FIG. 4 are read, the prediction model generation unit 141 generates a prediction model in which {audience rating} is designated as the objective variable, and {place}, {time period}, {day of the week}, {sex}, {age group}, and {content identification (ID)} are the explanatory variables; 94); (4) the prediction model is represented, for example, as in Eqn. 1; (5) next, the prediction model generation unit 141 reads the viewing data from the viewing data storage unit 514 (S903); (6) then, the prediction model generation unit 141 learns the prediction model in Eqn. 1 by use of the read viewing data as training data, and determines a value of a parameter (S904); (7) In this instance, when using, as the training data, viewing data in which a sex is {female}, the prediction model generation unit 141 may learn by substituting "1" for {female}, and substituting "0" for {male}; (8) moreover, when a value of an item indicates a numerical value, the prediction model generation unit 141 may substitute the numerical value of the item for an explanatory variable to relevant to the item, and learn; (9) a learning method used herein is, but not limited to, e.g., a regression analysis, and various schemes of determining a value of a parameter are conceivable; and (10) the prediction model generation unit 141 stores, in the prediction model storage unit 142, a prediction model for which a value of a parameter is determined.
Liu et al. (US 2019 / 0303368 A1, pub. date: 10'0302019) discloses in ABSTRACT and ¶¶ 0006[]-[0008] that (1) obtaining a model representing causal relations among a plurality of variables based on a set of observation data of the plurality of variables; (2) determining, based on the obtained model, a first and a second variables having direct causal relation in the plurality of variables; (3) determining whether the first and second variables are independent from each other; (4) in response to the first and second variables being independent from each other, deleting the direct causal relation between the first and second variables from the obtained model; and (5) pseudo-causes can be removed effectively so that causal relations among a plurality of variables can be represented more accurately. Liu further discloses un ¶¶ [0019]-[0022] and [0025]-[0047] with FIGS. 1-4 that (1) understand internal relations existed in big data, e.g., the causal relations among a plurality of factors (also referred to as "variable" in the present application) is determined based on analysis to the big data, so as to provide relevant decisions for specific fields; (2) in a conventional scheme, causal relations among a plurality of variables are normally discovered with statistical independence- based methods and score-based methods; (3) the statistical independence-based methods employ independence test to determine whether there is causal relation between variables and the direction of the causal relations; (4) representative algorithms include PC (Perter-Clark) algorithm, stable PC, PCI (Fast Causal Inference) and the like; (5) however, the accuracy of causal relation being discovered is not ideal due to constraints of accuracy of independence test and transmissibility of determination error during the causal relation discovery process; (6) score-based method may measure the degree of fitness between the observation data and the causal relation network by designing decomposable scoring criteria, and guide a search for the optimal causal network, e.g., GES (Greedy Equivalence Search), with the scoring criteria; (7) however, since most existing scoring criteria mainly consider fitting degree of observed variables to target variables, a large amount of false causal relation is retained and therefore, the obtained accuracy of causal relation is not ideal either; (8) enabling a computer to discover innate causal relations among a plurality of variables more accurately; (9) preliminary causal relations are obtained based on a set of observation data of a plurality of variables using, for instance, score-based causal relation learning method; (10) then, the preliminary causal relations are optimized using independence check and/or conditional independence check to remove pseudo-causes in the preliminary causal relations thereby obtaining optimized causal relations; (11) since score-based learning method and independence check-based learning method are synthesized reasonably, and a large amount of pseudo-causes induced by score-based method are eliminated using the independence check, compared with causal relations determined with a conventional method, accuracy of the optimized causal relations finally obtained is higher, so that a more accurate understanding of the complex mechanism and process of actions behind the system can be obtained, potential relations between variables can be discovered and more effective decision can be provided to the users; (12) the environment 100 may include a data storage system 120 which is used for storing a set of observation data X of a plurality of variables, which may be represented as an N*D matrix, where N is the number of observed samples and D is the number of dimensions of observed variables or number of the observed variables; (13) the environment 100 may further include a model training system 110 which receives a set X of observation data of a plurality of observed variables from the data storage system 120; (14) the model training system 110 may obtain a preliminary model (also referred to as a preliminary causal relation model below) representing causal relations among a plurality of variables based on the set of observation data using an existing technology e.g., score-based Bayesian causal relation network or other causal relation discovery technology; e.g., the preliminary causal relation model may be generated through training based on the set of observation data; (15) the preliminary causal relation model 200 is represented as a directed acyclic graph in which the nodes represent a plurality of variables and a directed edge between two nodes denotes the existence of direct causal relation between the two nodes and the direction of the causal relation, for example, the source node is a direct cause of the target node; (16) since the preliminary causal relation model 200 obtained with the model training system 110 is generally not accurate enough, the environment 100 may further include a model optimization system 140 which receives the preliminary causal relation model 200 from the model training system 110 and determines variables having direct causal relations based on the causal relation model 200, e.g., variables 5 and 6, variables 6 and 27, variables 13 and 9 and so on, as shown in FIG. 2; (17) the model optimization system 140 may determine whether two variables having direct causal relations are independent or conditionally independent based on a statistical method (independence and conditional independence are collectively referred to as independence); (18) for two variables being determined as independent, the model optimization system 140 may delete the direct causal relation between the two variables from the preliminary causal relation model; (19) the model optimization system 140 may repeat the above process for each direct causal relation in the preliminary causal relation model, so as to obtain an optimized causal relation model and output it, e.g., storing in a model storage system 130 for subsequent use; e.g., when an automated decision-making is performed with a computer, the optimized model may be obtained from the model storage system 130 and data analysis may be performed based on this model to provide a proper decision .e.g., in the above product retail field, the optimized casualty model may formulate strategies automatically for a user, or assist the user in formulating strategies for improving sales volume of umbrella or ice cream or sunscreen cream; (20) FIG. 3 illustrates an optimized causal relation model 300 outputted by the model optimization system 140; (21) compared with the preliminary causal relation model 200 shown in FIG. 2, in the optimized causal relation model 300 shown in FIG. 3, since the variables 6 and 27 are independent, the edge between the variables 6 and 27 is deleted; (22) in probability and statistics, random variables X and Y being independent means that the occurrence of the variable Y will not influence the variable X, that is, the variable Y will not be the cause of the variable X, or the variable X will not be the effect of the variable Y, and vise verse; (23) therefore, if two variables having direct causal relation in the preliminary causal relation model 200 are independent from each other, the direct causal relation between the two variables in the model 200 is indeed statistically false (namely, pseudo-cause), and should be deleted; (24) the model optimization system 140 actually deletes pseudo-causes exactly based on this principle so that the causal relation represented by the optimized causal relation model 300 outputted by the model optimization system 140 is more accurate; (25) on the other hand, since the model optimization system 140 performs independence check for variables having direct causal relation based on the preliminary causal relation model 200 outputted by the model training system 110 rather than performing independence check for any two among the plurality of variables, it can save computational resources and improve computing speed while improving accuracy; (25) as shown in FIGS. 2 and 3, the edge between node 6 and node 27, the edge between node 8 and node 32, the edge between node 31 and node 30, and the edge between node 16 and node 20 and so on are deleted because of independence or conditional independence between nodes, thereby obtaining the optimized causal relation model as shown in FIG. 3 in which pseudo-causes are removed; (26) two variables being independent or conditionally independent denotes that the occurrence probabilities of the two variables do not influence each other, namely, the occurrence of one variable does not influence the occurrence of the other; i.e., if two variables are independent or conditionally independent, it demonstrates that one of the two variables cannot be the cause or effect of the other; and (27) thus, in the obtained preliminary causal relation model at block 402, if the first and second variables having direct causal relation are independent or conditionally independent, it means that the direct causal relation between these two variables is not true, namely, pseudo-cause. Therefore, deleting the pseudo-cause from the operations described in blocks 404, 406 and 408 are repeated so as to delete all the direct causal relation determined as pseudo-cause from the preliminary causal relation model, thereby obtaining a more accurate optimized causal relation model which is outputted for subsequent data analysis, e.g., automated or semi-automated decision-making. m the preliminary causal relation model will make the model more accurate. Liu further discloses in ¶¶ [0048]-[0063] with FIG. 5 that (1) At block 502, the model optimization system 140 may determine an association degree between the first and second variables based on a set of observation data; (2) the association degree may be used as a measurement of the relationship between the probability of co-occurrence of the first and second variables and the probability that two variables occur separately; (3) to determine the association degree, the model optimization system 140 may determine types of the first and second variables and select an independence decision method based on the determined types; (4) at block 504, the model optimization system 140 may determine whether the association degree between the first and second variables is within a first threshold range; (5) in response to the association degree being within the first threshold range, at block 512, the model optimization system 140 may determine that the first and second variables are independent; (6) to eliminate more pseudo-causes that are not helpful in actual application, when the first and second variables are not unconditionally independent, the model optimization system 140 may further determine whether the first and second variables are conditionally independent; (7) in response to the association degree exceeding the first threshold range, at block 506, the model optimization system 140 may determine a first set of related variables associated with the first variable and a second set of related variables associated with the second variable; (8) the model optimization system 140 may determine a set of parent nodes and spouse nodes of the first variable as the first set of related variables, and the model optimization system 140 may determine a set of the parent nodes and spouse nodes of the second variable as the second set of related variables; (9) at block 508, the model optimization system 140 may determine an association degree between the first and second variables with a union set of the first and second sets of related variables as a condition, which may be abbreviated as conditional association degree; (10) at block 510, the model optimization system 140 may determine whether the conditional association degree is within a second threshold range; (12) at block 512, in response to the conditional association degree being within the second threshold range, the model optimization system 140 may determine that the first and second variables are independent from each other; (13) at block 514, in response to the conditional association degree exceeding the second threshold range, the model optimization system 140 may determine that the first and second variables are not independent; (14) it is first determined at blocks 502 and 504 whether the first and second variables are statistically unconditionally independent; (15) in response to the first and second variables not being unconditionally independent, it is determined at blocks 506, 508 and 510 whether the two variables are statistically conditionally independent. In this manner, more pseudo-causes may be deleted from the causal relation model; and (16) On the other hand, only when the two variables are not unconditionally independent, it is further determined whether they are conditionally independent so as to save computation resources and time required for determining the set of related variables.
Ikeda et al. (US 2021/0035001 A1, 02/04/2021) discloses in ABSTRACT and ¶¶ [0014]-[0015] that (1) a causality estimation device includes: (a) an input unit configured to input data of a temporally sequential multidimensional numerical vector; (b) a regression model learning unit configured to learn a non-linear regression model with which data at a time is predicted from data at a past time by using the input data of the temporally sequential multidimensional numerical vector; (c) a causality estimation unit configured to calculate the strength of causality of a dimension i due to a dimension j in the data of the temporally sequential multi-dimensional numerical vector by using the non-linear regression model; and (d) an output unit configured to output the strength of the causality calculated by the causality estimation unit; and (2) enable estimation of a non-linear causality relation between dimensions by using temporally sequential multivariate data obtained from a system. Ikeda further discloses in ¶¶ [0025]-[0041] with FIGS. 1 and 3 that (1) the input unit 101 receives inputting of external information such as temporally sequential multi-dimensional numerical vector data and various parameters to the causality estimation device 100; (2) the storage unit 102 holds data, models, parameters, and the like input through the input unit 101; (3) the causality estimation unit 103 calculates the strength of causality between dimensions; (4) the regression model learning unit 104 learns a non-linear regression model; (5) the output unit 105 outputs the strength of causality between dimensions, which is calculated by the causality estimation unit 103; (6) a nonlinear regression model is estimated by using input temporally sequential multi-dimensional numerical vector data and the causality between dimensions is estimated by using an impulse response function of the model; (7) S101) a temporally sequential multi-dimensional numerical vector data set X collected from a system through the input unit 101 is input; (8) S102) the regression model learning unit 104 learns the non-linear regression model x_t=c+f(x_t-τ, x_t-τ+1, …, x_t-1)+ε_t (where c represents a constant term, f represents an optional non-linear function, and ε _t represents an error term at time t) by using the input X; (9) the model function z=f(y) may be an optional model such as a power model z=a*y^b or an exponential model z=a*b^y; (10) the learning method may be an optional method such as regression using a least-square method; (11) S103) the causality estimation unit 103 calculates an impulse response function of the non-linear regression model based on the learned model; (12) the causality estimation unit 103 calculates the strength of causality of the dimension i due to the dimension j based on the calculated impulse response function; and (13) S104) the causality estimation unit 103 calculates the strength of causality for all combinations of dimensions, and the output unit 105 outputs an N×N matrix in which an element on the i-th row and the j-th column represents the strength of causality of the dimension i due to the dimension when N represents the number of dimensions. Ikeda also discloses in ¶¶ [0052]-[0058] with FIGS. 3-4 that (1) the regression model learning unit 104 performs non-linear regression by using a neural network which has advantage of achieving various kinds of non-linear regression with simple modeling, and advantage of easily calculating the differential term by using the chain rule; (2) the number of intermediate layers and the number of dimensions in the neural network, an activation function, and learning parameters (such as a batch size and the number of learning epochs) may be determined and stored in the storage unit 102 in advance or may be provided and specified through the input unit 101; (3) once the differential is calculated by the back-propagation method in this manner, the impulse response function and the strength of causality of the dimension i due to the dimension j are calculated similarly to S103; (4) the strength of causality may be calculated only by using coefficients in place of the differential calculation; (5) the strength of causality of the dimension i due to the dimension j time p before may be calculated by summing the product of weights on a link connecting x_{t-p,j} of the input layer and x_{t,i} of the output layer over all paths; and (6) FIG. 4 illustrates an example in which, in a three-layer neural network including an input layer, an intermediate layer, and an output layer, the strength of causality of the dimension i=3 due to the dimension j=1 time p=1 before is calculated to be w^1_11*w^1_12*w^1_13 + w^2_13*w^2_23*w^2_33, which is obtained by summing the product of weights on a link connecting x_{t-1,1} of the input layer and x_ {t,3} of the output layer for all paths.
However, closest arts of records, as discussed above, singly or in combination do not teach or suggest at least following features "wherein for the plurality of input variables, the control unit estimates a causal graph regarding a nearest node using the nearest node included in a hidden layer closest to the prediction model as an objective variable, and selects the first explanatory variable as the reason from the input variables having a direct causal relationship with the objective variable" when combining with all other limitations of the claim as a whole.
Claim 12
Claim 12 is dependent on claim 11, and thus, similar to claim 11, contains allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HWEI-MIN LU whose telephone number is (313)446-4913. The examiner can normally be reached Mon - Fri: 9:00 AM - 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D. Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HWEI-MIN LU/Primary Examiner, Art Unit 2142
Prosecution Insights