DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
During a telephone conversation with Applicant’s attorney-of-record, Carl Pellegrini (Reg. No. 40,766) on Aug. 27, 2026, a provisional election was made without traverse to prosecute the invention of Group 1, claims 1-10 and 12. Affirmation of this election must be made by applicant in replying to this Office action. Claim 11 is withdrawn from further consideration by the examiner, 37 CFR 1.142(b), as being drawn to a non-elected invention.
Priority
Regarding Japanese Patent App. No. JP2023-091920 (filed 6/2/2023), receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement submitted on 5/30/2024 has been considered.
Claim Objections
Claims 1 and 12 are objected to because of the following informalities:
In claim 1, in the 2nd from the last line from the bottom, “the first set” should read “the predetermined first set” so that line 13 provides antecedent basis for the claim term.
In claim 12, in the 2nd from the last line from the bottom, “the first set” should read “the predetermined first set” so that line 11 provides antecedent basis for the claim term.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 and 12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Step 1 of the Alice/Mayo framework, Claims 1-10 are directed to a device (a machine), and Claim 12 is directed to a method (a process), which each fall within one of the four statutory categories of inventions.
Regarding Claim 1
Step 2A, prong 1 (Is the claim directed to a law of nature, a natural phenomenon or an abstract idea).
Claim 1 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components (e.g., “processor”, “memory”).
calculate a confidence score for each partial model of a target model, wherein the target model includes the partial model for each of a plurality of ways of performing the first class classification, wherein the partial model indicates, for each class in a class classification performed using a combination of the first class classification and the second class classification, a degree to which an element of a second set generated for each partial model from a predetermined first set is classified into the class, and wherein the confidence score indicates a degree to which the element of the second set is classified into a class in the first class classification, which is performed with respect to the explanatory variable value list included in the target data, and a class in the second classification, which is identified by the target variable value included in the target data (under the broadest reasonable interpretation, a human can mentally traverse a simple decision tree (such as the tree in Fig. 4 of the instant specification), and calculate a confidence score using the mental processes explained on page 30 of the instant specification, or using another mental process for mentally calculating a score that reflects the confidence in the accuracy of the simple decision tree for performing the first and second class classifications)
evaluate a possibility that the target data is included in the first set based on the confidence score of each partial model. (under the broadest reasonable interpretation, a human can mentally consider the confidence score for a partial model (such as a decision tree in a random forest), and mentally consider and evaluate the possibility that the target data is included in the first set based on the confidence score, such as a confidence score of 1.00 indicating a 100% probability that the target is included in the first set)
Step 2A, prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?).
The judicial exception is not integrated into a practical application.
Regarding the “A risk evaluation device comprising: at least one memory configured to store instructions; and at least one processor configured to execute the instructions to” limitation, such limitations are recited at a high-level of generality and amount to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional elements of generic devices, processors, and memories. These additional elements are recited at a high-level of generality and amount to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Regarding the “acquire target data including an explanatory variable value list and a target variable value, wherein the explanatory variable value list is a list of values of classification items representing items used in a first class classification, and the target variable value is a value that identifies a class in a second class classification” limitation, such additional element of a data gathering step is recited at a high level of generality and amounts to extra-solution activity of receiving data, i.e. pre-solution activity of gathering data for use in the claimed process (see MPEP 2106.05(g)).
Accordingly, at Step 2A, prong two, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not integrate the judicial exception into a practical application.
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?)
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
Regarding the “A risk evaluation device comprising: at least one memory configured to store instructions; and at least one processor configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding the “acquire target data including an explanatory variable value list and a target variable value, wherein the explanatory variable value list is a list of values of classification items representing items used in a first class classification, and the target variable value is a value that identifies a class in a second class classification” limitation, as discussed above, the additional element of a data gathering step is recited at a high level of generality and amounts to extra-solution activity of receiving data, i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B after considering all claim elements individually and as an ordered combination, it is determined that the claims do not integrate the judicial exception into a practical application.
Regarding Claim 2
Step 2A, Prong 1
wherein the partial model is a decision tree representing the first class classification by branching. (under the broadest reasonable interpretation, a human can mentally perform a partial model in the form of a simple decision tree, such as the simple decision tree in Fig. 4 of the instant specification)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 3
Step 2A, Prong 1
wherein the partial model represents for each class in a class classification performed using a combination of the first class classification and the second class classification, a number of elements among elements of the second set that are classified into said class, (under the broadest reasonable interpretation, a human can mentally use a decision tree model that performs first and second class classifications, because as explained on p. 15, paragraph 3 of the instant specification, the recited “first class classification” follows the branching from root to leaf as shown in Figs. 6-7 of YAMAGAMI, and the recited “second class classification” corresponds to the actual classifications at the leaf nodes (C1-C8) in Figs. 6-7 of YAMAGAMI, where the process of using the decision tree of Fig. 4 to create the table of Fig. 6 is a mental process of traversing the decision tree)
calculate the confidence score, which indicates, for a single class in the first class classification, a ratio of a number of elements among elements of the second set that are classified into each class of the second class classification. (under the broadest reasonable interpretation, (under the broadest reasonable interpretation, a human can mentally traverse a simple decision tree (such as the tree in Fig. 4 of the instant specification), and calculate a confidence score using the mental processes explained on page 30 (including the teachings with respect to ratios) of the instant specification, or using another mental process for mentally calculating a score that reflects the confidence in the accuracy of the simple decision tree for performing the first and second class classifications using ratios)
Step 2A, Prong 2
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a processor. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a processor). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 4
Step 2A, Prong 1
generate target data subjected to calculation of the confidence score by setting, to target data in which values of one or more classification items are unknown, candidate values of a classification item with an unknown value. (under the broadest reasonable interpretation, a human can mentally (or using pencil and paper) generate target data used to calculate the confidence score by setting certain values as an unknown value as set forth in this claim)
Step 2A, Prong 2
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a processor. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a processor). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 5
Step 2A, Prong 1
calculate, for each candidate value included in a list of candidate values of classification items with an unknown value, a non-applicability score that indicates, for target data in which the candidate value has been set, a number of partial models indicating that there are no elements among elements of the second set that are classified into a class that has been classified in the first class classification, which is performed with respect to the explanatory variable value list included in the target data, and a class of the second class classification, which is identified by the target variable value included in the target data. (under the broadest reasonable interpretation, a human can mentally calculate such a non-applicability score that indicates the number of partial models (decision trees) that indicate that there are no elements among elements of the second set that are classified into a class that has been classified in the first class classification as set forth herein)
Step 2A, Prong 2
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a processor. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a processor). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 6
Step 2A, Prong 1
set, among candidate values included in a list of candidate values of a classification item with an unknown value, a candidate value having a lowest non-applicability score, as an estimated value of the classification item. (under the broadest reasonable interpretation, a human can mentally set candidate values that are an unknown value, with the lowest non-applicability score as set forth herein)
Step 2A, Prong 2
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a processor. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a processor). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 7
Step 2A, Prong 1
set, among candidate values included in a list of candidate values of a classification item with an unknown value, an estimated value of the classification item as undetermined in a case where there are a plurality of candidate values having a lowest non-applicability score. (under the broadest reasonable interpretation, a human can mentally set candidate values that are an unknown value, with an estimated value as “undetermined” as set forth herein)
Step 2A, Prong 2
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a processor. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a processor). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 8
Step 2A, Prong 1
set, in a case where a size of a difference between a lowest value of the non-applicability scores and a next lowest value after a lowest value is smaller than a predetermined threshold, an estimated value of the classification item as undetermined. (under the broadest reasonable interpretation, a human can mentally set an estimated value as undetermined using the predetermined threshold as set forth herein)
Step 2A, Prong 2
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a processor. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a processor). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 9
Step 2A, Prong 1
set, in a case where a lowest value of the non-applicability scores is larger than a predetermined threshold, an estimated value of the classification item as undetermined. (under the broadest reasonable interpretation, a human can mentally set an estimated value of the classification item as undetermined as set forth herein)
Step 2A, Prong 2
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a processor. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a processor). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 10
Step 2A, Prong 1
generate a list of pairs including, among the plurality of target data in which the values of one or more classification item are unknown, target data in which the estimated values of the classification items with unknown values have been determined, and the estimated values. (under the broadest reasonable interpretation, a human can mentally (or using pencil and paper) generate a list of pairs including target data and estimated values, as set forth herein)
Step 2A, Prong 2
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a processor. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a processor). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the at least one processor is configured to execute the instructions to” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 12
Step 2A, Prong 1
Claim 12 recites a method that corresponds to the device of claim 1, and therefore the analysis under Step 2A, Prong 1 with respect to claim 1 also applies to this claim 12.
Step 2A, Prong 2
Claim 12 recites a method that corresponds to the device of claim 1, and therefore the analysis under Step 2A, Prong 2 with respect to claim 1 also applies to this claim 12. While claim 12 recites additional generic computing components (“computer”), such additional generic computing component does not change the analysis under Step 2A, Prong 2. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Claim 12 recites a method that corresponds to the device of claim 1, and therefore the analysis under Step 2B with respect to claim 1 also applies to this claim 12. While claim 12 recites additional generic computing components (“computer”), such additional generic computing component does not change the analysis under Step 2B because the limitations merely provide instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over US 20180005126 A1, hereinafter referenced as YAMAGAMI, in view of US 9942264 B1, hereinafter referenced as KENNEDY.
Regarding Claim 1
YAMAGAMI teaches:
A risk evaluation device comprising: (YAMAGAMI, para. 0079: “The decision tree generating apparatus of the embodiment is not limited to the above-described configuration.”;
Examiner’s Note: the “risk evaluation” portion of the preamble is non-limiting intended use, where the body of the claim fully and intrinsically sets forth all of the limitations of the claimed invention as set forth in MPEP 2111.02)
at least one memory configured to store instructions; and (YAMAGAMI, para. 0007: “It should be noted that general or specific embodiments may be implemented as a system, a method, an integrated circuit, a computer program, a storage medium, or any selective combination thereof.”;
YAMAGAMI, para. 0057: “The inquiry system includes a memory that stores a decision tree that is generated by the decision tree generating apparatus, an inquirer that outputs an inquiry in accordance with the decision tree stored on the memory, an acquirer that acquires the user's answer responsive to the inquiry from the inquirer, and a generator that generates the classification results responsive to the user's answer acquired by the acquirer.”)
at least one processor configured to execute the instructions to: (YAMAGAMI, para. 0007: “It should be noted that general or specific embodiments may be implemented as a system, a method, an integrated circuit, a computer program, a storage medium, or any selective combination thereof.”;
YAMAGAMI, para. 0115: “The user's answer result processor 504, the current decision tree node memory 506, and the medical interview controller 507 generate classification results responsive to the acquired answers, and the display 511 displays an explanation of the classification results.”)
acquire target data including an explanatory variable value list and a target variable value, wherein the explanatory variable value list is a list of values of classification items representing items used in a first class classification, and the target variable value is a value that identifies a class in a second class classification; (YAMAGAMI, para. 0069: “FIG. 2 illustrates contents of the classification target data set. Referring to FIG. 2, each piece of the classification target data is assigned attribute values (1 or 0) corresponding to four attributes x1, x2, x3, and x4. Each piece of the classification target data has one of categories (C1 through C8) in which each piece of the classification target data is classified. The classification target data having such a data structure is stored on the classification target data memory 14.”;
YAMAGAMI, para. 0101: “FIG. 6 illustrates an example of the decision tree that is generated by calculating the information gain from the classification target data of FIG. 2 using the reliability of the user's answer of FIG. 3 in accordance with the process described above. FIG. 7 illustrates an example of the decision tree by calculating the information gain using the amount of reduction of entropy rather than using the reliability of the user's answer. FIG. 6 and FIG. 7 illustrate the nodes, edges, categories, attributes, and attribute values of the decision tree using identical symbols.”
Examiner’s Note: as shown in Fig. 2, for each data ID, the list of attributes (x1 – x4) corresponds to the recited “explanatory variable value list” and the category corresponds to the recited “target variable value”; as explained on p. 15, paragraph 3 of the instant specification, the recited “first class classification” follows the branching from root to leaf as shown in Figs. 6-7 of YAMAGAMI, and the recited “second class classification” corresponds to the actual classifications at the leaf nodes (C1-C8) in Figs. 6-7 of YAMAGAMI)
wherein the partial model indicates, for each class in a class classification performed using a combination of the first class classification and the second class classification, a degree to which an element of a second set generated for each partial model from a predetermined first set is classified into the class, which is performed with respect to the explanatory variable value list included in the target data, and a class in the second classification, which is identified by the target variable value included in the target data (YAMAGAMI, para. 0031: “FIG. 2 illustrates a classification target data set containing eight pieces of data. Each piece of the data has four attributes x1, x2, x3, and x4 with each attribute having 0 or 1 assigned thereto as an attribute value. The decision tree is generated in accordance with the information gain as illustrated in FIG. 7.”
YAMAGAMI, para. 0043: “When a classification target data set including a plurality of pieces of classification target data respectively having mutually different attributes with attribute values assigned thereto is hierarchically segmented into a plurality of subsets in a form of a decision tree, the information gain calculator calculates an amount of reduction in entropy of the pre-segmentation classification target data set caused by segmentation on each attribute of each piece of the classification target data included in a pre-segmentation data set, and calculates an information gain when the pre-segmentation data set is segmented in accordance with the attribute value of each attribute, based on the amount of reduction in the entropy and reliability that is an index representing correctness or incorrectness of a user's answer responsive to an inquiry asking about the attribute.”
YAMAGAMI, para. 0075: “FIG. 3 illustrates an example of reliability of a user's answer of each attribute calculated by the user's answer reliability calculator 13. In the example of FIG. 3, the reliability on the attribute x1, namely, the correct answer rate to an inquiry asking about the reliability on the attribute x1 is 60%. The reliabilities of the attributes x2, x3, and x4 are respectively 80%, 90%, and 70%. The correct answer rate of the attribute x3 is the highest, and the correct answer rate of the attribute x1 is the lowest. The reliabilities of the answers to the inquiries about the attributes calculated by the user's answer reliability calculator 13 are used as inputs to the information gain calculator 12 that calculates the information gain.”
Examiner’s Note: each decision tree (corresponding to a partial model of a random forest) indicates how the attributes are used to do the classification (corresponding to the recited “first and second class classifications”), and the “degree to which an element of a second set generated for each partial model from a predetermined first set is classified into the class” is reflective of the percentages shown in Fig. 3, which are taken into account when calculating information gain to determine the ordering of the nodes and leaves in the decision tree, where the classification target data set corresponds to the recited “first set” and the subsets formed are the recited “second set”)
However, YAMAGAMI fails to explicitly teach:
calculate a confidence score for each partial model of a target model, wherein the target model includes the partial model for each of a plurality of ways of performing the first class classification,
wherein the confidence score indicates a degree to which the element of the second set is classified into a class in the first class classification,
evaluate a possibility that the target data is included in the first set based on the confidence score of each partial model.
However, in a related field of endeavor (random forest models using decision trees, see col. 4, lines 47-52), KENNEDY teaches and makes obvious:
calculate a confidence score for each partial model of a target model, wherein the target model includes the partial model for each of a plurality of ways of performing the first class classification, (KENNEDY, col. 4, lines 47-52: “The terms “forest,” “random forest,” “classification forest,” and “forest model” as used herein generally refer to a heuristic ensemble learning technique for classification and/or sorting of file data. A forest may include a plurality of decision trees that are each trained independently using a common set of data.”;
KENNEDY, col. 9, lines 15-21: “Forest model 402 may include any suitable number of trees, without limitation. For example, forest model 402 may include N trees, represented as trees 410-1, 410-2, and 410-N. Each tree may include any suitable number of leaf nodes, without limitation. For example, each of trees 410-1 through 410-N illustrated in FIG. 4 may include M trees, represented as leaf nodes 412-1, 412-2, and 412-M”;
KENNEDY, col. 9, lines 34-49: “According to at least one embodiment, each leaf node 412-1 through 412-M may include a particular confidence score 414 indicative of a probability that the data items in the respective leaf node 412-1 through 412-M belong to the particular class indicated by the leaf node. For example, if a leaf node of leaf nodes 412-1 through 412-M with a confidence score of 80% indicates that a file includes malware, the probability that the file actually includes malware may be 80%. Confidence scores 414 may be determined using previously categorized data. For example, general use forest model 206 may include confidence scores that are determined through building and training the trees of general use forest model 206 using categorized data, such as categorized file data, from a sample of files taken from a plurality of computing systems, including computing systems outside of organization network 220.”
KENNEDY, col. 10, lines 55-60: “In some examples, conviction threshold 420 may be associated with confidence scores 414 of trees 410-1 through 410-N of forest 402 that are determined by running a set of categorized data down decision trees 410-1 through 410-N.”;
Examiner’s Note: KENNEDY teaches that each decision tree leaf in a random first model has its own calculated confidence score (and therefore since every decision tree has at least one leaf, at least one confidence score is done for each tree), and that the target forest model has a plurality of decision trees; the YANAGAMI-KENNEDY combination now combines the decision trees of YANAGAMI into a random forest model as in KENNEDY, where each decision tree leaf has a calculated confidence score as in KENNEDY)
wherein the confidence score indicates a degree to which the element of the second set is classified into a class in the first class classification, (KENNEDY, col. 9, lines 34-49: “According to at least one embodiment, each leaf node 412-1 through 412-M may include a particular confidence score 414 indicative of a probability that the data items in the respective leaf node 412-1 through 412-M belong to the particular class indicated by the leaf node. For example, if a leaf node of leaf nodes 412-1 through 412-M with a confidence score of 80% indicates that a file includes malware, the probability that the file actually includes malware may be 80%. Confidence scores 414 may be determined using previously categorized data. For example, general use forest model 206 may include confidence scores that are determined through building and training the trees of general use forest model 206 using categorized data, such as categorized file data, from a sample of files taken from a plurality of computing systems, including computing systems outside of organization network 220.”;
Examiner’s Note: the YANAGAMI-KENNEDY combination now tailors the leaf-confidence scores of KENNEDY to be used in YANAGAMI, where the confidence score relates to a classification probability as disclosed by KENNEDY that the category of YANAGAMI will end up at a particular leaf in the decision tree)
evaluate a possibility that the target data is included in the first set based on the confidence score of each partial model. (KENNEDY, col. 9, lines 34-49: “According to at least one embodiment, each leaf node 412-1 through 412-M may include a particular confidence score 414 indicative of a probability that the data items in the respective leaf node 412-1 through 412-M belong to the particular class indicated by the leaf node. For example, if a leaf node of leaf nodes 412-1 through 412-M with a confidence score of 80% indicates that a file includes malware, the probability that the file actually includes malware may be 80%. Confidence scores 414 may be determined using previously categorized data. For example, general use forest model 206 may include confidence scores that are determined through building and training the trees of general use forest model 206 using categorized data, such as categorized file data, from a sample of files taken from a plurality of computing systems, including computing systems outside of organization network 220.”;
KENNEDY, col. 10, lines 47-67: “In at least one embodiment, forest 402 may include a specified conviction threshold 420. A conviction threshold 420 may be calculated and assigned to forest 402 of general use forest model 206 and/or organization-specific forest model 208 as an indicator of whether or not data that is run through forest 402 includes malware. In one embodiment, obtaining a classification result satisfying conviction threshold 420 may indicate whether an unknown file is more likely to include malware or to be free from malware. In some examples, conviction threshold 420 may be associated with confidence scores 414 of trees 410-1 through 410-N of forest 402 that are determined by running a set of categorized data down decision trees 410-1 through 410-N. In one embodiment, conviction threshold 420 for forest 402 may be based at least in part on an aggregate of confidence scores 414 for leaf nodes 412-1 through 412-M of decision trees 410-1 through 410-N. For example, conviction threshold 420 may be based at least in part on a summation and/or an average calculation of confidence scores 414 derived from running categorized data through decision trees 410-1 through 410-N.”
Examiner’s Note: the YANAGAMI-KENNEDY combination now uses the leaf-confidence scores of KENNEDY to be used in YANAGAMI, such that a classification (C1-C8) can be determined to be within the classification target data set (corresponding to the recited “first set”) where the confidence score for the leaf sets forth the probability that the particular category (c1-c8) is within the classification target data set).
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of YANAGAMI and KENNEDY as explained above. As disclosed by KENNEDY, one of ordinary skill would have been motivated to do so in order to use a combination of confidence scores for decision trees to determine a “conviction threshold” for indicating whether data has a particular category, such as (is malware vs. not malware). (col. 10, lines 48-52). One of ordinary skill would further be motivated to do so in order to generate organization-specific decision trees and forests. (col. 1, lines 42-46).
Regarding Claim 2
YANAGAMI and KENNEDY teach the device of claim 1. YANAGAMI further teaches:
wherein the partial model is a decision tree representing the first class classification by branching. (YAMAGAMI, para. 0031: “FIG. 2 illustrates a classification target data set containing eight pieces of data. Each piece of the data has four attributes x1, x2, x3, and x4 with each attribute having 0 or 1 assigned thereto as an attribute value. The decision tree is generated in accordance with the information gain as illustrated in FIG. 7.”)
Claim 12 recites a method that corresponds to the device of claim 1 and is rejected for the same reasons explained above with respect to claim 1.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over YANAGAMI in view of KENNEDY and further in view of US 11308212 B1, hereinafter referenced as ZHU.
Regarding Claim 4
YANAGAMI and KENNEDY teach the device of claim 1. However, YANAGAMI and KENNEDY fail to explicitly teach:
generate target data subjected to calculation of the confidence score by setting, to target data in which values of one or more classification items are unknown, candidate values of a classification item with an unknown value.
However, in a related field of endeavor (data classification, see col. 1, lines 7-10), ZHU teaches and makes obvious:
generate target data subjected to calculation of the confidence score by setting, to target data in which values of one or more classification items are unknown, candidate values of a classification item with an unknown value. (ZHU, col. 11, lines 32-38: “By blacklisting or whitelisting a directory/sub-directory, the files with unknown reputation under the directory/sub-directory may be also classified correspondingly, and thus a reputation score may be automatically assigned to a file with unknown reputation based on a default value set for each classification. In this way, the reputation score for a file with unknown reputation may be determined.”;
Examiner’s Note: the YANAGAMI-KENNEDY-ZHU combination now assigns a default value to any classification items that are unknown as in ZHU)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of YANAGAMI with KENNEDY and ZHU as explained above. As disclosed by ZHU, one of ordinary skill would have been motivated to do so in order to assign a default value to an known value in a manner that provides “more useful, meaningful information to a user, compared to just a placeholder value.” (col. 9, lines 38-45).
Allowable Subject Matter
Claims 3 and 5-10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, provided that the rejections under 35 U.S.C. 101 are overcome.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding dependent claim 3, if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and provided that the rejections under 35 U.S.C. 101 are overcome, such claim would be considered allowable because none of the references of record either alone or in combination fairly disclose or suggest the combination of limitations specified in the claim, including at least:
wherein the partial model represents for each class in a class classification performed using a combination of the first class classification and the second class classification, a number of elements among elements of the second set that are classified into said class, and
wherein the at least one processor is configured to execute the instructions to calculate the confidence score, which indicates, for a single class in the first class classification, a ratio of a number of elements among elements of the second set that are classified into each class of the second class classification.
The closest prior art of record discloses:
US 20180005126 A1, hereinafter referenced as YAMAGAMI teaches a classification target data set, whereas shown in Fig. 2, for each data ID, there is a list of 4 attributes and a final category, where in each case a decision tree is determined to represent the attributes and category. (paras. 0031, 0043, 0069).
US 9942264 B1, hereinafter referenced as KENNEDY discloses calculating confidence scores for leaf nodes in decision trees in a random forest. (col. 4, lines 47-52 and col. 9, lines 15-21 and col. 10, lines 55-60).
US 20180300668 A1, hereinafter referenced as FARLEY teaches “In particular, the probability formula calculates a number of times a collection of particular words and phrases have been identified as representing a particular item's class and category, which is later used to calculate a confidence factor. In accordance with an example embodiment of the present invention, the determined class and category for an item are associated with a confidence factor, which is used to set a bar for acceptance of the algorithm's decision or prompting the review team to audit the decision and approve or adjust the decision.” (para. 0034).
However, the examiner has found that the distinct feature of the Applicant's claimed invention over the prior art is the explicit claiming of the aforementioned limitations in combination with all the other limitations as specified in dependent claim 3. Further, the examiner finds that one of ordinary skill would not have been motivated to modify the prior art of record in the manner specifically recited by claim 3 without the hindsight aid of Applicant’s disclosure. Therefore, because the prior art of record does not anticipate nor make obvious the limitations recited in claim 3, claim 3 would be allowed over the prior art if rewritten in independent form including all of the limitations of the base claim and any intervening claims, provided that the rejections under 35 U.S.C. 101 are overcome.
Regarding dependent claim 5, if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and provided that the rejections under 35 U.S.C. 101 are overcome, such claim would be considered allowable because none of the references of record either alone or in combination fairly disclose or suggest the combination of limitations specified in the claim, including at least:
calculate, for each candidate value included in a list of candidate values of classification items with an unknown value, a non-applicability score that indicates, for target data in which the candidate value has been set, a number of partial models indicating that there are no elements among elements of the second set that are classified into a class that has been classified in the first class classification, which is performed with respect to the explanatory variable value list included in the target data, and a class of the second class classification, which is identified by the target variable value included in the target data.
The closest prior art of record discloses:
US 20180005126 A1, hereinafter referenced as YAMAGAMI teaches a classification target data set, whereas shown in Fig. 2, for each data ID, there is a list of 4 attributes and a final category, where in each case a decision tree is determined to represent the attributes and category. (paras. 0031, 0043, 0069).
US 9942264 B1, hereinafter referenced as KENNEDY discloses calculating confidence scores for leaf nodes in decision trees in a random forest. (col. 4, lines 47-52 and col. 9, lines 15-21 and col. 10, lines 55-60).
US 20140279299 A1, hereinafter referenced as ERINRICH, teaches determining a “confidence score between zero and one equal to the percent of decision trees in the random forest algorithm used by the classifier 255 that determine that both merchant ID sets 210 in the pair should be linked to the same company.” (para. 0068).
However, the examiner has found that the distinct feature of the Applicant's claimed invention over the prior art is the explicit claiming of the aforementioned limitations in combination with all the other limitations as specified in dependent claim 5. Further, the examiner finds that one of ordinary skill would not have been motivated to modify the prior art of record in the manner specifically recited by claim 5 without the hindsight aid of Applicant’s disclosure. Therefore, because the prior art of record does not anticipate nor make obvious the limitations recited in claim 5, claim 5 would be allowed over the prior art if rewritten in independent form including all of the limitations of the base claim and any intervening claims, provided that the rejections under 35 U.S.C. 101 are overcome.
Claims 6-10 depend from claim 5, and would each be allowed for the same reasons explained with respect to claim 5, provided that the rejections under 35 U.S.C. 101 are overcome.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20210350930 A1 (Baron). “Each decision tree in a random forest model can be generated in a training process over patient data of a set of data categories, as well as survival statistics, of a large population of patients. In addition, the training process can determine the subsets of data categories assigned to each decision tree, the classification criteria at each parent node of the decision trees, as well as the value of the cumulative hazard function at each terminal node.” (para. 0048).
US 12039414 B1 (Srivastava). “When each analysis sub-model is trained, their respective probability score data 136 will agree for the labeled data items 130. The respective probability score data 136 will not agree for first unlabeled data items due to the different sets of weights assigned to the weighted values in each of the functions. Accordingly, the combined probability score data will result in a spread of distributions in which no class has a substantially higher probability than all of the other classes. Thus, if all of the analysis sub-models are in agreement about a given data item, then there is a high degree of confidence in the classification of the data item. If the analysis sub-models do not agree about the classification of a data item, then there is low confidence in in the classification.” (col. 13, lines 17-30).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL C LEE whose telephone number is (571)272-4933. The examiner can normally be reached M-F 12:00 pm - 8:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL C. LEE/Examiner, Art Unit 2128
Prosecution Insights