DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The following FINAL Office Action is in response to Applicant’s communication filed 06/22/2026 regarding Application 18/843,639.
Status of Claim(s)
Claim(s) 1-14 is/are currently pending and are rejected as follows
Response to Arguments – 103 Rejection
Applicant’s arguments in regard to the previously applied 103 rejection are rendered moot in view of the amended prior art rejection below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morimura (US 2012/0072259 A1) in view of Yellapragada (US 2020/0387814 A1) and Takigawa (US 2018/0247256 A1)
Claim(s) 1, 7, and 11 –
Morimura discloses the following:
A memory (Morimura: Paragraph 114, “Any combination of one or more computer readable medium(s) can be utilized. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium can include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.”)
A processor configured to function as a functional unit by executing a program loaded in the memory (Morimura: Paragraph 115, “Computer program code for carrying out operations for aspects of the present invention can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer. Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.”)
wherein an input unit, a risk evaluation unit, and a display unit are provided as the functional unit (Morimura: Paragraph 37, “FIG. 1 illustrates an overall configuration of information system 100 according to an embodiment of the invention. Information system 100 includes user computer 110, optimal action decision system 120, transition probability storing unit 130, and reward parameter storing unit 140. User computer 110 according to the embodiment of the invention sends a request to optimal action decision system 120 and receives information on optimal actions returned.”; Paragraph 93, “The information processing device accepts input from an input device, such as keyboard 6 and a mouse 7, via a keyboard/mouse controller 5. The information processing device is connected with a display device 11 for presenting visual data to the user by way of a DAC/LCDC 10”)
Wherein the input unit receives an input of data of a plurality of operation process candidates from a user and stores the data in a data storage unit, and the data of each of the operation process candidates includes a step flow…wherein an influence evaluation table in which influence of a conclusion of an operation process, which is a content of a final step of the operation process candidate, on a related person and an influence evaluation value are registered, and wherein a transition probability table in which a transition probability of a branch included in the step flow is registered, (Morimura: Paragraph 31, “Information system 100 according to embodiments of the invention is provided in advance with data on state transition probability, parameters associated with reward for an action that is executed in a certain state, and/or etc. In accordance with a request from a user, optimal actions are determined for respective terms (T.sub.1, T.sub.2, . . . T.sub.n) of a time horizon, where the time horizon starts from a later term and proceeds toward an earlier term. Optimal actions determined are finally presented to the requesting user as information for making a decision.”; Paragraph 32, “Embodiments of the invention calculate a value measure that considers risk in conformance with predetermined preferences for each action candidate that can be executed in each state that is possible within a term, and determine an action candidate having the best value measure to be the optimal action. An optimal action determined for a state is associated with the state and stored along with a value measure (hereinafter called "optimal value" and sometimes denoted as "v*(s, t)") corresponding to the optimal action. The stored optimal value is used for determining optimal actions for the immediately preceding term.”; Paragraph 45, “Specifically, transition probability storing unit 130 according to embodiments of the invention stores a transition probability for each combination of a pair of a transition source state and an action (s, a), and a target state, s. For example, the probability that no state transition occurs and state remains S.sub.1 as a result of executing action a.sub.1 in state S.sub.1 is 25%, and the probability that transition to state s.sub.2 occurs is 40%. Note that the total transition probability in each row of the table shown in FIG. 2 (e.g., 0.25+0.40+ . . . +0.30 in the row (s.sub.1, a.sub.1)) is 1.”; Paragraph 50, “Optimal action decision system 120 includes user interface unit 405, state transition pattern generating unit 410, state transition pattern storing unit 415, probability distribution calculating unit 420, risk measure calculating unit 425, weighting function decision unit 430, value measure calculating unit 435, an optimal action decision unit 440, and optimal action/value measure storing unit 445.”; Paragraph 51, “User interface unit 405 according to embodiments of the invention functions as an interface for exchanging information data with user computer 110. Specifically, user interface unit 405 has state transition pattern generating unit 410 generate state transition patterns upon receiving a request for information about optimal actions from user computer 110, and triggers a sequence of processing for determining optimal actions for the state transition patterns. User interface unit 405 also has a function to send information about optimal actions determined back to user computer 110.”; Paragraph 56, “State transition pattern generating unit 410 according to embodiments of the invention generates such state transition patterns in the following procedure. First, the user sets the last term (T.sub.n) as the current term and generates states that are specified as possible states for the current term. Then, assuming that the second last term (T.sub.n-1) is the current term, possible states for the current term are generated, and each of the states is combined with action candidates that can be executed in that state and the transition probability of transitioning to each target state if each action candidate is executed. Then, on the condition that the optimal action and optimal value have been determined for all possible states in the current term T.sub.n-1, the third last term (T.sub.n-2) is assumed to be the current term, states possible in the current term are generated, and each of the states is combined with action candidates that can be executed in that state and transition probability of transitioning to each target state if each action candidate is executed. Embodiments of the invention repeat this process up to the first term (T.sub.1). Generation of state transition patterns will be described in greater detail later.”; Paragraph 62, “Optimal action decision unit 440 according to embodiments of the invention compares value measures for action candidates that can be executed in a state possible at a certain time step calculated by value measure calculating unit 435, and determines an optimal action for that state. Upon having determined optimal actions for all possible states for the current state, optimal action decision unit 440 assumes the term immediately before the current term is now the current term and sends an instruction to state transition pattern generating unit 410 to generate state transition patterns that are possible in the new current term.”; Paragraph 68, “Then, at step 545, probability distribution of the evaluation value X is determined according to the expression shown below for the action `a` selected at step 540. In the expression, r represents reward, and v*(s', t+1) is the optimal value relating to state s' of the next term (the term immediately following t, here T.sub.n). p.sub.t(s', r|s, a) is defined as the probability that transition to state s' occurs and reward r is obtained if action `a` is executed in state s in the current term (t=T.sub.n-1). A sum represented by .SIGMA. is determined by adding up p.sub.t(s', r|s, a) for all pairs of s' and r that make r+v*(s',t+1)=x. In short, X represents such a discrete distribution that value x is assumed with a certain probability and, in this embodiment, corresponds to the sum of reward obtained in the current term and the optimal value for the next term.”)
Wherein the risk evaluation unit calculates, for each route leading to a conclusion that is to be taken by each of the operation process candidates, a risk score based on an occurrence probability of the route…and calculates, as a risk score of the operation process candidate, a sum of the risk scores calculated for a plurality of the routes that is to be taken for each respective operation process candidate, and (Morimura: Paragraph 45, “Specifically, transition probability storing unit 130 according to embodiments of the invention stores a transition probability for each combination of a pair of a transition source state and an action (s, a), and a target state, s. For example, the probability that no state transition occurs and state remains S.sub.1 as a result of executing action a.sub.1 in state S.sub.1 is 25%, and the probability that transition to state s.sub.2 occurs is 40%. Note that the total transition probability in each row of the table shown in FIG. 2 (e.g., 0.25+0.40+ . . . +0.30 in the row (s.sub.1, a.sub.1)) is 1.”; Paragraph 47, “More specifically, reward parameter storing unit 140 according to embodiments of the invention stores, for each combination of a pair of a source state and an action (s, a) and a target state s, a mean value of reward that is obtained as a result of transition from the source to the target state. For example, a mean reward that is obtained when transition to state s.sub.2 occurs as a result of executing action a.sub.1 in state s.sub.1 is $4.5 and the mean reward obtained when transition to state s.sub.2 occurs is $2.1.”; Paragraph 70, “At step 555, a weighting function w(x) that depends on the risk measure V.sub..alpha. determined at step 550 is performed. Various types of function can be employed as the weighting function w(x) according to preferences about risk. For example, for a preference to balance maximization of cumulative reward that will be finally obtained and minimization of risk, the weighting function w(x) shown below can be established, where Pr(X=x) represents the probability that random variable X assumes the value of instance, x. In this case, the extent of risk taking can be adjusted by varying the value of b in the expression.”; Paragraph 76, “At step 560, a value measure (VM(s, a, t)) is determined by determining the weighted sum of weighting function w(x) calculated at step 555 according to the expression below. Note that the expression needs to be modified as appropriate when loss Y is used instead of evaluation value X and a weighting function is established as w(y) for a preference to perform optimization using ICTE as described above. As those skilled in the art can make such modification as desired, it is not described in greater detail herein.”; Paragraph 99, “A second embodiment of the invention will be described. The second embodiment takes into account cumulative reward, c, which is the total amount of reward obtained from the first term up to the current term, when determining an optimal action. It can be desirable to take cumulative reward c into consideration in determination of an optimal action for a certain term. This can be the case when one possesses a small asset in a certain state and it is desired to avoid high-risk actions that can lead to bankruptcy and execute low-risk actions, for example. In such a case, it is preferable to determine optimal actions in consideration of cumulative reward c as an asset he possesses.”; Paragraph 101, “FIG. 17 shows an example data structure of transition probability storing unit 130 according to the second embodiment of the invention. Transition probability storing unit 130 stores the transition probability of transitioning to a specific pair of a state and a cumulative reward when an action is executed with each of combinations of states and cumulative rewards. Cumulative reward is represented by variable c, specifically, c.sub.1, c.sub.2, . . . , c.sub.q and so on. A set of possible cumulative rewards is defined as C. Accordingly, c.epsilon.C. As designations relating to states and actions are similar to those of the first embodiment, their detailed descriptions are omitted. Transition probability storing unit 130 according to embodiments of the invention stores a transition probability for each combination of a set of a source state, cumulative reward, and an action (s, c, a), and a pair of a target state and a cumulative reward (s', c').”; Paragraph 103, “The second embodiment differs from the first embodiment in how to calculate probability distribution of an evaluation value X. Specifically, the probability distribution of the evaluation value X calculated by probability distribution calculating unit 420 at step 545 is calculated according to the expression below for a selected action "a" with reference to transition probability storing unit 130 that takes into consideration cumulative reward c, an example of which is shown in FIG. 17. In the expression below, v*(s', c', t+1) represents the optimal value associated with the pair of state s' in the next term and cumulative reward c'. p.sub.t(s', c'|s, c, a) represents the probability of transitioning to the pair of state s' and cumulative reward c' when action a is executed on the pair of state s and cumulative reward c in the current term. The sum represented by .SIGMA. is determined by adding up p.sub.t(s',c'|s,c,a) for all pairs of s' and c' that make v*(s',c',t+1)=x. Note that, unlike the first embodiment, X in the second embodiment corresponds to the optimal value for the next term.”; Paragraph 108, “Because the second embodiment of the invention realizes determination of optimal actions that also considers cumulative reward c, information about preferable actions can be obtained when it is desirable to take cumulative reward c into account at the time of determining optimal actions for a certain term.”; Paragraph 109, “Although the second embodiment of the invention has been described thus far on the assumption that it uses the optimal value v*(s', c', t+1) derived using a finite number of cumulative rewards c, optimal value v*(s', c',t+1) can be made a continuous value using linear interpolation or the like and continuous optimal value v*(s', c',t+1) can be used.”)
the display unit displays, to the user, the step flows of the plurality of operation process candidates and the risk scores of the operation process candidates calculated by the risk evaluation unit. (Morimura: Paragraph 41, “Data stored in transition probability storing unit 130 and reward parameter storing unit 140 can be generated in advance by analyzing information on past research history or the like and updated. Such data can be generated and updated by the user operating computer 110 to access transition probability storing unit 130 and reward parameter storing unit 140 through optimal action decision system 120 and editing data, or by an administrator directly accessing transition probability storing unit 130 and reward parameter storing unit 140 to edit data. Alternatively, information acquired by a computer through automatic and periodical access to an external information source can be analyzed to automatically generate and update such data. As details of generation and update of data can be implemented by so-called skilled artisans as appropriate, they are not described here in further detail.”; Paragraph 42, “Optimal action decision system 120 according to embodiments of the invention determines an optimal action that takes risk into consideration for each of states that are possible in each term of the time horizon (T.sub.1, T.sub.2, . . . , T.sub.n) according to specific preferences. Information on optimal actions determined is presented to the user via user computer 110.”; Paragraph 51, “User interface unit 405 according to embodiments of the invention functions as an interface for exchanging information data with user computer 110. Specifically, user interface unit 405 has state transition pattern generating unit 410 generate state transition patterns upon receiving a request for information about optimal actions from user computer 110, and triggers a sequence of processing for determining optimal actions for the state transition patterns. User interface unit 405 also has a function to send information about optimal actions determined back to user computer 110.”; Paragraph 82, “If it is determined at step 585 that the current term (t) is the first term (t=T.sub.1), the flow proceeds to step 590 through YES arrow. At step 590, optimal actions for all possible states from the first term (t=T.sub.1) through the last term (t=T.sub.n) are presented to the user as information for making a decision. The process then proceeds to step 595 through NO arrow and terminates.”)
Morimura does not explicitly disclose the following, however, in analogous art of process determination and risk analysis, Yellapragada teaches the following:
… including a step of performing an inference by artificial intelligence… (Yellapragada: Paragraph 48, “Thus, through the use of computed overall model desirability value as discussed above, the apparatus, system, and method considers various statistical and business metrics as well as constraints guiding automated machine learning machinery while it trains, evaluates, and compares suitable AI/IL models. In this disclosure, AI/IL models and prediction models are the same and may be used interchangeably. The apparatus, system, and method guides a user and drives automation to select the best AI/IL model(s) for deployment, given the expected business value associated with the selected model while also considering business requirements and constraints.”; Paragraph 51, “An algorithm is the specific computational method used to estimate the parameters of a prediction model. An analytic method refers to the process by predictive models are generated, for example, by using automatic machine learning (AutoML) or by selecting a-priori certain statistical or machine learning algorithms to use for building predictive models from model training data. An analytic approach defines a category of algorithm for a specific purpose or having certain common characteristics; for example, there are analytic approaches for classification tasks, that include many machine learning algorithms suitable for predicting classifications such as whether or not a credit card customer will or will not default on debt payments, or image classification; or, for example, there are analytic approaches for regression tasks, that include many algorithms for regression prediction of continuous outcome variables. There are many ways how machine learning algorithms can be classified into types of analytic approaches, and the specific manner by which analytic approaches can be defined is usually domain specific (different in different business domains).”)
Morimura in view of Yellapragada does not explicitly disclose the following, however, in analogous art of process determination and risk analysis, Takigawa teaches the following:
… calculates a disadvantage score that is calculated based on a negative value of the influence evaluation value of the influence generated in the route, and calculates the disadvantage score of a first route leading to an incorrect conclusion as a sum of a negative value of the influence evaluation value of the influence generated in the first route and a positive value of the influence evaluation value of the influence generated in a second route leading to a correct conclusion in a step flow the same as the first route… (Takigawa: Paragraph 173, “The fourth learning unit 5-4 makes a comparison between the state quantity data and a state quantity in a normal operation recorded in advance, to determine whether or not a malfunction or failure has occurred in the apparatus 3 (step S502). When a malfunction or failure is determined to have occurred in the apparatus 3, the fourth learning unit 5-4 extracts the state quantity of the apparatus 3 including a history of the state quantity of the failing apparatus 3, such as failure information, characteristic change information and the like stored until the time of occurrence of the malfunction (step S503). The fourth learning unit 5-4 estimates a failing component from the extracted state quantity, with reference to a value function, which is a learning result of the fourth learning unit 5-4, for determining the value of a failing component estimation result (step S504). Based on the failing component estimation result, the decision making unit 15 determines the failing component estimation result to be outputted, with reference to the learning result by the fourth learning unit 5-4, and outputs the failing component estimation result to the information management device 2 (step S505).”; Paragraph 174, “Next, upon receiving information about the occurrence of a malfunction or failure in a certain apparatus 3, the information management device 2 checks the failing component of the apparatus 3 (step S506). The determination result obtainment unit 12 of the fourth machine learning device 4-4 obtains the check result as a determination result, and outputs the determination result to the reward calculation unit 13 (step S507). The failing component is replaced or the like, to repair the failing apparatus 3 (step S508).”; Paragraph 175, “Subsequently, the fourth learning unit 5-4 determines whether or not the actual failing component checked in step S506 coincides with the failing component estimation result outputted from the decision making unit 15 (step S509). When the actual failing component coincides with the failing component estimation result, the reward calculation unit 13 provides a positive reward (step S510). When the actual failing component does not coincide with the failing component estimation result, the reward calculation unit 13 provides a negative reward (step S511). The value function update unit 14 updates a value function for determining the value of the failing component estimation result, in accordance with the reward (step S512). Furthermore, the fourth machine learning device 4-4 adds failure information on the failing apparatus 3 to a failing information history for update (step S513).”; Paragraph 177, “As described above, by repetitions of a series of processes from step S501 to step S514, the fourth learning unit 5-4 can learn the failure component estimation result on the apparatus 3, in association with the state quantity of the apparatus 3 including the history of the state quantity of the apparatus 3, and the determination result corresponding to the failing component estimation result. Therefore, the fourth learning unit 5-4 can diagnose the failure in the apparatus 3.”; Paragraph 178, “The failure component estimation result to be outputted from the fourth machine learning device 4-4 may include not only a first candidate having the highest probability, but also a plurality of candidates that are listed in decreasing order of probability. In this case, the higher ranked the candidate the actual failing component coincides with, the larger positive reward the reward calculation unit 13 provides. When the actual failing component coincides with none of the listed candidates, the reward calculation unit 13 provides a negative reward. Based on the rewards provided under such conditions, the value function update unit 14 updates the value function.”; Paragraph 250, “the fourth learning unit has a value function to determine the value of the estimation result as to the failing designated component, and the fourth learning unit includes a reward calculation unit for providing a positive reward when the estimation result is right, while providing a negative reward when the estimation result is wrong, and a value function update unit for updating the value function in accordance with the reward, and”; Paragraph 255, “the higher ranked the candidate of the plurality of candidates the actual failing designated component coincides with, the larger positive reward the reward calculation unit provides, while when the actual failing component coincides with none of the listed candidates, the reward calculation unit provides the negative reward.”; Paragraph 187, “In the machine learning device 4 connected to the information management device 2, a state quantity observation unit 6 observes at least one of the various state quantities described above, as the state quantity of the laser apparatus 16, and a determination result obtainment unit 12 receives failure information and characteristic change information of each component constituting the laser apparatus 16. Furthermore, a learning unit 5 provided in the machine learning device 4 may learn cumulative failure rate curves and a first acceleration coefficient of components the failure sign of which is unknown in order to perform inventory management based on the sum of cumulative failure rates of the designated components having the same specifications, and may learn characteristic change curves and a second acceleration coefficient of a characteristic deteriorating component the characteristic of which gradually deteriorate by driving in order to perform inventory management and preventive maintenance on the characteristic deteriorating component. The learning unit 5 may detect a failure sign of a component the failure sign of which has not been known, and may learn a time variation in a failure occurrence probability after appearance of the detected failure sign, and the dependence of a component showing the sign on a driving condition, in order to perform inventory management and preventive maintenance of the component showing the sign. The learning unit 5 may learn a value function to determine the value of a failure component estimation result in order to increase a failure diagnostic ability to identify a failing component.”; Paragraph 188, “The laser apparatus 16 includes a component the failure rate and acceleration coefficient of which are difficult to calculate precisely by a test of the component by itself. For example, as to a component that may be damaged by reflected light, the failure rate of the component cannot be precisely calculated by itself. It is necessary to calculate the failure rate of the component in a state in which the component is installed in the laser apparatus 16. In the inventory management system 1 according to this embodiment, since the cumulative failure rate and the acceleration coefficient of the component are learned by machine learning in a state in which the component is installed in the laser apparatus 16, it is possible to calculate the failure rate and the acceleration coefficient of the component installed in the laser apparatus 16 with high precision. Therefore, the inventory management system according to the present invention can be beneficially applied to the laser apparatus 16.”)
Morimura discloses a method for determining optimal actions along a series of steps for a process. Yellapragada discloses a method for determining an optimal prediction model for making informed decisions. Takigawa discloses a method for training a model to make decisions for predicted action items. At the time of Applicant’s filed invention one of ordinary skill in the art would have deemed it obvious to combine the methods of Morimura with the teachings of Yellapragada in order to improve the model for determining optimal decisions as disclosed by Yellapragada (Yellapragada: Paragraph 30, “This in turn will significantly shorten the time from defining a business problem that can be addressed using Machine Learning and AI, to the successful and continuously sustainable deployment of ML/AI-based solutions given current business constraints.”). It would have been further obvious to combine the art of Morimura in view of Yellapragada with the teachings of Takigawa, in order to improve the time efficiency and reliability of training a model as disclosed by Takigawa (Takigawa: Paragraph 196, “Since the machine learning devices 4 share information such as the learning result information, it is possible to perform learning in a shorter time and obtain more reliable learning results.”)
Claim(s) 2 –
Morimura in view of Yellapragada and Takigawa disclose the limitations of claim 1
Morimura further discloses the following:
wherein the data of the operation process candidates includes a change list in which presence or absence of a change is determined by comparing a step flow of the operation process candidate with a step flow of an operation process that does not include a step of performing an inference by artificial intelligence, and (Morimura: Paragraph 62, “Optimal action decision unit 440 according to embodiments of the invention compares value measures for action candidates that can be executed in a state possible at a certain time step calculated by value measure calculating unit 435, and determines an optimal action for that state. Upon having determined optimal actions for all possible states for the current state, optimal action decision unit 440 assumes the term immediately before the current term is now the current term and sends an instruction to state transition pattern generating unit 410 to generate state transition patterns that are possible in the new current term.”; Paragraph 79, “At step 570, value measures VM calculated for possible action candidates for state s calculated in the loop from step 540 to 560 are compared with each other, and an action having the best value measure is determined to be the optimal action for state s. Then, at step 575, the optimal action determined at step 570 and the value measure associated with the optimal action are stored as the optimal value v*(s, t).”; Paragraph 88, “The above described process is executed on the other action candidates (a.sub.2, a.sub.3) (the loop from steps 540 through 565) to result in VM (a.sub.1, s.sub.1, T.sub.k), VM (a.sub.2, s.sub.1, T.sub.k), VM (a.sub.3, s.sub.1, T.sub.k), which are value measures for the individual action candidates a.sub.1, a.sub.2, a.sub.3 that can be executed in possible state s.sub.1 at T.sub.k (see FIG. 12). The value measures are compared with one another, an action candidate corresponding to the best value measure is determined to be the optimal action in possible state s.sub.1 at T.sub.k (step 570), and the optimal action and the corresponding value measure (the optimal value) are stored (step 575). In this example, VM (a.sub.2, s.sub.1, T.sub.k) is best among the three value measures and accordingly action a.sub.2 is determined to be the optimal action (see FIG. 13).”)
Claim(s) 3 –
Morimura in view of Yellapragada and Takigawa disclose the limitations of claim 1
Morimura does not explicitly disclose the following, however, in analogous art of process determination and risk analysis, Yellapragada teaches the following:
wherein the data of the operation process candidates includes an ease evaluation list in which ease of detecting incorrectness of an inference performed by artificial intelligence in the step flow of the operation process candidate is determined, and (Yellapragada: Paragraph 36, “With this approach, models selected based on multiple evaluation criteria, such as cost, consistency with organization's goals, Key Performance Indicators (KPI's), constraints, and potential risks, or any combination of criteria thereof, are aligned and unified with processes and methods that are used to build and select AWL models for deployment. The model building process, as well as other properties of the AWL models, are considered inputs to a multi-objective optimization problem reflecting model desirability which is a function of multiple evaluation criteria such as model accuracy or deployment cost as well as the expected benefits of models, while also reflecting in the overall model desirability values applicable constraints with respect to performance requirements and resources, as well as possible liabilities and risks, e.g., due to unintended discriminatory impact or inaccuracies in certain input regions of an algorithmic model.”; Paragraph 41, “model complexity and the time required to score a new observation;”; Paragraph 46, “the fidelity of the distributions of predictions with respect to actual distributions of values”; Paragraph 47, “statistical and prediction-accuracy-derived performance indicators.”; Paragraph 49, “A desirability function is a function used and well understood in the area of statistics and data science. The desirability function defines the mapping of values and value ranges over different evaluation criteria into the overall desirability value for a prediction model with respect to the respective organizations' goals, policies, or other key performance indicators. Criteria, such as scoring cost or model accuracy and others described in previous paragraphs, are the dimensions that must be considered by the respective organization in order to assess the overall desirability of a model. A constraint defines permissible values or value ranges for criterion, criteria or combinations of criteria. Constraints can be hard constraint or soft. A hard constraint, for example, can be one where the desirability of the prediction model will be 0 (zero) if the value of a criterion falls outside the permissible bound defining the constraint (e.g., a specific model may have a discriminatory impact that is greater than what is allowed by law, and hence cannot be chosen and used by the organization). A soft constraint, for example, can be one where certain value ranges of criteria or combination of criteria will make a model increasingly less desirable. For example, if the cost of scoring a single new case, row, or instance given a certain prediction model exceeds some value, then the degree to which the cost of scoring exceeds that value will rapidly decrease the desirability of that model towards 0 (zero). The desirability function can transform specific measured or assigned values for criterion or criteria, circumstances, facts, or influences into a numeric value that indicates overall desirability with respect to the undesirable consequences and desired benefits that are expected to accrue to the business or organization when the respective prediction model is used for scoring new data, and when those predictions in turn are used to inform process decisions that will affect organizational outcomes. Criterion values that map to small desirability values make a respective model less desirable; criterion values that map to larger desirability values make the respective model more desirable.”; Paragraph 53, “The business model 14 is a data set of variables that describe a process, such as organizational structure encompassing an organization's departmental structure, processes, and products. The data science and business analysis module 16 is a module that allows a user to enter select variables, such as predictor variables and predictor variable types and outcome variables and outcome variable types, model analytics type, algorithmic model accuracy criterion or criteria, evaluation criterion or criteria for algorithmic model quality, and an evaluation criterion or criteria that will impact model desirability. The algorithmic model generator 12 comprises an AutoML based or other systems equivalent thereto for generating prediction models 12a and a desirability function module 12b. The model data set 18 is a database repository comprising model data sets relevant to a business model that are used to train algorithmic models to obtain predictive results for certain predictor and outcome variables and types. The Display generator 20 is an interactive graphics display and programmed User Interface (UI) module that generates graphical displays for displaying the algorithmic models, graphs, charts, criterion and criteria, and variables generated from the algorithmic model generator 12 and provides user feedback to the algorithmic model generator 12.”)
the risk evaluation unit calculates a risk score for each route based on a corrected occurrence probability obtained by correcting, based on the determination of the ease evaluation list, the occurrence probability of the route leading to the conclusion that is to be taken by the operation process candidate. (Yellapragada: Paragraph 54, “In an embodiment, the desirability function module 12b processes outcome and predictor variables, outcome and predictor variable types, model analytics types, algorithmic model accuracy criterion or criteria, evaluation criterion or criteria for algorithmic model quality, and an evaluation criterion or criteria that will affect model deployment desirability, such as but not limited to the monetary costs, value, risks, degree of compliance with policy or regulatory requirements, and others as enumerated in 48-52 of FIG. 2a and FIG. 2b, in order to generate a desirability function. The Model Building/Generating or automatic AutoML based system 12a samples the model data sets to generate a set of trained algorithmic models and predictive results, i.e. observations, based on outcome, input and predictor variables, outcome, input and predictor variable types, model analytics types, and the desirability function provided by the desirability function module 12b. The display generator 20 generates an interactive logical interface using a graphical display language, such as a java based HTML (Hypertext Markup Language), and syncs the interface with the display module 20. FIG. 1B is an illustration of display generator 20 generated metrics for a group of identified or generated predictive models where the Model Building/Generating or AutoML based system 12a uses only performance criterion to generate or identify the predictive models. FIG. 1B is an illustration of display generator 20 generated metrics for a group of identified or generated predictive models where the Model Building/Generating or AutoML based system 12a and desirability function module 12b uses multivariate criteria to generate or identify the predictive models.”; Paragraph 56, “Referring now FIGS. 2A-2B, illustrated is an algorithmic flow diagram of logic for the algorithmic model generator 12, denoted generally as 40. At block 44, based on a particular process, outcome variables, outcome types, predictor variable, predictor variable types, input variables, input variable types, and analytic methods, algorithm types, and approaches for ML, AI, or linear models for simple interpretation and traceability to causes can be selected. At block 46, the evaluation or evaluations for model accuracy, complexity, and fidelity are selected. Measures for model accuracy, complexity, and fidelity can be selected. The order of importance for each criterion and a weight for each criterion based on the importance can also be selected.”; Paragraph 58, “At block 50, other business criteria or criterion can be selected. A trained algorithmic models quality criterion for accuracy, complexity, fidelity, deployment, value, and cost can be selected. Each criteria listed in block 46 and 48 can be assigned a constraint. Also, constraints can be assigned for: trained algorithmic models performance value, cost or both; trained algorithmic models for model estimation cost; limits for trained algorithmic models deployment cost; maximum number of inputs for interpretability of trained algorithmic models; and criteria or criterion per variable against which to evaluate for discriminatory impact of algorithmic models.”; Paragraph 60, “Distribution comparison for deploying and scoring trained algorithmic models can be considered. A comparison between the distributions of target values from the training, testing, and holdout datasets compared to the predictions on the hold out dataset to evaluate model fit. On large number of observations, the distributions of observed and predicted values must be similar. Comparison of outputs from simulated data with respect to certain predictors, such as demographic properties, that would indicate bias and discriminatory impact of a trained algorithmic model can be considered.”)
Morimura discloses a method for determining optimal actions along a series of steps for a process. Yellapragada discloses a method for determining an optimal prediction model for making informed decisions. Takigawa discloses a method for training a model to make decisions for predicted action items. At the time of Applicant’s filed invention one of ordinary skill in the art would have deemed it obvious to combine the methods of Morimura with the teachings of Yellapragada in order to improve the model for determining optimal decisions as disclosed by Yellapragada (Yellapragada: Paragraph 30, “This in turn will significantly shorten the time from defining a business problem that can be addressed using Machine Learning and AI, to the successful and continuously sustainable deployment of ML/AI-based solutions given current business constraints.”).
Claim(s) 4, 8, and 12 –
Morimura in view of Yellapragada and Takigawa disclose the limitations of claims 1, 7, and 11
Morimura further discloses the following:
a cost evaluation unit as the functional unit, wherein the data of the operation process candidates includes a checking ratio list defining a checking ratio at which the conclusion of the operation process candidate is checked, and the checking ratio is determined according to a risk score of the step flow of the operation process candidate, (Morimura: Paragraph 40, “Reward parameter storing unit 140 according to embodiments of the invention stores parameters indicative of probability distribution of reward that is obtained when transition to a state occurs as a result of executing an action in each of possible states. Reward can be defined as profit or loss that is produced if one sells part of or adds to an asset portfolio he possesses, for example. Parameters indicating reward probability distribution can be a mean and a variance for probability distribution compliant with normal distribution, for example.”; Paragraph 42, “Optimal action decision system 120 according to embodiments of the invention determines an optimal action that takes risk into consideration for each of states that are possible in each term of the time horizon (T.sub.1, T.sub.2, . . . , T.sub.n) according to specific preferences. Information on optimal actions determined is presented to the user via user computer 110.”; Paragraph 69, “The process then proceeds to step 550, where predetermined risk measure V.sub..alpha. is calculated for the probability distribution of X determined at step 545. Embodiments of the invention determine risk measure V.sub..alpha. using Value at Risk. In embodiments of the invention, risk measure V.sub..alpha. is defined as the maximum amount of loss that is produced with a predetermined probability .alpha. (e.g., .alpha.=1%), and risk measure V.sub..alpha. for the case can be determined according to Expression 2 below using probability distribution of X (for example, the amount of loss that is produced with the percentage of .alpha.=1% with respect to probability distribution of X can be defined as V.sub..alpha.).”; Paragraph 73, “ICTE is calculated as follows. CTE for future loss Y as viewed from certain time step T.sub.k-1 is calculated. CTE at time step T.sub.k-1 can be deemed as a random variable when viewed from time step T.sub.k-2, and CTE at T.sub.k-1 varies depending on what occurs between T.sub.k-2 and T.sub.k-1. Thus, regarding CTE at time step T.sub.k-1 as "loss", CTE for the loss can be calculated at time step T.sub.k-2. Likewise, CTE at time step T.sub.k-2 is a random variable when seen from T.sub.k-3 and CTE for CTE at T.sub.k-2 can be calculated at time step T.sub.k-3. A risk measure determined by repetitively calculating CTE for CTE through iteration of this process represents ICTE (for details, see M. R. Hardy and J. L. Wirch, "The iterated CTE: A dynamic risk measure," The North American Actuarial Journal, 62-75, 2004.).”; Paragraph 74, “When a preference to use such ICTE for optimization, e.g., a preference to minimize ICTE[Y] which is ICTE for loss Y, is adopted, the following weighting function w(y) can be established, where V.sub..alpha. represents Value at Risk for Y. Extent of risk taking can be adjusted again by varying the value of .alpha.. Note that although loss Y is used here instead of evaluation value X for facilitating understanding of ICTE, Y is generally the negative of X.”; Paragraph 79, “At step 570, value measures VM calculated for possible action candidates for state s calculated in the loop from step 540 to 560 are compared with each other, and an action having the best value measure is determined to be the optimal action for state s. Then, at step 575, the optimal action determined at step 570 and the value measure associated with the optimal action are stored as the optimal value v*(s, t).”)
the cost evaluation unit calculates, for each step flow of the operation process candidate, a checking cost based on a checking ratio corresponding to the risk score of the step flow, and calculates, as a checking cost of the operation process candidate, a sum of the checking costs calculated for the step flows of the operation process candidate, and (Morimura: Paragraph 40, “Reward parameter storing unit 140 according to embodiments of the invention stores parameters indicative of probability distribution of reward that is obtained when transition to a state occurs as a result of executing an action in each of possible states. Reward can be defined as profit or loss that is produced if one sells part of or adds to an asset portfolio he possesses, for example. Parameters indicating reward probability distribution can be a mean and a variance for probability distribution compliant with normal distribution, for example.”; Paragraph 42, “Optimal action decision system 120 according to embodiments of the invention determines an optimal action that takes risk into consideration for each of states that are possible in each term of the time horizon (T.sub.1, T.sub.2, . . . , T.sub.n) according to specific preferences. Information on optimal actions determined is presented to the user via user computer 110.”; Paragraph 69, “The process then proceeds to step 550, where predetermined risk measure V.sub..alpha. is calculated for the probability distribution of X determined at step 545. Embodiments of the invention determine risk measure V.sub..alpha. using Value at Risk. In embodiments of the invention, risk measure V.sub..alpha. is defined as the maximum amount of loss that is produced with a predetermined probability .alpha. (e.g., .alpha.=1%), and risk measure V.sub..alpha. for the case can be determined according to Expression 2 below using probability distribution of X (for example, the amount of loss that is produced with the percentage of .alpha.=1% with respect to probability distribution of X can be defined as V.sub..alpha.).”; Paragraph 73, “ICTE is calculated as follows. CTE for future loss Y as viewed from certain time step T.sub.k-1 is calculated. CTE at time step T.sub.k-1 can be deemed as a random variable when viewed from time step T.sub.k-2, and CTE at T.sub.k-1 varies depending on what occurs between T.sub.k-2 and T.sub.k-1. Thus, regarding CTE at time step T.sub.k-1 as "loss", CTE for the loss can be calculated at time step T.sub.k-2. Likewise, CTE at time step T.sub.k-2 is a random variable when seen from T.sub.k-3 and CTE for CTE at T.sub.k-2 can be calculated at time step T.sub.k-3. A risk measure determined by repetitively calculating CTE for CTE through iteration of this process represents ICTE (for details, see M. R. Hardy and J. L. Wirch, "The iterated CTE: A dynamic risk measure," The North American Actuarial Journal, 62-75, 2004.).”; Paragraph 74, “When a preference to use such ICTE for optimization, e.g., a preference to minimize ICTE[Y] which is ICTE for loss Y, is adopted, the following weighting function w(y) can be established, where V.sub..alpha. represents Value at Risk for Y. Extent of risk taking can be adjusted again by varying the value of .alpha.. Note that although loss Y is used here instead of evaluation value X for facilitating understanding of ICTE, Y is generally the negative of X.”; Paragraph 79, “At step 570, value measures VM calculated for possible action candidates for state s calculated in the loop from step 540 to 560 are compared with each other, and an action having the best value measure is determined to be the optimal action for state s. Then, at step 575, the optimal action determined at step 570 and the value measure associated with the optimal action are stored as the optimal value v*(s, t).”)
Morimura does not explicitly disclose the following, however, in analogous art of process determination and risk analysis, Yellapragada teaches the following:
the display unit displays, to the user, the step flows of 37 the plurality of operation process candidates and the checking costs of the operation process candidates calculated by the cost evaluation unit. (Yellapragada: Paragraph 33, “In general, the formulas that are proposed combine into a single desirability value the desirability of specific values or value ranges for multiple measured or ranked KPI's (Key Performance Indicators) or quality criteria. The values that make up the single desirability value, which is a function of quality criteria, can be classified as a cost, a neutral, a benefit, or something there between. In this disclosure, term value and benefit with respect to a value associated with a criterion have the same meaning and may be used interchangeably.”; Paragraph 36, “With this approach, models selected based on multiple evaluation criteria, such as cost, consistency with organization's goals, Key Performance Indicators (KPI's), constraints, and potential risks, or any combination of criteria thereof, are aligned and unified with processes and methods that are used to build and select AWL models for deployment. The model building process, as well as other properties of the AWL models, are considered inputs to a multi-objective optimization problem reflecting model desirability which is a function of multiple evaluation criteria such as model accuracy or deployment cost as well as the expected benefits of models, while also reflecting in the overall model desirability values applicable constraints with respect to performance requirements and resources, as well as possible liabilities and risks, e.g., due to unintended discriminatory impact or inaccuracies in certain input regions of an algorithmic model.”; Paragraph 55, “The algorithmic model generator 12 can build predictive models based on the model data sets 18 and user provided input variables and outcome variables for predicting the outcome variable from the input variables, as chosen by a respective organization. The input variables can be independent variables (causation related) and predictor variables. The algorithmic model generator 12 can implement a fully automated AutoML system as previously described, or it can cycle over a list of different analytic approaches or algorithms. Further, the Model Building/Generating or AutoML based system 12a can use the desirability function module 12b to generate at least one algorithmic model; for example, the Model Building/Generating or AutoML based system 12a can use an overall desirability value or values generated by the desirability function module 12b based on certain criteria or criterion, e.g. cost of misclassifying (predicting incorrectly) a row or case, depending on the specific category to which the respective case belongs and the specific category to which it was misclassified, to generate predictive classification models that satisfy the overall desirability value or values. However, even if the model generated is not generated based on criteria, each generated model can be associated with specific values for specific criteria as previously discussed; those criteria can be converted into a desirability value for each algorithmic model by the desirability function depicted in 12b. The resulting table of prediction models, the values of associated criteria for each model, and the value of the overall desirability for each model is then displayed to the user via display generator 20. The user can review these results via display generator 20 for each model and choose a model or configure the system and UI to choose automatically the most desirable model for final deployment for the respective designated business process where the respective prediction model is to be applied. Details of the results, values of criteria, and desirability values can be displayed through display generator 20. Display generator 20 also allows users to manually select different variables or choose to modify the specific algorithms and analytic approaches used in the algorithmic model generator block 12.”; Paragraph 90, “a desirability function module configured to generate a desirability function, wherein the desirability function defines: at least one outcome variable and outcome variable type and at least one predictor variable and at least one predictor variable type; and at least one algorithmic model accuracy criterion, at least one model analytics type, at least one evaluation criterion for algorithmic model quality, and at least one evaluation criterion for model deployment cost; an automated machine learning module configured to: generate at least one algorithmic model having a variable set selected according to the desirability function; and train the at least one algorithmic model against the model data set; and a UI (User Interface) module configured to generate a user interface to display the at least one algorithmic model accuracy criterion, the at least one model analytics type, the at least one evaluation criterion for algorithmic model quality, and the at least one evaluation criterion for model deployment cost; wherein the displayed criteria and cost are selectable and definable;”)
Morimura discloses a method for determining optimal actions along a series of steps for a process. Yellapragada discloses a method for determining an optimal prediction model for making informed decisions. Takigawa discloses a method for training a model to make decisions for predicted action items. At the time of Applicant’s filed invention one of ordinary skill in the art would have deemed it obvious to combine the methods of Morimura with the teachings of Yellapragada in order to improve the model for determining optimal decisions as disclosed by Yellapragada (Yellapragada: Paragraph 30, “This in turn will significantly shorten the time from defining a business problem that can be addressed using Machine Learning and AI, to the successful and continuously sustainable deployment of ML/AI-based solutions given current business constraints.”).
Claim(s) 5, 9, and 13 –
Morimura in view of Yellapragada and Takigawa disclose the limitations of claims, 1, 4, 7-8, and 11-12
Morimura does not explicitly disclose the following, however, in analogous art of process determination and risk analysis, Yellapragada teaches the following:
wherein the checking ratio list defines a checking ratio for correctness and incorrectness checking of checking correctness and incorrectness of a conclusion of an operation process and a checking ratio for performance deviation checking of checking whether a deviated determination is made from a viewpoint of AI logic. (Yellapragada: Paragraph 33, “In general, the formulas that are proposed combine into a single desirability value the desirability of specific values or value ranges for multiple measured or ranked KPI's (Key Performance Indicators) or quality criteria. The values that make up the single desirability value, which is a function of quality criteria, can be classified as a cost, a neutral, a benefit, or something there between. In this disclosure, term value and benefit with respect to a value associated with a criterion have the same meaning and may be used interchangeably.”; Paragraph 36, “With this approach, models selected based on multiple evaluation criteria, such as cost, consistency with organization's goals, Key Performance Indicators (KPI's), constraints, and potential risks, or any combination of criteria thereof, are aligned and unified with processes and methods that are used to build and select AWL models for deployment. The model building process, as well as other properties of the AWL models, are considered inputs to a multi-objective optimization problem reflecting model desirability which is a function of multiple evaluation criteria such as model accuracy or deployment cost as well as the expected benefits of models, while also reflecting in the overall model desirability values applicable constraints with respect to performance requirements and resources, as well as possible liabilities and risks, e.g., due to unintended discriminatory impact or inaccuracies in certain input regions of an algorithmic model.”; Paragraph 55, “The algorithmic model generator 12 can build predictive models based on the model data sets 18 and user provided input variables and outcome variables for predicting the outcome variable from the input variables, as chosen by a respective organization. The input variables can be independent variables (causation related) and predictor variables. The algorithmic model generator 12 can implement a fully automated AutoML system as previously described, or it can cycle over a list of different analytic approaches or algorithms. Further, the Model Building/Generating or AutoML based system 12a can use the desirability function module 12b to generate at least one algorithmic model; for example, the Model Building/Generating or AutoML based system 12a can use an overall desirability value or values generated by the desirability function module 12b based on certain criteria or criterion, e.g. cost of misclassifying (predicting incorrectly) a row or case, depending on the specific category to which the respective case belongs and the specific category to which it was misclassified, to generate predictive classification models that satisfy the overall desirability value or values. However, even if the model generated is not generated based on criteria, each generated model can be associated with specific values for specific criteria as previously discussed; those criteria can be converted into a desirability value for each algorithmic model by the desirability function depicted in 12b. The resulting table of prediction models, the values of associated criteria for each model, and the value of the overall desirability for each model is then displayed to the user via display generator 20. The user can review these results via display generator 20 for each model and choose a model or configure the system and UI to choose automatically the most desirable model for final deployment for the respective designated business process where the respective prediction model is to be applied. Details of the results, values of criteria, and desirability values can be displayed through display generator 20. Display generator 20 also allows users to manually select different variables or choose to modify the specific algorithms and analytic approaches used in the algorithmic model generator block 12.”; Paragraph 90, “a desirability function module configured to generate a desirability function, wherein the desirability function defines: at least one outcome variable and outcome variable type and at least one predictor variable and at least one predictor variable type; and at least one algorithmic model accuracy criterion, at least one model analytics type, at least one evaluation criterion for algorithmic model quality, and at least one evaluation criterion for model deployment cost; an automated machine learning module configured to: generate at least one algorithmic model having a variable set selected according to the desirability function; and train the at least one algorithmic model against the model data set; and a UI (User Interface) module configured to generate a user interface to display the at least one algorithmic model accuracy criterion, the at least one model analytics type, the at least one evaluation criterion for algorithmic model quality, and the at least one evaluation criterion for model deployment cost; wherein the displayed criteria and cost are selectable and definable;”)
Morimura discloses a method for determining optimal actions along a series of steps for a process. Yellapragada discloses a method for determining an optimal prediction model for making informed decisions. Takigawa discloses a method for training a model to make decisions for predicted action items. At the time of Applicant’s filed invention one of ordinary skill in the art would have deemed it obvious to combine the methods of Morimura with the teachings of Yellapragada in order to improve the model for determining optimal decisions as disclosed by Yellapragada (Yellapragada: Paragraph 30, “This in turn will significantly shorten the time from defining a business problem that can be addressed using Machine Learning and AI, to the successful and continuously sustainable deployment of ML/AI-based solutions given current business constraints.”).
Claim(s) 6, 10, and 14 –
Morimura in view of Yellapragada and Takigawa disclose the limitations of claims 1, 7, and 11
Morimura further discloses the following:
a cost evaluation unit by being loaded into the memory and executed by the processor, (Morimura: Paragraph 40, “Reward parameter storing unit 140 according to embodiments of the invention stores parameters indicative of probability distribution of reward that is obtained when transition to a state occurs as a result of executing an action in each of possible states. Reward can be defined as profit or loss that is produced if one sells part of or adds to an asset portfolio he possesses, for example. Parameters indicating reward probability distribution can be a mean and a variance for probability distribution compliant with normal distribution, for example.”; Paragraph 43, “Thus, information system 100 according to the first embodiment aims to output for presentation to the user optimal actions that consider risk in accordance with specific preferences in response to a request from the user, when provided beforehand with data on state transition probability and/or rewards for actions executed in a state.”)
the data of the operation process candidates includes an execution cost list indicating an execution cost for each step included in the operation process candidate, (Morimura: Paragraph 46, “FIG. 3 shows an example of data structure for reward parameter storing unit 140 according to embodiments of the invention. Reward parameter storing unit 140 stores a parameter indicating the probability distribution of reward that is obtained when transition to a certain state occurs as a result of executing an action in each of possible states.”; Paragraph 48, “Since probability distribution cannot be determined only with a mean for some kinds of reward distribution, reward parameter storing unit 140 stores similar tables to the one shown in FIG. 3 respectively for variance and other required parameters (not shown). Other required parameters can include skewness and location parameters for stable distribution, for example. As a data structure for storing these parameters is similar to the above-described one except that the mean value is replaced with other such parameters, it is not described in greater detail.”; Paragraph 55, “3) Attribute data representing actions that can be executed in each state, the transitional probability of transitioning to each target state if each action is executed, and the probability distribution of reward that is obtained when each action is executed.”; Paragraph 68, “Then, at step 545, probability distribution of the evaluation value X is determined according to the expression shown below for the action `a` selected at step 540. In the expression, r represents reward, and v*(s', t+1) is the optimal value relating to state s' of the next term (the term immediately following t, here T.sub.n). p.sub.t(s', r|s, a) is defined as the probability that transition to state s' occurs and reward r is obtained if action `a` is executed in state s in the current term (t=T.sub.n-1). A sum represented by .SIGMA. is determined by adding up p.sub.t(s', r|s, a) for all pairs of s' and r that make r+v*(s',t+1)=x. In short, X represents such a discrete distribution that value x is assumed with a certain probability and, in this embodiment, corresponds to the sum of reward obtained in the current term and the optimal value for the next term.”; Paragraph 72, “The ICTE is calculated by repetitively applying CTE (Conditional Tail Expectation). CTE is also known as Conditional Value at Risk or Expected Short Fall, calculated using Value at Risk. To be specific, CTE is determined as an expectation of loss Y that is produced when Y exceeds V.sub..alpha., according to the expression below using Value at Risk V.sub..alpha. which indicates that loss does not exceed V.sub..alpha. with the probability of .alpha.(0<.alpha.<1) (however, the expression below assumes that probability distribution of Y is continuous):”; Paragraph 99, “A second embodiment of the invention will be described. The second embodiment takes into account cumulative reward, c, which is the total amount of reward obtained from the first term up to the current term, when determining an optimal action. It can be desirable to take cumulative reward c into consideration in determination of an optimal action for a certain term. This can be the case when one possesses a small asset in a certain state and it is desired to avoid high-risk actions that can lead to bankruptcy and execute low-risk actions, for example. In such a case, it is preferable to determine optimal actions in consideration of cumulative reward c as an asset he possesses.”)
the cost evaluation unit calculates, for each step flow of the operation process candidate, an execution cost based on an occurrence probability of the step flow and a total execution cost of the step flow calculated based on the execution cost list, and calculates, as an execution cost of the operation process candidate, a sum of the execution costs calculated for the step flows of the operation process candidate, and (Morimura: Paragraph 46, “FIG. 3 shows an example of data structure for reward parameter storing unit 140 according to embodiments of the invention. Reward parameter storing unit 140 stores a parameter indicating the probability distribution of reward that is obtained when transition to a certain state occurs as a result of executing an action in each of possible states.”; Paragraph 48, “Since probability distribution cannot be determined only with a mean for some kinds of reward distribution, reward parameter storing unit 140 stores similar tables to the one shown in FIG. 3 respectively for variance and other required parameters (not shown). Other required parameters can include skewness and location parameters for stable distribution, for example. As a data structure for storing these parameters is similar to the above-described one except that the mean value is replaced with other such parameters, it is not described in greater detail.”; Paragraph 55, “3) Attribute data representing actions that can be executed in each state, the transitional probability of transitioning to each target state if each action is executed, and the probability distribution of reward that is obtained when each action is executed.”; Paragraph 68, “Then, at step 545, probability distribution of the evaluation value X is determined according to the expression shown below for the action `a` selected at step 540. In the expression, r represents reward, and v*(s', t+1) is the optimal value relating to state s' of the next term (the term immediately following t, here T.sub.n). p.sub.t(s', r|s, a) is defined as the probability that transition to state s' occurs and reward r is obtained if action `a` is executed in state s in the current term (t=T.sub.n-1). A sum represented by .SIGMA. is determined by adding up p.sub.t(s', r|s, a) for all pairs of s' and r that make r+v*(s',t+1)=x. In short, X represents such a discrete distribution that value x is assumed with a certain probability and, in this embodiment, corresponds to the sum of reward obtained in the current term and the optimal value for the next term.”; Paragraph 72, “The ICTE is calculated by repetitively applying CTE (Conditional Tail Expectation). CTE is also known as Conditional Value at Risk or Expected Short Fall, calculated using Value at Risk. To be specific, CTE is determined as an expectation of loss Y that is produced when Y exceeds V.sub..alpha., according to the expression below using Value at Risk V.sub..alpha. which indicates that loss does not exceed V.sub..alpha. with the probability of .alpha.(0<.alpha.<1) (however, the expression below assumes that probability distribution of Y is continuous):”; Paragraph 99, “A second embodiment of the invention will be described. The second embodiment takes into account cumulative reward, c, which is the total amount of reward obtained from the first term up to the current term, when determining an optimal action. It can be desirable to take cumulative reward c into consideration in determination of an optimal action for a certain term. This can be the case when one possesses a small asset in a certain state and it is desired to avoid high-risk actions that can lead to bankruptcy and execute low-risk actions, for example. In such a case, it is preferable to determine optimal actions in consideration of cumulative reward c as an asset he possesses.”; Paragraph 101, “FIG. 17 shows an example data structure of transition probability storing unit 130 according to the second embodiment of the invention. Transition probability storing unit 130 stores the transition probability of transitioning to a specific pair of a state and a cumulative reward when an action is executed with each of combinations of states and cumulative rewards. Cumulative reward is represented by variable c, specifically, c.sub.1, c.sub.2, . . . , c.sub.q and so on. A set of possible cumulative rewards is defined as C. Accordingly, c.epsilon.C. As designations relating to states and actions are similar to those of the first embodiment, their detailed descriptions are omitted. Transition probability storing unit 130 according to embodiments of the invention stores a transition probability for each combination of a set of a source state, cumulative reward, and an action (s, c, a), and a pair of a target state and a cumulative reward (s', c').”; Paragraph 109, “Although the second embodiment of the invention has been described thus far on the assumption that it uses the optimal value v*(s', c', t+1) derived using a finite number of cumulative rewards c, optimal value v*(s', c',t+1) can be made a continuous value using linear interpolation or the like and continuous optimal value v*(s', c',t+1) can be used.”)
Morimura does not explicitly disclose the following, however, in analogous art of process determination and risk analysis, Yellapragada teaches the following:
the display unit displays, to the user, the step flows of the plurality of operation process candidates and the execution costs of the operation process candidates calculated by the cost evaluation unit. (Yellapragada: Paragraph 55, “The algorithmic model generator 12 can build predictive models based on the model data sets 18 and user provided input variables and outcome variables for predicting the outcome variable from the input variables, as chosen by a respective organization. The input variables can be independent variables (causation related) and predictor variables. The algorithmic model generator 12 can implement a fully automated AutoML system as previously described, or it can cycle over a list of different analytic approaches or algorithms. Further, the Model Building/Generating or AutoML based system 12a can use the desirability function module 12b to generate at least one algorithmic model; for example, the Model Building/Generating or AutoML based system 12a can use an overall desirability value or values generated by the desirability function module 12b based on certain criteria or criterion, e.g. cost of misclassifying (predicting incorrectly) a row or case, depending on the specific category to which the respective case belongs and the specific category to which it was misclassified, to generate predictive classification models that satisfy the overall desirability value or values. However, even if the model generated is not generated based on criteria, each generated model can be associated with specific values for specific criteria as previously discussed; those criteria can be converted into a desirability value for each algorithmic model by the desirability function depicted in 12b. The resulting table of prediction models, the values of associated criteria for each model, and the value of the overall desirability for each model is then displayed to the user via display generator 20. The user can review these results via display generator 20 for each model and choose a model or configure the system and UI to choose automatically the most desirable model for final deployment for the respective designated business process where the respective prediction model is to be applied. Details of the results, values of criteria, and desirability values can be displayed through display generator 20. Display generator 20 also allows users to manually select different variables or choose to modify the specific algorithms and analytic approaches used in the algorithmic model generator block 12.”; Paragraph 64, “Cost of data for deploying and scoring a trained algorithmic model can be considered. In some modeling and deployment scenarios, obtaining (measuring) the values for some predictor variables can be expensive. For example, when modeling customer data, specific household and demographic information may have to be purchased from data brokers; in manufacturing applications, certain measurements may require destructive or otherwise costly testing. Using this metric, the cost of acquiring data for specific predictors both for training (re-calibration) and for scoring new data points can be considered.”; Paragraph 90, “a system for generating algorithmic models used for generating predictive analytics from a model data set for a process, the system comprising: a desirability function module configured to generate a desirability function, wherein the desirability function defines: at least one outcome variable and outcome variable type and at least one predictor variable and at least one predictor variable type; and at least one algorithmic model accuracy criterion, at least one model analytics type, at least one evaluation criterion for algorithmic model quality, and at least one evaluation criterion for model deployment cost; an automated machine learning module configured to: generate at least one algorithmic model having a variable set selected according to the desirability function; and train the at least one algorithmic model against the model data set; and a UI (User Interface) module configured to generate a user interface to display the at least one algorithmic model accuracy criterion, the at least one model analytics type, the at least one evaluation criterion for algorithmic model quality, and the at least one evaluation criterion for model deployment cost; wherein the displayed criteria and cost are selectable and definable;”)
Morimura discloses a method for determining optimal actions along a series of steps for a process. Yellapragada discloses a method for determining an optimal prediction model for making informed decisions. Takigawa discloses a method for training a model to make decisions for predicted action items. At the time of Applicant’s filed invention one of ordinary skill in the art would have deemed it obvious to combine the methods of Morimura with the teachings of Yellapragada in order to improve the model for determining optimal decisions as disclosed by Yellapragada (Yellapragada: Paragraph 30, “This in turn will significantly shorten the time from defining a business problem that can be addressed using Machine Learning and AI, to the successful and continuously sustainable deployment of ML/AI-based solutions given current business constraints.”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Philip N Warner whose telephone number is (571)270-7407. The examiner can normally be reached Monday-Friday 7am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O’Connor can be reached at 571-272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Philip N Warner/Examiner, Art Unit 3624
/Jerry O'Connor/Supervisory Patent Examiner,Group Art Unit 3624
Prosecution Insights