CTNF 18/486,344 CTNF 89242 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Objections 07-29-01 AIA Claim s 12-15, 21-22, 24 are objected to because of the following informalities: In Claim 12, line 3, “an expert” was probably meant to be: the expert. The same objection is made for Claim 14. In Claim 13, line 3, “a non-expert” was probably meant to be: the non-expert. The same objection is made for Claim 15. In Claim 21, lines 2-3, “a causal structure model” was probably meant to be: the causal structure model. The same objection is made for Claim 22. In Claim 24, line 4, “a data set” was probably meant to be: the data set . Appropriate correction is required. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. 07-34-01 Claims 2, 21, 24-25 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 2, line 2, recites the limitation “the first data sample” that lacks antecedent basis (perhaps this should be recited in the independent claim, see other independent claims). The same rejection is made for Claim 21 (see lines 4-5). Claim 24, line 7, recites the limitation “the first data” that lacks antecedent basis (was this meant to be: the first data sample ). The same rejection is made for Claim 25 (see lines 7-8). 07-30-03-h AIA Claim Interpretation 07-30-03 AIA The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph: An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. 07-30-06 This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a collection apparatus configured to collect a data set consisting of a plurality of events”; and “a training apparatus configured to train, based on the first data, a causal structure model”; and “an inference apparatus for inferring a data sample of a time step”; in Claim 25. Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-25 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Step 1 : All claims are directed towards either a method, a system or an apparatus and thus satisfies Step 1 as falling into one of the statutory categories. Step 2A, Prong One : Independent Claim 1 recites (the same analysis applies to the similar limitations of independent Claims 24 and 25): for inferring a causal relationship relating to the plurality of events . this limitation, under its broadest reasonable interpretation, covers concepts that can be performed in the human mind and therefore would fall under the “Mental Processes” groupings of abstract ideas. That is a person can infer causal relationships in data relating to a plurality of events by using observation and evaluation. Step 2A, Prong Two : Claim 1 recites the additional elements of (the same analysis applies to the similar limitations of independent Claims 24 and 25): acquire a data set consisting of a plurality of events, the data set including first data that includes first action data relating to an expert ; this limitation is considered as adding insignificant extra-solution activity (acquiring data) to the judicial exception - see MPEP 2106.05(g). and train, based on the first data, a causal structure model this limitation is considered as merely using a model as a tool to perform an abstract idea, which includes its training - see MPEP 2106.05(f). The last limitation of independent Claim 25 is also considered as using the model as a tool to perform the abstract idea (that is, performing inference) - see MPEP 2106.05(f). The further additional element of “processing circuitry” as recited in independent Claim 1 is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims are therefore directed to an abstract idea. Step 2B : The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements are considered as simply appending well-understood, routine, conventional activities previously known to the industry (data gathering), specified at a high level of generality, to the judicial exception - see MPEP 2106.05(d); and using the model as a tool to perform the abstract idea - see MPEP 2106.05(f). The further additional element of “processing circuitry” as recited in independent Claim 1 amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims are therefore not patent eligible. Dependent Claims 2-4 are considered as adding insignificant extra-solution activity (under Step 2A, Prong 2) to the judicial exception - see MPEP 2106.05(g), and subsequently as appending well-understood, routine, conventional activities previously known to the industry (under Step 2B, part of the data gathering), specified at a high level of generality, to the judicial exception - see MPEP 2106.05(d). Dependent Claim 5 is considered as merely using a model as a tool to perform an abstract idea, which includes its training - see MPEP 2106.05(f). Dependent Claim 6, first limitation is considered as adding insignificant extra-solution activity (under Step 2A, Prong 2) to the judicial exception - see MPEP 2106.05(g), and subsequently as appending well-understood, routine, conventional activities previously known to the industry (under Step 2B, part of the data gathering), specified at a high level of generality, to the judicial exception - see MPEP 2106.05(d); the second limitation is considered as falling under the “Mental Processes” groupings of abstract ideas. That is a person can determine differences between data by using observation and evaluation. The last limitation is considered as using a model as a tool to perform an abstract idea, which includes its updating - see MPEP 2106.05(f). Dependent Claims 7 and 9, first limitations are considered as falling under the “Mental Processes” groupings of abstract ideas. That is a person can tag or “reward” data by using observation and evaluation. The last limitation is considered as using a model as a tool to perform an abstract idea, which includes its updating - see MPEP 2106.05(f). Dependent Claim 8 is considered as using a model as a tool to perform an abstract idea, which includes its updating - see MPEP 2106.05(f). Dependent Claim 10, is considered as falling under the “Mental Processes” groupings of abstract ideas. That is a person can determine rewards using a function and with pen and paper. Dependent Claims 11, 14-15, 17, 21-22 are considered as merely using a model as a tool to perform an abstract idea, which includes its training - see MPEP 2106.05(f). Dependent Claims 12-13, 16, 18-19 are considered as adding insignificant extra-solution activity (under Step 2A, Prong 2) to the judicial exception - see MPEP 2106.05(g), and subsequently as appending well-understood, routine, conventional activities previously known to the industry (under Step 2B, part of the data gathering), specified at a high level of generality, to the judicial exception - see MPEP 2106.05(d). Claims 20, 23 are considered as linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h). Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15-03-aia AIA Claim s 1-8, 12-15, 17-19, 21-25 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Yu , US 2023/0080424 A1 . Regarding Claim 1 , Yu teaches: A medical learning apparatus comprising processing circuitry configured to (paragraphs 26, 82): acquire a data set consisting of a plurality of events, the data set including first data that includes first action data relating to an expert (paragraph 4: “obtaining, via an acquisition component, demonstrations of a target task from experts for training a model”. The demonstrations of a target task from experts representative of data that includes action data from an expert. Examiner’s note: see also Fung, US 2011/0202486 A1, for example paragraph 29, where healthcare data of a patient, including a doctor/expert notes are acquired); and train, based on the first data, a causal structure model for inferring a causal relationship relating to the plurality of events (paragraph 4: “obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state, encoding, via a causal encoding component, discovered causal relationships by updating state variable embeddings”. The dynamic causal graphs being the causal structure model. Examiner’s note: see also the applicant’s NPL of Zhu, for example Abstract). Regarding Claim 2 , Yu further teaches: The medical learning apparatus of claim 1, wherein the first data sample includes first attribute data corresponding to the first action data (paragraph 19: “the exemplary methods aim to learn a self-explainable imitator by discovering the casual relationship between states and actions. In other words, taking observable state variables and candidate actions as nodes, the neural agent can generate a DAG to depict the underlying dependency between states and actions, with edges representing causal relationships. For example, in the medical domain, the obtained DAG can include relations”. The relations being the attribute data). Regarding Claim 3 , Yu further teaches: The medical learning apparatus of claim 1, wherein the data set further includes a second data sample that includes second action data relating to a non-expert (paragraph 26: “in the healthcare domain, the sequential medical treatment history of a patient is one expert demonstration. State variables include health records and symptoms, and actions are the usage of treatments. Relationships between symptoms and treatments could vary when patients are in different health conditions. Given a patient and the health states, the exemplary method needs to identify the current causal dependency between symptoms and actions taken by the imitation learning agent”. The imitation learning agent being the non-expert). Regarding Claim 4 , Yu further teaches: The medical learning apparatus of claim 3, wherein the second data sample includes second attribute data corresponding to the second action data (paragraph 26: “State variables include health records and symptoms”. The symptoms being the attribute data). Regarding Claim 5 , Yu further teaches: The medical learning apparatus of claim 1, wherein the processing circuitry trains the causal structure model based on an evaluation function relating to near-optimality relating to the first action data (paragraph 47: “The sparsity regularizer applies the L1 norm on the causal graph templates to encourage sparsity of discovered causal relations so that those non-causal edges could be removed”. The L1 norm representative of the evaluation function). Regarding Claim 6 , Yu further teaches: The medical learning apparatus of claim 5, wherein the data set further includes a second data sample that includes second action data relating to a non-expert (paragraph 26: “in the healthcare domain, the sequential medical treatment history of a patient is one expert demonstration. State variables include health records and symptoms, and actions are the usage of treatments. Relationships between symptoms and treatments could vary when patients are in different health conditions. Given a patient and the health states, the exemplary method needs to identify the current causal dependency between symptoms and actions taken by the imitation learning agent”. The imitation learning agent being the non-expert), the evaluation function is a first evaluation function relating to a difference between the first action data and the second action data (paragraph 39: “The dynamic causal discovery for an imitation learning method (510) includes a learning unit/component, a causal discovery unit/component, and a causal encoding unit/component. The learning component computes the action to be taken with respect to states (512). The policy is updated based on the imitation loss learning”. The loss learning representative of the difference between the first action data and the second action data), and the processing circuitry updates a parameter of the causal structure model in such a manner that the first evaluation function is maximized (paragraph 39: “The causal graph structure is updated based on regularization terms and policy performance in imitation learning”). Regarding Claim 7 , Yu further teaches: The medical learning apparatus of claim 5, wherein the evaluation function is a second evaluation function relating to a reward given to the first action data, and the processing circuitry updates a parameter of the causal structure model based on the second evaluation function (paragraph 17: “Deep Reinforcement Learning (DRL) methods, that is, low sampling efficiency and reward sparsity. Following demonstrations that return near-optimal rewards, an imitator can prevent a vast number of unreasonable attempts during explorations and has been shown to be promising in many real-world applications”; And paragraph 36: “the causal encoding module/component is designed to update representations of state variables based on the discovered causal graph”). Regarding Claim 8 , Yu further teaches: The medical learning apparatus of claim 7, wherein the processing circuitry updates a parameter of the causal structure model in such a manner that the second evaluation function is maximized (paragraph 40: “Traditionally, deep reinforcement learning dedicates to learn a policy model… to select actions given states… which can maximize long-term rewards”. Maximizing the rewards maximizes the evaluation function of the reinforcement learning). Regarding Claim 12 , Yu further teaches: The medical learning apparatus of claim 1, wherein the first action data includes data generated based on a policy function of an expert (paragraph 4: “The method includes obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy”; And paragraph 25: “The agent model interacts with the environment by taking actions following its learnt policy”). Regarding Claim 13 , Yu further teaches: The medical learning apparatus of claim 3, wherein the second action data includes data generated based on a policy function of a non-expert (paragraph 67: “The proposed policy model is adversarially trained with a discriminator D to imitate expert decisions”. The discriminator representative of the non-expert). Regarding Claim 14 , Yu further teaches: The medical learning apparatus of claim 12, wherein the policy function of an expert is trained through reinforcement learning or imitation learning (paragraph 3: “Imitation learning, which learns agent policy by mimicking expert demonstration, has shown promising results in many applications such as medical treatment regimens and self-driving vehicles”). Regarding Claim 15 , Yu further teaches: The medical learning apparatus of claim 13, wherein the policy function of a non-expert is trained through reinforcement learning or imitation learning (paragraph 3: “Imitation learning, which learns agent policy by mimicking expert demonstration, has shown promising results in many applications such as medical treatment regimens and self-driving vehicles”). Regarding Claim 17 , Yu further teaches: The medical learning apparatus of claim 1, wherein the processing circuitry trains the causal structure model further based on an evaluation function relating to causal identifiability conditions (paragraph 80: “the exemplary methods could discover the causal relations among states and action variables by being trained together with the imitation learning agent and making the agent be dependent upon discovered causal edges. The exemplary methods propose a dynamic causal relation discovery module/component with a latent causal graph template set. It can both model different causal graphs for different environment states and provide similar causal graph for similar states. The exemplary methods further propose a causal encoding module/component so that discovered causal edges can be encoded into state embeddings, and the quality of discovered causal relations can be improved using gradients from the agent model. The exemplary methods further use a set of regularization terms to further improve the quality of obtained causal graphs, like sparsity constraint and acyclicity constraint. This feature enables it to obtain more realistic causal graphs”. The regularization terms and constraints representative of the evaluation function relating to causal identifiability conditions used in obtaining a better quality causal structure model). Regarding Claim 18 , Yu further teaches: The medical learning apparatus of claim 17, wherein the evaluation function relating to the causal identifiability conditions is at least one of a regression error of data generated from a causal structure, restriction conditions for generating a directed acyclic graph, and a regularization term relating to a complexity of a graph structure or a neural network (paragraph 80: “the exemplary methods could discover the causal relations among states and action variables by being trained together with the imitation learning agent and making the agent be dependent upon discovered causal edges. The exemplary methods propose a dynamic causal relation discovery module/component with a latent causal graph template set. It can both model different causal graphs for different environment states and provide similar causal graph for similar states. The exemplary methods further propose a causal encoding module/component so that discovered causal edges can be encoded into state embeddings, and the quality of discovered causal relations can be improved using gradients from the agent model. The exemplary methods further use a set of regularization terms to further improve the quality of obtained causal graphs, like sparsity constraint and acyclicity constraint. This feature enables it to obtain more realistic causal graphs”). Regarding Claim 19 , Yu further teaches: The medical learning apparatus of claim 17, wherein the evaluation function relating to the causal identifiability conditions is at least one of a conditional reference or an information criterion (paragraph 18: “some efforts compute saliency maps to highlight critical features using gradient information”; And paragraph 80: “The exemplary methods further propose a causal encoding module/component so that discovered causal edges can be encoded into state embeddings, and the quality of discovered causal relations can be improved using gradients from the agent model”. The gradients representative of an information criterion). Regarding Claim 21 , Yu further teaches: The medical learning apparatus of claim 1, wherein the processing circuitry trains a causal structure model for inferring a causal relationship relating to an event, except for the first action data included in the first data sample (Abstract: “obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state”; And, paragraph 21: “A causal discovery module or component is designed to uncover causal relations among variables, and extracted causes are encoded into the embedding of outcome variables before action prediction”; And paragraph 23: “The main contributions are at least studying a novel problem of learning dynamic causal graphs to uncover knowledge captured, as well as latent causes behind agent's decisions, introducing a novel framework called GAIL, which is able to learn dynamic DAGs to capture the casual relation between state variables and actions, and adopt the DAGs for decision making in imitation learning”. That is after the causal graph is learned on first data it is used to make predictions/inferences on new data). Regarding Claim 22 , Yu further teaches: The medical learning apparatus of claim 2, wherein the processing circuitry trains a causal structure model for inferring a causal relationship relating to the first attribute data (Abstract: “obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state”). Regarding Claim 23 , Yu further teaches: The medical learning apparatus of claim 1, wherein the causal structure model is at least one of a skeleton, a directed graph, a partially directed acyclic graph, a directed acyclic graph, or a topological order (paragraph 19: “the exemplary methods aim to learn a self-explainable imitator by discovering the casual relationship between states and actions. In other words, taking observable state variables and candidate actions as nodes, the neural agent can generate a DAG to depict the underlying dependency between states and actions, with edges representing causal relationships”. DAG being a directed acyclic graph) . Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Yu , US 2023/0080424 A1, in view of Yoon , US 2022/0405682 A1 . Regrading Claim 9 , with Yu teaching those limitations of the claim as previously pointed out, Yu may not have taught all of the following, however, Yoon shows: The medical learning apparatus of claim 7, wherein the second evaluation function further includes a distribution of the reward, and the processing circuitry updates the causal structure model in such a manner that a difference between the distribution of the reward and a target distribution of a reward becomes small (paragraph 12: “the reward network generation unit may initialize the weight of the reward network and the weight of the policy agent using a Gaussian distribution and generate the reward network and the policy agent through an iterative learning process”; And, paragraph 49: “the reward network generation unit 110 may acquire a distributional difference between rewards on the basis of a first reward for the first trajectory acquired through the reward network and a second reward for the second trajectory acquired through the reward network and may update the weight of the reward network. For example, the reward network generation unit 110 may acquire the distributional difference between rewards on the basis of the first reward and the second reward through the evidence of lower bound (ELBO) algorithm and may update the weight of the reward network. That is, the ELBO may be calculated through a method of calculating a distributional difference in distribution called Kullback-Leibler (KL) divergence. The ELBO theory explains that the method of minimizing divergence is a method of increasing the lower bound of the distribution and that it is possible to ultimately reduce the distribution gap by increasing the minimum value”. Reducing the distribution gap being equivalent to making the difference between the distribution of the reward and a target distribution of a reward becoming small). It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the teachings of Yoon with that of Yu for including a distribution of the reward, and updating the causal structure model in such a manner that a difference between the distribution of the reward and a target distribution of a reward becomes small. The ordinary artisan would have been motivated to modify Yu in the manner set forth above for the purposes of using distribution of the reward such that the action and the state of a policy agent are continuous values, not discrete values in statistical theory [Yoon: paragraph 49]. Regarding Claim 10 , Yoon further teaches: The medical learning apparatus of claim 7, wherein the reward is determined based on a reward function (paragraph 54: “the present invention imitates the action characteristics of a motorcycle delivery worker through a reinforcement learning policy agent configured using an artificial neural network, and an inverse reinforcement learning reward network (i.e., a reward function) configured using an artificial neural network modeling a distributional difference between an action pattern imitated by the policy agent and an actual action pattern of the motorcycle delivery worker (i.e., expert) and assigns a reward to the policy agent”). Regarding Claim 11 , Yoon further teaches: The medical learning apparatus of claim 10, wherein the reward function is trained by inverse reinforcement learning (paragraph 54: “the present invention imitates the action characteristics of a motorcycle delivery worker through a reinforcement learning policy agent configured using an artificial neural network, and an inverse reinforcement learning reward network (i.e., a reward function) configured using an artificial neural network modeling a distributional difference between an action pattern imitated by the policy agent and an actual action pattern of the motorcycle delivery worker (i.e., expert) and assigns a reward to the policy agent”; And, paragraph 87: “the present invention infers the distribution of rewards for given expert trajectories of motorcycle delivery workers. To ensure that the reward function according to the present invention is trained from the actions of the motorcycle delivery worker in order to distinguish between a non-abuser action that uses a vehicle normally and other actions of an abuser who uses a motorcycle, it is important that the training set should not contain latent abusers”) . 07-21-aia AIA Claim s 16, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yu , US 2023/0080424 A1, in view of Dalli , US 2022/0147876 A1 . Regrading Claim 16 , with Yu teaching those limitations of the claim as previously pointed out, Yu may not have taught all of the following, however, Dalli shows: The medical learning apparatus of claim 1, wherein the data set includes a third data set generated by a world model (paragraph 87: “An exemplary XRL may include explanations x as part of the model/environment. The world model 4500 can give back a partial or full explanation about the state”. The explanations representative of data generated by the world model). It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the teachings of Dalli with that of Yu for having the data set include a third data set generated by a world model. The ordinary artisan would have been motivated to modify Yu in the manner set forth above for the purposes of having causal models to explain the world around them [Dalli: paragraph 35]. Regarding Claim 20 , Dalli further teaches: The medical learning apparatus of claim 1, wherein the causal structure model is a world model (paragraph 35: “This method builds on the advantage of the prominent theory that humans develop and deploy causal models to explain the world around them and have adapted a structural causal model (SCM) to mimic this for a model-free RL system”; And, paragraph 42: “XRL introduces the concept of explanations as part of the RL agent model and optionally the world/environment model”; And paragraph 87: “An exemplary XRL may include explanations x as part of the model/environment. The world model 4500 can give back a partial or full explanation about the state”). Claim 24 is similar to Claim 1 and is rejected under the same rationale as stated above for that claim. Claim 25 is similar to Claim 1 (the last limitation also being taught by Yu, see for example paragraphs 42-43) and is rejected under the same rationale as stated above for that claim. Examiner's Note : The Examiner cites particular pages, sections, columns, line numbers, and/or paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in its entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner and the additional related prior arts made of record that are considered pertinent to applicant's disclosure to further show the general state of the art. The Examiner's interpretations in parenthesis are provided with the cited references to assist the applicants to better understand how the examiner interprets the prior art to read on the claims. Such comments are entirely consistent with the intent and spirit of compact prosecution . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892 and applicant’s IDS for the relevant prior art where for example the applicant’s NPL of Zhu teaches learning with causal structured models . Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVE MISIR whose telephone number is (571)272-5243. The examiner can normally be reached M-R 8-5 pm, F some hours. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at 5712703169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /DAVE MISIR/Primary Examiner, Art Unit 2127 Application/Control Number: 18/486,344 Page 2 Art Unit: 2127 Application/Control Number: 18/486,344 Page 3 Art Unit: 2127 Application/Control Number: 18/486,344 Page 4 Art Unit: 2127 Application/Control Number: 18/486,344 Page 5 Art Unit: 2127 Application/Control Number: 18/486,344 Page 6 Art Unit: 2127 Application/Control Number: 18/486,344 Page 7 Art Unit: 2127 Application/Control Number: 18/486,344 Page 8 Art Unit: 2127 Application/Control Number: 18/486,344 Page 9 Art Unit: 2127 Application/Control Number: 18/486,344 Page 10 Art Unit: 2127 Application/Control Number: 18/486,344 Page 11 Art Unit: 2127 Application/Control Number: 18/486,344 Page 12 Art Unit: 2127 Application/Control Number: 18/486,344 Page 13 Art Unit: 2127 Application/Control Number: 18/486,344 Page 14 Art Unit: 2127 Application/Control Number: 18/486,344 Page 15 Art Unit: 2127 Application/Control Number: 18/486,344 Page 16 Art Unit: 2127 Application/Control Number: 18/486,344 Page 17 Art Unit: 2127 Application/Control Number: 18/486,344 Page 18 Art Unit: 2127 Application/Control Number: 18/486,344 Page 19 Art Unit: 2127 Application/Control Number: 18/486,344 Page 20 Art Unit: 2127 Application/Control Number: 18/486,344 Page 21 Art Unit: 2127 Application/Control Number: 18/486,344 Page 22 Art Unit: 2127 Application/Control Number: 18/486,344 Page 23 Art Unit: 2127 Application/Control Number: 18/486,344 Page 24 Art Unit: 2127 Application/Control Number: 18/486,344 Page 25 Art Unit: 2127
Prosecution Insights