Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-2, 4, 6, 8-9, 11, 13, 15-16, 18, and 20 have been amended. Claims 1-20 are currently pending and have been considered by the Examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“an action prediction component” in claim 16, line 2 (the limitation is equivalent to “an action prediction component for conducting an imitation learning task…”).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In claim 1, line 5, the limitation “training the model the learning component computing actions” renders the claim indefinite. This limitation is not grammatically correct, and there is insufficient antecedent basis for the limitation “the learning component” in the claim. Examiner treats the limitation in lines 5-6 to mean “training the model by computing actions to be taken with respect to states;”
Claim 1 recites the limitation "the state variable embeddings" in line 11. There is insufficient antecedent basis for this limitation in the claim, and it is unclear if any step in lines 1-9 have generated state variable embeddings. Examiner treats “the state variable embeddings” as “state variables”.
The term “similar” in claim 1, line 13 is a relative term which renders the claim indefinite. The term “similar” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The term “similar” is an approximation under MPEP 2173.05(b) subsection III. (C). How would a person ascertain a trajectory similar to expert demonstrations from a trajectory that is dissimilar from expert demonstrations? At what point does a similar trajectory become a dissimilar trajectory? Examiner treats “outputting… the learned policy including trajectories similar to the demonstrations from the experts” as “outputting… the learned policy including trajectories based on the demonstrations from the experts”.
Claims 2-7 are rejected for failing to cure the deficiencies of claim 1.
The term “similar” in claim 7, line 3 is a relative term which renders the claim indefinite. The term “similar” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The term “similar” is an approximation under MPEP 2173.05(b) subsection III. (C). Paragraph 0060 explains that time steps are similar if they have been clustered into the same group, but Claim 7 does not positively recite clustering time steps. It is unclear if a step of selecting templates across similar time steps happens, since the limitation “enable consistency in template selection across similar time steps” is phrased as an intended use. Examiner treats “similar time steps” merely as “time steps”.
Claims 8 and 14 are rejected because they recite the same indefinite limitations as claims 1 and 7, respectively. Claims 9-14 are rejected for failing to cure the deficiencies of claim 8.
Claim 15 is rejected because it recites the same indefinite limitations as claim 1. Claims 16-20 are rejected for failing to cure the deficiencies of claim 15.
Claim limitation “an action prediction component” in claim 16, line 2 invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. Figure 2 and specification paragraph [0029] disclose representations of these components, but the written disclosure does not clearly disclose physical structures of these components. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-7 recite a method, claims 8-14 recite a non-transitory computer-readable storage medium (a product), and claims 15-20 recite a system comprising a processor (a system). Each of a method, a product, and a system falls within one of the four statutory categories of patent eligible subject matter.
Claim 1
Step 2A Prong 1: Discovering causal relationships between states and actions is a judgment and evaluation mental process which can reasonably be performed in the human mind with the aid of pencil and paper.
Computing actions to be taken with respect to states is a judgment and evaluation mental process which can reasonably be performed in the human mind with the aid of pencil and paper. Computing actions is also a mathematical calculation. Specification paragraphs [0042]-[0043] discloses computing an action prediction at.
Generating dynamic causal graphs for each environment state in the learned policy, wherein each dynamic causal graph is a continuous directed acyclic graph over state variables and actions and is generated prior to action prediction is a judgment and evaluation mental process which can reasonably be performed in the human mind with the aid of pencil and paper. Specification paragraph [0027] discloses that Fig. 1 depicts a causal graph in the top right section of the figure. A person can draw the causal graph on a piece of paper.
Encoding discovered causal relationships by updating the state variable embeddings is a judgment and evaluation mental process which can reasonably be performed in the human mind with the aid of pencil and paper. A person can reasonably discover causal relationships using his or her mind and can update state variable embeddings on a piece of paper.
Additionally, encoding discovered causal relationships by updating the state variable embeddings is a mathematical calculation. Specification paragraphs [0053]-[0055] and [0062]-[0065] include equations (4) and (8) which encode a trajectory of states and generates an embedding of state variables.
Determining edge strengths in the dynamic causal graph by a Granger causality objective is a mathematical concept. Specification paragraph [0021] discloses Granger causality.
The action prediction is computed from the state variable embeddings updated according to the dynamic causal graph is a mathematical calculation. Specification paragraphs [0042]-[0043] discloses computing an action prediction at. The claim recites an abstract idea.
Step 2A Prong 2: Learning a self-explainable imitator amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Obtaining demonstrations of a target task from experts for training a model to generate a learned policy amounts to mere data-gathering, an insignificant extra-solution activity under MPEP 2106.05(g).
Training a model to generate a learned policy amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Examiner treats the limitation in lines 5-6 as “training the model by computing actions to be taken with respect to states”. This limitation amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Outputting the learned policy including trajectories similar to the demonstrations from the experts amounts to an insignificant extra-solution activity under MPEP 2106.05(g).
The additional elements as disclosed above, alone or in combination, do not integrate the abstract ideas into a practical application as they are mere insignificant extra-solution activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is directed to an abstract idea.
Step 2B: Learning a self-explainable imitator amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Obtaining demonstrations of a target task from experts for training a model to generate a learned policy is analogous to receiving data over a network, which the courts have recognized as a well-understood, routine, conventional activity under MPEP 2106.05(d)(II).
Training a model to generate a learned policy amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Examiner treats the limitation in lines 5-6 as “training the model by computing actions to be taken with respect to states”. This limitation amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Outputting the learned policy including trajectories similar to the demonstrations from the experts is analogous to presenting offers and gathering statistics, which the courts have recognized as a well-understood, routine, conventional activity under MPEP 2106.05(d)(II).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the abstract ideas as they are well-understood, routine and conventional activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is not patent eligible.
Claim 2 incorporates the rejection from claim 1.
Step 2A Prong 1: The abstract ideas from claim 1 are incorporated.
Step 2A Prong 2 and Step 2B: Conducting an imitation learning task and a state regression task, via an action prediction component, by employing the updated state variable embeddings as input evidence amount to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim 3 incorporates the rejection from claim 2.
Step 2A Prong 1: The abstract ideas from claim 1 are incorporated.
Step 2A Prong 2 and Step 2B: The state regression task is used to provide auxiliary signals for learning causal edges among state variables amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim 4 incorporates the rejection from claim 2.
Step 2A Prong 1: The abstract ideas from claim 1 are incorporated.
Step 2A Prong 2 and Step 2B: For the imitation learning task, the learned policy is implemented as a three-layer Multilayer Perceptron (MLP), with two layers shared between all branches of the learned policy, where the MLP conducts one prediction task amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim 5 incorporates the rejection from claim 1.
Step 2A Prong 1: The abstract ideas from claim 1 are incorporated.
Step 2A Prong 2 and Step 2B: The state variable embeddings are updated with propagated messages from variables it depends on by employing an edge-aware update layer amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim 6 incorporates the rejection from claim 1.
Step 2A Prong 1: The abstract ideas from claim 1 are incorporated. Constructing an explicit dictionary as Directed Acrylic Graph (DAG) templates, the DAG templates being randomly initialized is a judgement and evaluation mental process which can reasonably be performed in the human mind with the aid of pencil and paper. A person can reasonably construct DAG templates and randomly initialize them.
Step 2A Prong 2 and Step 2B: The dynamic causal discovery component amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim 7 incorporates the rejection from claim 1.
Step 2A Prong 1: The abstract ideas from claim 1 are incorporated. A template selection regularization loss is employed to enable consistency in template selection across similar time steps is a mathematical calculation. Specification paragraphs [0060]-[0061] disclose the loss function.
Step 2A Prong 2 and Step 2B: A sparsity constraint and an acyclicity constraint are employed to optimize the dynamic causal graphs amounts to mere instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim 8 recites a product which implements the same features as the method of claim 1 and is therefore rejected for at least the same reasons.
In Step 2A Prong 2 and Step 2B, a non-transitory computer-readable storage medium comprising a computer-readable program… wherein the computer-readable program when executed on a computer causes the computer to perform steps amounts to generic computer components and instructions for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claims 9-14 each recites a product which implements the same features as the methods of claims 2-7, respectively, and are therefore rejected for at least the same reasons.
Claim 15 recites a system which implements the same features as the method of claim 1 and is therefore rejected for at least the same reasons.
In Step 2A Prong 2 and Step 2B, a memory and one or more processors in communication with the memory amounts to generic computer components for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claims 16-20 each recites a system comprising a processor which implements the same features as the methods of claims 2-6, respectively, and are therefore rejected for at least the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5, 8-10, 12, 15-17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Haan et al. (“Causal Confusion in Imitation Learning”, cited in IDS filed 07/29/2022) in view of Pamfil et al. (“DYNOTEARS: Structure Learning from Time-Series Data”, cited in PTO-892 issued 12/29/2025), Theocharous et al. (US 20210089958 A1, cited in PTO-892 issued 12/29/2025), and Arnold et al. (“Temporal Causal Modeling with Graphical Granger Methods”).
Regarding claim 1, Haan teaches: A method for learning a self-explainable imitator by discovering causal relationships between states and actions, the method comprising: obtaining demonstrations of a target task from experts for training a model to generate a learned policy; (Page 2, § 3, lines 1-5; Page 4, lines 17-18; and Page 5, § 4.1, lines 7-13 discloses training a neural network fφ representing a policy. A model as claimed is a neural network.)
training the model the learning component computing actions to be taken with respect to states; (This limitation is treated as “training the model by computing actions to be taken with respect to states”. Page 5, § 4.1, lines 7-13 discloses training.)
generating i wherein the causal graph is a continuous directed acyclic graph. Examiner treats “action prediction” as predictions on validation data. The sentence on page 4, lines 20-21 indicates the causal graph G is generated before the trained policy predicts an action on held-out validation data.)
encoding discovered causal relationships by [using] k to an action A. The vector contains state variable embeddings, and trained policy encodes the discovered causal relationships.)
outputting the learned policy including trajectories similar to the demonstrations from the experts, wherein the action prediction is computed from the state variable embeddings [used] φ, which indicates the learned policy from § 4.1 has been output. With respect to trajectories, Page 3, lines 3-12 and Fig. 2 disclose a trajectory of state variables and actions across times t-1 and t. On page 5, § 4.1, lines 7-11 and Equation (1) teach that the policy network fφ computes an action prediction based on state variable embeddings contained in the parameterized vector of G. The sentence on page 4, lines 20-21 teaches predicting actions on held-out validation data.)
However, Haan does not explicitly teach (underlines indicate limitations not taught): generating dynamic causal graphs
encoding discovered causal relationships by updating the state variable embeddings, wherein edge strengths in the dynamic causal graph are determined by a Granger causality objective; and
But Pamfil teaches: generating dynamic causal graphs (Page 1, col. 2, lines 3-15; Page 2, Fig. 1 and its caption; and Page 2 § 2.1, lines 1-8 discloses generating a dynamic causal graph such as the one depicted in Fig. 1.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the dynamic feature of Pamfil’s causal graphs into Haan. A motivation for the combination is to understand a time series whose variables influence each other in both a contemporaneous and a time-lagged manner. (Pamfil, Page 2 § 2.1, lines 1-8)
However, Haan and Pamfil do not explicitly teach: encoding discovered causal relationships by updating the state variable embeddings, wherein edge strengths in the dynamic causal graph are determined by a Granger causality objective; and
But Theocharous teaches: encoding discovered causal relationships by updating state variable embeddings, ([0020], lines 1-11; [0028], lines 1-5; [0075]-[0076]; [0083] and [0084], lines 1 to “505” in line 4 discloses updating a state conditional function (β) which maps states to an embedding space. The causal relationships between states S and S’ via an action is encoded into the embedding space due to both the inverse dynamics function (φ) and state conditional function (β).)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have updated Haan’s embeddings similar to Theocharous. A motivation for the combination is to account for non-stationary conditions in a reinforcement learning model. (Theocharous, [0002]-[0003])
However, Haan, Pamfil, and Theocharous do not explicitly teach: wherein edge strengths in the dynamic causal graph are determined by a Granger causality objective;
But Arnold teaches: wherein edge strengths in the
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have applied the regression of equation (3) to causal graphs in the combination of Haan, Pamfil, and Theocharous. A motivation for the combination is to eliminate edges between conditionally independent features in a causal graph. (Page 67, col. 2, § 2.1, lines 1-13)
Regarding claim 2, the combination of Haan, Pamfil, Theocharous, and Arnold teaches: The method of claim 1,
Haan teaches: further comprising conducting an imitation learning task
Haan teaches a linear regression during expert query intervention in Algorithm 1, line 14. However, Haan, Pamfil, and Arnold do not appear to explicitly teach: conducting a state regression task by employing the updated state variable embeddings as input evidence.
But Theocharous teaches: conducting a state regression task by employing the updated state variable embeddings as input evidence. (All [0020]; [0083], [0084], lines 1-2, [0088], lines 1-10 discloses updating a state conditional function (β) which maps states to an embedding space, and computing an updated policy function. The updated state variable embedding is evidence of learning.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated Theocharous’ loss function to compute an updated state conditional function and an updated policy function. A motivation for the combination is the same as the motivation given in claim 1. Note that the claim limitations are met as long as either of the imitation learning task and the state regression task employs the updated state variable embedding as evidence.
Regarding claim 3, the combination of Haan, Pamfil, Theocharous, and Arnold teaches: The method of claim 2,
However, Haan, Pamfil, and Arnold do not explicitly teach: wherein the state regression task is used to provide auxiliary signals for learning causal edges among state variables.
But Theocharous teaches: wherein the state regression task is used to provide auxiliary signals for learning causal edges among state variables. (All of [0072], [0075]-[0076] discloses training to accurately predict which action caused a transition from s to s’. See [0088], lines 1-10 for the state regression task as explained in claim 2.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have parameterized the overall policy into two components in the combination of Haan, Pamfil, Theocharous, and Arnold. A motivation for the combination is to teach the agent the cause for a transition between states. (Theocharous, [0075])
Regarding claim 5, the combination of Haan, Pamfil, Theocharous, and Arnold teaches: The method of claim 1,
However, Haan, Pamfil, and Arnold do not explicitly teach: wherein the state variable embeddings are updated with propagated messages from variables it depends on by employing an edge-aware update layer.
But Theocharous teaches: wherein the state variable embeddings are updated with propagated messages from variables it depends on by employing an edge-aware update layer. ([0019]-[0020], [0028], [0030] discloses that when the system identifies an additional set of actions, it may update the state conditional function. “Propagated messages” are updates to the state conditional function when new actions are identified. This also implies an “edge-aware update layer” which performs the updates, where edges represent transitions between states when the agent performs an action.)
A further motivation for incorporating Theocharous’ embedding space into the combination of references is to perform automatic sequential decision-making in a manner that adapts to new actions while taking advantage of learning that occurred using the previous action set. (Theocharous, [0019])
Claim 8 recites a product which implements the same features as the method of claim 1 and is therefore rejected for at least the same reasons over Haan in view of Pamfil, Theocharous, and Arnold.
However, Haan, Pamfil, and Arnold do not explicitly teach: A non-transitory computer-readable storage medium comprising a computer-readable program… , wherein the computer-readable program when executed on a computer causes the computer to perform steps.
But Theocharous teaches: A non-transitory computer-readable storage medium comprising a computer-readable program… , wherein the computer-readable program when executed on a computer causes the computer to perform steps. ([0101], lines 11-14)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated Theocharous’ memory and processor into the combination of Haan, Pamfil, Theocharous, and Arnold. A motivation for the combination is to execute the method on a computer.
Claims 9-10 and 12 each recites a product which implements the same features as the method of claims 2-3 and 5, respectively, and are therefore rejected for at least the same reasons over Haan in view of Pamfil, Theocharous, and Arnold.
Claim 15 recites a system which implements similar features as the method of claim 1 and is therefore rejected for at least the same reasons over Haan in view of Pamfil, Theocharous, and Arnold.
However, Haan, Pamfil, and Arnold do not explicitly teach: a system comprising: a memory; and one or more processors in communication with the memory.
But Theocharous teaches: a system comprising: a memory; and one or more processors in communication with the memory. (All limitations are taught by [0101], lines 11-14)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated Theocharous’ memory and processor into the combination of Haan, Pamfil, Theocharous, and Arnold. A motivation for the combination is to execute the method on a computer.
Claims 16-17 and 19 each recites a system which implements the same features as the method of claims 2-3 and 5, respectively, and are therefore rejected for at least the same reasons over Haan in view of Pamfil, Theocharous, and Arnold.
Claims 4, 11, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Haan et al. (“Causal Confusion in Imitation Learning”, cited in IDS filed 07/29/2022) in view of Pamfil et al. (“DYNOTEARS: Structure Learning from Time-Series Data”, cited in PTO-892 issued 12/29/2025), Theocharous et al. (US 20210089958 A1, cited in PTO-892 issued 12/29/2025), Arnold et al. (“Temporal Causal Modeling with Graphical Granger Methods”), and Asaoka et al. (US 20180281256 A1, cited in PTO-892 issued 12/29/2025).
Regarding claim 4, the combination of Haan, Pamfil, Theocharous, and Arnold teaches: The method of claim 2,
Haan teaches: wherein, for the imitation learning task, the learned policy is implemented as a
Theocharous, [0108], lines 6-12 teaches that neural network nodes are aggregated into different layers including at least an input layer and an output layer. However, Haan, Pamfil, Theocharous, and Arnold do not explicitly teach: a three-layer Multilayer Perceptron (MLP), with two layers shared between all branches of the learned policy,
But Asaoka teaches: a three-layer Multilayer Perceptron (MLP), with two layers shared between all branches (Examiner treats “two layers shared between all branches” to mean every path starts at an input layer and ends at an output layer of the MLP. See [0063], lines 4-6 and [0065]-[0067])
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have used Asaoka’s three-layer neural network in the combination of Haan, Pamfil, Theocharous, and Arnold to represent Haan’s policy. A motivation for the combination is to extract hidden features from the input data.
Claim 11 recites a product which implements the same features as the method of claim 4 and is therefore rejected for at least the same reasons.
Claim 18 recites a system which implements the same features as the method of claim 4 and is therefore rejected for at least the same reasons.
Claims 6, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Haan et al. (“Causal Confusion in Imitation Learning”, cited in IDS filed 07/29/2022) in view of Pamfil et al. (“DYNOTEARS: Structure Learning from Time-Series Data”, cited in PTO-892 issued 12/29/2025), Theocharous et al. (US 20210089958 A1, cited in PTO-892 issued 12/29/2025), Arnold et al. (“Temporal Causal Modeling with Graphical Granger Methods”), and Wortsman et al. (US 20200380342 A1, cited in PTO-892 issued 12/29/2025).
Regarding claim 6, the combination of Haan, Pamfil, Theocharous, and Arnold teaches: The method of claim 1,
Haan teaches: wherein the dynamic causal discovery component includes construction of an explicit dictionary as Directed Acrylic Graph (DAG) templates, (Page 5, § 4.1, lines 2-7. Each of the 2n possible graphs is a template.)
However, Haan, Pamfil, Theocharous, and Arnold do not explicitly teach: the DAG templates being randomly initialized.
But Wortsman teaches: the DAG
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have randomly initialized edge weights for the DAG templates in the combination of Haan, Pamfil, Theocharous, and Arnold, similar to randomly initializing edge weights for Wortsman’s DAG. A motivation for the combination is that randomly initializing edge weights breaks symmetry and helps the model converge faster, as is known to a person having ordinary skill in the art.
Claim 13 recites a product which implements the same features as the method of claim 6 and is therefore rejected for at least the same reasons.
Claim 20 recites a system which implements the same features as the method of claim 6 and is therefore rejected for at least the same reasons.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Haan et al. (“Causal Confusion in Imitation Learning”, cited in IDS filed 07/29/2022) in view of Pamfil et al. (“DYNOTEARS: Structure Learning from Time-Series Data”, cited in PTO-892 issued 12/29/2025), Theocharous et al. (US 20210089958 A1, cited in PTO-892 issued 12/29/2025), Arnold et al. (“Temporal Causal Modeling with Graphical Granger Methods”), and Singer et al. (US 10438129 B1, cited in PTO-892 issued 12/29/2025).
Regarding claim 7, the combination of Haan, Pamfil, Theocharous, and Arnold teaches: The method of claim 1,
However, Haan, Theocharous, and Arnold do not explicitly teach: wherein a sparsity constraint and an acyclicity constraint are employed to optimize the dynamic causal graphs, and a template selection regularization loss is employed to enable consistency in template selection across similar time steps.
But Pamfil teaches: wherein a sparsity constraint and an acyclicity constraint are employed to optimize the dynamic causal graphs, and (Page 1, § 1, from line 5 to col. 2, line 11, and Page 3, § 2.2, from line 1 to the first line below equation (5) discloses optimizing a dynamic causal graph.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated a sparsity constraint and an acyclicity constraint into the combination of Haan, Pamfil, Theocharous, and Arnold. Motivations for the combination is to ensure that the whole network is acyclic and that regularization is especially useful in cases with much fewer samples than variables. (Pamfil, Page 3, col. 2, § 2.2, lines 4-6 and below equation (5), lines 1-3)
However, Haan, Pamfil, Theocharous, and Arnold do not explicitly teach: a template selection regularization loss is employed to enable consistency in template selection across similar time steps.
But Singer teaches: a template selection regularization loss is employed to enable consistency in template selection across similar time steps. (Examiner treats “similar time steps” as “time steps”. C. 1, L. 49-53; C. 4, L. 28-38; C. 5, L. 49-52 and L. 64 to C. 6, L. 4; and C. 8, L. 8-16 disclose a template selection regularization penalty. The penalty is maintained across a particular number of training iterations (time steps).)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated Singer’s regularization penalty into the combination of Haan, Pamfil, Theocharous, and Arnold. A motivation for the combination is that a regularization penalty may control model complexity and improve model generalization on inference data. (Singer, C. 5, L. 64 to C. 6, L. 4)
Claim 14 recites a product which implements the same features as the method of claim 7 and is therefore rejected for at least the same reasons.
Response to Arguments
The following is the Examiner’s response to the Applicant’s arguments filed 03/30/2026.
Applicant’s Arguments Under 35 U.S.C. 112(b): On page 9, the Applicant asserts that the term “similar” in claim 1 is not an approximation. It reflects how the learned policy has been trained to imitate expert decisions per specification paragraph 0071. On page 10, the Applicant asserts that the term “similar” in claim 7 is not a relative term. “Similar time steps” according to specification paragraph 0060 reflects time steps that belong in the same group based on a clustering result.
Examiner’s Response: Applicant's arguments have been fully considered but they are not persuasive. Besides the term “a self-explainable imitator” in the preamble, claim 1 makes no mention of the policy model being adversarially trained with a discriminator to imitate expert decisions, or the learned policy aiming to generate realistic trajectories that can mimic the expert policy to fool the discriminator. Paragraph 0071 describes an embodiment of adversarially training the proposed policy model with a discriminator D to imitate expert decisions, but the specification does not explain that a “similar” trajectory means a model policy that fools a discriminator. How would a person ascertain a trajectory that is similar to expert demonstrations from a trajectory that is dissimilar from expert demonstrations? What are characteristics of a similar trajectory and a dissimilar trajectory?
Regarding claim 7, paragraph 0060 explains that time steps are similar if they have been clustered into the same group. Claim 7 does not positively recite clustering time steps. It is unclear if a step of selecting templates across similar time steps happens, since the limitation “enable consistency in template selection across similar time steps” is phrased as an intended use.
Applicant’s Arguments Under 35 U.S.C. 101: On page 10, the Applicant asserts that the claims are integrated into a practical application because the claims amount to an improvement in the technological field of medical treatment. On page 11, the Applicant argues the claim language improves technology because it improves medical treatments by exposing causal relations in AI designed for medical treatment. This improves medical treatments because by seeing how the AI arrived at a prescribed medical treatment, an individual can trust the output by analyzing the AI’s work process. In other words, the claimed invention makes AI show the AI’s work so one can understand how the AI arrived at their conclusion. This allows one to verify if the AI is hallucinating or not when determining medical treatments. On page 12, the Applicant argues that the amended limitations in claim 1, lines 8-9, lines 12-13, and 15-16 provide improvements in technology.
Examiner’s Response: Applicant's arguments have been fully considered but they are not persuasive. It is noted that the features upon which applicant relies (i.e., “medical treatment”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The claims do not recite any limitations related to medical technology or healthcare. The claims do not recite an AI agent predicting medical treatments, or an AI agent building trust with a user by explaining how it arrived at conclusions.
In Step 2A Prong 1, the limitation “generating dynamic causal graphs for each environment state in the learned policy, wherein each dynamic causal graph is a continuous directed acyclic graph over state variables and actions and is generated prior to action prediction” is a judgment and evaluation mental process which can reasonably be performed in the human mind with the aid of pencil and paper. Specification paragraph [0027] discloses that Fig. 1 depicts a causal graph in the top right section of the figure. A person can draw the causal graph on a piece of paper.
The limitation “encoding discovered causal relationships by updating the state variable embeddings” is a judgment and evaluation mental process which can reasonably be performed in the human mind with the aid of pencil and paper. A person can reasonably discover causal relationships using his or her mind and can update state variable embeddings on a piece of paper.
Additionally, “encoding discovered causal relationships by updating the state variable embeddings” is a mathematical calculation. Specification paragraphs [0053]-[0055] and [0062]-[0065] include equations (4) and (8) which encode a trajectory of states and generates an embedding of state variables.
The limitation “edge strengths in the dynamic causal graph are determined by a Granger causality objective” is a mathematical concept. Specification paragraph [0021] discloses Granger causality.
The limitation “the action prediction is computed from the state variable embeddings updated according to the dynamic causal graph” is a mathematical calculation. Specification paragraphs [0042]-[0043] discloses computing an action prediction at.
MPEP 2106.05(a) states, “It is important to note, the judicial exception alone cannot provide the improvement.” MPEP 2106.05(a), II. states, “it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology.”
Applicant’s Arguments Under 35 U.S.C. 103: On page 14, the Applicant asserts that the references Haan, Pamfil, and Theocharous do not disclose or suggest at least the amended limitations in claim 1, lines 12-13 and 15-16.
Examiner’s Response: Applicant's arguments have been fully considered but they are not persuasive. Haan teaches “encoding discovered causal relationships by [using] the state variable embeddings”. Page 5, § 4.1, lines 2-16 discloses parameterizing the structure G of a causal graph as a vector of n binary variables, each indicating the presence of an arrow from a state Xk to an action A. The vector contains state variable embeddings, and trained policy encodes the discovered causal relationships.
Haan further teaches “outputting the learned policy including trajectories similar to the demonstrations from the experts, wherein the action prediction is computed from the state variable embeddings [used] according to the φ, which indicates the learned policy from § 4.1 has been output. With respect to trajectories, Page 3, lines 3-12 and Fig. 2 disclose a trajectory of state variables and actions across times t-1 and t. On page 5, § 4.1, lines 7-11 and Equation (1) teach that the policy network fφ computes an action prediction based on state variable embeddings contained in the parameterized vector of G. The sentence on page 4, lines 20-21 teaches predicting actions on held-out validation data.
However, Haan does not explicitly teach (underlines indicate limitations not taught): generating dynamic causal graphs, and encoding discovered causal relationships by updating the state variable embeddings
But Pamfil teaches “generating dynamic causal graphs”. Page 1, col. 2, lines 3-15; Page 2, Fig. 1 and its caption; and Page 2 § 2.1, lines 1-8 discloses generating a dynamic causal graph such as the one depicted in Fig. 1.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the dynamic feature of Pamfil’s causal graphs into Haan. A motivation for the combination is to understand a time series whose variables influence each other in both a contemporaneous and a time-lagged manner. (Pamfil, Page 2 § 2.1, lines 1-8)
However, Haan and Pamfil do not explicitly teach: encoding discovered causal relationships by updating the state variable embeddings,
But Theocharous teaches “encoding discovered causal relationships by updating state variable embeddings”. [0020], lines 1-11; [0028], lines 1-5; [0075]-[0076]; [0083] and [0084], lines 1 to “505” in line 4 discloses updating a state conditional function (β) which maps states to an embedding space. The causal relationships between states S and S’ via an action is encoded into the embedding space due to both the inverse dynamics function (φ) and state conditional function (β).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have updated Haan’s embeddings similar to Theocharous. A motivation for the combination is to account for non-stationary conditions in a reinforcement learning model. (Theocharous, [0002]-[0003])
Applicant’s arguments with respect to the claim 1 limitation “wherein edge strength in the dynamic causal graph are determined by a Granger causality objective” have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Asher H. Jablon whose telephone number is (571)270-7648. The examiner can normally be reached Monday - Friday, 9:00 am - 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.H.J./Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights