Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment and remarks filed April 2nd, 2026. In the amendment, claims 1, 6, 10, 15, and 19 were amended and no claims were cancelled or added. As such, claims 1-23 are pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on January 15th, 2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s argument, see Pages 10-13, filed April 2nd, 2026, with respect to 35 U.S.C. 101 has been fully considered and is persuasive. The specification objection has been withdrawn. The 35 U.S.C. 101 rejections of claims 1-9 and 19-23 and the specification objection have been withdrawn.
Applicant’s arguments with respect to the rejections of claims 1-23 under 35 U.S.C § 103 are
not persuasive for the following reasons:
35 U.S.C 103:
Applicant argues that the cited portions of Kimura, Ristoski, and Triplet fail to teach or suggest the limitation: “wherein the LOA computing framework comprises (1) an interface for entry of (i) at least one dataset comprising information relating to existing polymers and (ii) constraining rules comprising predicates and logical operations, that limit the score of the information in the at least one dataset”. Applicant argues that the references specifically fail to teach “using constraining rules comprising predicates and logical operations, that limit the scope of the information in the at least one dataset” (See Pages 10-12 of Remarks). Examiner respectfully disagrees.
Kimura discloses converting observations into first-order logic facts, which correspond to predicates (Section 3.2.1; Section 4, Fig. 1, e.g., <find x>, <visited x>, <initial x>). Kimura further teaches logical operations including conjunction, disjunction, and negation within its Logical Neural Network structure. Additionally, Kimura combines such predicates with logical operations to form rules that constrain decision-making. For example, the rule in Section 4 applies logical operators over predicates to restrict permissible actions of the agent based on observation data. Such rules limit the scope of information considered from the dataset.
Regarding the newly added “refinement” limitation (e.g., “wherein the decisions that advance the goal are used to refine the LNN policy rules”), Applicant added this wherein clause to independent claims 1, 15 and 19. Kimura teaches updating LNN policy rules based on reward signals. Specifically, Kimura discloses training the LNN by calculating a loss from reward values and optimizing network weights (Algorithm 1) which corresponds to refining policy rules based on decisions that advance a goal. Therefore, the 35 U.S.C. 103 rejection are maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4 are rejected under 35 U.S.C. 103 as being unpatentable over Kimura (“Neuro-Symbolic Reinforcement Learning with First-Order Logic”, 2021) in view of Ristoski (“US 20210303762 A1”).
Regarding claim 1,
Kimura teaches programming a reinforcement learning (RL) computing environment comprising an RL algorithm acting as a reward-based RL agent programmed to make decisions in furtherance of a goal… (Pg. 2 Paragraph 1 of Section 3.1, “As text-based games are sequential decision-making problems, they can naturally be applied to RL. These games are partially observable… where the observation text does not include the entire information of the environment… The objective for the agent is to maximize the expected discounted reward E”);
and integrating a logical optimal action (LOA) computing framework (neuro-symbolic framework) integrated with the RL computing environment, wherein the LOA computing framework comprises (Page 1 Under the Figure Caption of the Introduction, “In order to train logical rules, a recent neuro-symbolic framework called the Logical Neural Network (LNN) (Riegel et al., 2020) has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning). The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss.”, Page 2 Paragraph 2 of Introduction, “In this paper, we propose an action knowledge acquisition method featuring a neuro-symbolic LNN framework for the RL algorithm.”
The specification of the instant case states that the logical optimal action refers to neuro-symbolic LNN that is capable of optimizing the LNN. In this case, the reference teaches using a neuro-symbolic framework using Logical Neural Network integrated with the reinforced learning environment.)
(ii) constraining rules comprising predicates and logical operations, that limit the scope of the information in the at least one dataset (Page 5 Paragraph 1 Under Discussion about ethics, “The data set used in our experiment does not contain any sensitive information.”, Pg. 1 Under Figure 1 Caption in Introduction Section, “…the Logical Neural Network (LNN)… has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning). The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss. Every neuron in the LNN has a component for a formula of weighted real-valued logics from a unique logical conjunction, disjunction, or negation nodes… the trained LNN can extract obtained logical rules by selecting high weighted connections that represent the important rules for an action policy”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts…", Page 2 Introduction, “The method then converts first-order logical (predicates) facts from the propositional logics and categories of each word, such as ∃x ∈ {south, north},⟨find x⟩ = True and ∃x ∈ {east, west},⟨find x⟩ = False.”, Page 4 Section 4, “We attached the extracted rules from the medium level games here: ∃x ∈ W_direction (⟨find x⟩∧¬⟨visited x⟩ ∧¬⟨initial x⟩)∨ (⟨find x⟩∧⟨all are visited⟩ ∧ ⟨initial x⟩) → ⟪go x⟫,”
The "constraining rules comprising predicates and logical operations that limit the scope of the information in the at least one dataset" correspond to the symbolic logical rules used in the Logical Neural network (LNN). The FOL facts are explicitly predicates (e.g., find x, visited x, initial x) combined via logical operations (AND, OR, NOT) within the LNN’s neuron structure. The rules are constrained to logical predicate rules, thus limiting the scope of the information in the dataset based on the results of the FOL converter and the predicate logic of the LNN.),
and (2) a logical neural network (LNN) that is trained on the at least one dataset and the constraining rules and establishes LNN policy rules based on the training (Page 1 of Introduction Under Figure 1 Caption, “The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss… At the same time, the trained LNN can extract obtained logical rules by selecting high weighted connections that represent the important rules for an action policy.”, See Figure 1,
PNG
media_image1.png
418
423
media_image1.png
Greyscale
Here, the dataset corresponds to the observation in the Figure, the constraining rules correspond to the symbolic logical rules generated from the FOL converter, and the LNN policy rules corresponds to the output of the logical neural network which provides the rules used for the agent to make decisions. The observation and the rules are fed into the LNN to generate the policy rules for the agent to perform the actions.),
wherein the at least one dataset and the LNN policy rules are input into the RL computing environment where the RL agent makes decisions on the information in the at least one dataset that comply with the LNN policy rules and the RL agent is rewarded for decisions that advance the goal (Page 5 Paragraph 1 Under Discussion about ethics, “The data set used in our experiment does not contain any sensitive information.”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts… the agent understands an opened direction from the current room. The agent then retrieves the class type c of the word meaning in propositional logic l_i,t by using Concept Net… or the network of another word’s definition.”, Page 3 Paragraph 1 of Section 3.2.2, “The LNN training component is for obtaining an action policy from the given FOL logics.”, See Figure 1,
PNG
media_image1.png
418
423
media_image1.png
Greyscale
The reference teaches that observation data from the environment, in the form of natural language text, constitutes a dataset that is processed and converted into first-order logical facts and input into a Logical Neural Network (LNN), as shown in Figure 1 and described in Section 3.2. The LNN learns logical policy rules that are input into the reinforcement learning computing environment and directly governs the RL agent’s action selection. The RL agent makes decisions that comply with the LNN policy rules and is rewarded based on the outcome of those decisions.).
wherein the decisions that advance the goal are used to refine the LNN policy rules (Page 3 Section 3.2.2, “The LNN trains by this fact inputs and reward; that means it forwards from input facts through LNN, calculates a loss value from the reward value, and optimizes weights in LNN.”, See Algorithm 1 on Page 3,
PNG
media_image2.png
408
381
media_image2.png
Greyscale
Kimura teaches that goal-advancing decisions are stored in the replay buffer are used to update the LNN weights via gradient updates (line 14 of Algorithm 1). The reward signal, which corresponds to decisions that advance the goal of maximizing expected discounted reward, drives the optimization of the LNN weights, which in turn refines the policy rules extracted from the LNN. Specifically, Kimura shows that the reward from goal advancing decisions is used to update the LNN weights at line 14 of Algorithm 1. The LNN weights encode the logical connections that define the LNN policy rules. Thus, updating the weights based on goal advancing decisions directly refines those policy rules.).
Kimura does not teach a system processor set one or more computer-readable storage media: and program instructions stored on the one or more computer-readable storage media to cause the processor set to perform operations comprising: programming, an interface for entry of (i) at least one dataset comprising information relating to existing polymers, and a dataset relating to the discovery of new polymer materials.
Ristoski, in the same field of endeavor, teaches [a] system comprising a processor set one or more computer-readable storage media: and program instructions stored on the one or more computer-readable storage media to cause the processor set to perform operations comprising: (Paragraph 9 of Ristoski, “A system, in one aspect, can include a hardware processor and a memory device coupled with the hardware processor. The hardware processor can be configured to generate a set of material candidates expected to yield materials with target properties. The hardware processor can also be configured to receive subject matter expert's decision indicating accepted and rejected material candidates from the set of material candidates. The hardware processor can also be configured to, based on the subject matter expert's input, train a machine learning model to replicate the subject matter expert's decision.”, Paragraph 12, “A computer readable storage medium storing a program of instructions executable by a machine to perform one or more methods described herein also may be provided.”),
an interface for entry of (i) at least one dataset comprising information relating to existing polymers (Paragraph 0091, “For instance, at least one hardware processor 1102 may receive training data set which can include candidate material for polymerization, identify one or more desired features in the candidate material, and train a machine learning model to generate a new material having one or more of the desired features.”, Paragraph 0088, “In an embodiment, e.g., optionally, the identified desired features can be presented or caused to be presented on a user interface, e.g., for user interaction or view. In an embodiment, a user (e.g., an SME) may select from the desired features, a desired feature to include in the new material being generated.”
The reference teaches an interface or GUI that allows for the user to select desired features, which is considered an entry due to the fact that the user is entering constraints into the model.)
relating to the discovery of new polymer materials (Paragraph 0080, “FIG. 8 is a diagram illustrating a method of polymer discovery in one embodiment… At 802, the method can include generating a set of material candidates expected to yield materials with target properties. Examples of material candidates include, but not limited to, molecules, monomers, and/or polymer repeat units.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura’s teaching of integrating a neuro-symbolic Logical Neural Network (LNN) framework with a reinforcement learning environment for decision making with Ristoski’s teaching of providing a user interface for entering polymer related constraints in order to apply the reinforcement learning and LNN to the discovery of new polymer materials, thereby improving the efficiency and guidance of polymer discovery (Paragraph 0002 of Ristoski).
Regarding claim 2,
Kimura does not teach the constraining rules limit the scope of the information in the at least one dataset to available materials and laboratory equipment.
Ristoski, in the same field of endeavor, teaches the constraining rules limit the scope of the information in the at least one dataset to available materials and laboratory equipment (Paragraph 0042, “The general model tries to learn general rules for synthesizing new materials that are shared across all labs and SMEs, e.g., each monomer suitable for polymerization must contains a polymerizable group and should not contain pendant groups that will be active under polymerization conditions.”, Paragraph 0050, “By way of example only, a training data set to train or build the classification model 106 can include a random sample from a combinatorial library of polymerizable components suitable for the preparation of polyimides, including dianhydrides, dicarboxylic acids, and diamines.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify Kimura’s teaching of integrating a neuro-symbolic Logical Neural Network (LNN) framework with a reinforcement learning environment based on game rules by incorporating Ristoski’s teaching of constraining rules derived from polymer materials and laboratory equipment in order to improve practical applicability and reliability of the learned policies (Paragraph 0002 of Ristoski).
Regarding claim 3,
Kimura teaches the LOA (neuro-symbolic framework) further comprises (3) an internal regressor (FOL converter) that parses the at least one dataset into experiments and outcomes (Page 1 Under Figure 1 Caption, “In order to train logical rules, a recent neuro-symbolic framework called the Logical Neural Network (LNN)… has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning).”, Page 1 Figure 1 Caption, “…and the first-order logical facts are extracted from an FOL converter that uses a semantic parser, Concept Net, and history.” Page 3 Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts. The method first converts text into propositional logics li, t by a semantic parser from ot, such as, the agent understands an opened direction from the current room.”
The experiments correspond to the RL agent’s action executions in the environment based on the observation, and the outcomes correspond to the resulting state information and rewards from the logical facts generated by the FOL converter and LNN.),
wherein the LNN converts the experiments and outcomes into symbolic language understood by the RL agent (Figure 1 Caption, “The agent takes a text observation from the environment, and the first-order logical facts are extracted from an FOL converter that uses a semantic parser, Concept Net, and history… when the agent finds a direction x and the direction x has not been visited, the agent takes a “Go x” action. Dashed lines show the initial connections before training.”, See Figure 1,
PNG
media_image3.png
420
422
media_image3.png
Greyscale
).
Regarding claim 4,
Kimura teaches the LNN policy rules are updated to reflect decisions made by the RL agent that successfully advance the goal and the updated LNN policy rules direct future decisions by the RL agent to achieve the goal (Page 2 Section 3.1, “The objective for the agent is to maximize the expected discounted reward E…”, Page 3 Under Section 3.2.2, “The LNN trains by this fact inputs and reward; that means it forwards from input facts through LNN, calculates a loss values from the reward value, and optimizes weights in LNN.”, See Algorithm 1 RL by FOL-LNN,
PNG
media_image4.png
414
392
media_image4.png
Greyscale
The LNN uses the extracted logic from the FOL, and is used to train the Logic Neural Network in order to provide an output for the agent to take an action and get an action. The reward of the action that the agent takes is used to update the rules from the LNN in order to improve and optimize the agent’s future moves.)
Claims 5-23 are rejected under 35 U.S.C. 103 as being unpatentable over Kimura (“Neuro-Symbolic Reinforcement Learning with First-Order Logic”, 2021) in view of Ristoski (“US 20210303762 A1”) and in further view of Triplet (“US 20240007504 A1”).
Regarding claim 5,
Kimura does not teach a subject matter expert (SME) establishes the constraining rules and reviews the decisions by the RL agent to eliminate decisions that do not advance the goal.
Ristoski, in the same field of endeavor, teaches a subject matter expert (SME)… reviews the decisions by the RL agent to eliminate decisions that do not advance the goal (Paragraph 0002 of Ristoski, “Approaches have emerged, such as computational screening, inverse design, generative modeling, reinforcement learning as ways to accelerate the design of polymer materials. The drawback of these approaches is that they generate a large number of candidates for new molecules, which then need to be manually reviewed by subject matter experts who select only a dozen for further investigation.”, Paragraph 0063, “SeqGAN is a Sequence Generative Adversarial Network using reinforcement learning. The system includes generator and discriminator, where the generator is treated as reinforcement learning agent… the generated tokens represent the state and the action is the next token to be generated.”, Paragraph 0004, “Subject matter experts (SMEs) such as polymer chemists and synthetic organic chemists review many pages of candidates and select the ones that appear viable for further testing. With the very large number of candidates, and the fact that only a few will be selected for experiment manually”
The reinforcement learning agent generates material candidates and the outcome of the agent is used to have a SME review the decisions of the agent, and selects certain candidates for further testing).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura’s reinforcement learning system employing neuro-symbolic policy reasoning with Ristoski’s teaching of subject matter expert review of generated material candidates from the reinforcement learning agent in order to apply Ristoski’s SME to evaluate and eliminate agent decisions that do not advance the goal, thus improving the quality and relevance of selected outcomes (Paragraph 0032 of Ristoski).
Kimura in view Ristoski do not teach a subject matter expert (SME) establishes the constraining rules.
Triplet, in the same field of endeavor, teaches a subject matter expert (SME) establishes the constraining rules (Paragraph 0097 of Triplet, “The methods may detect compliance violations in real-time and predict remediation actions to non-compliant events, such as by using a) rules manually defined by SME to map each policy to a remediation template, b) collaborative filtering algorithms to predict which template can best remediate a given violation based on historical data, c) supervised ML to predict which template can best remediate a given violation, d) reinforcement learning algorithms for more complex scenarios, to predict a sequence of remediations actions on one or several devices, among others.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura in view of Ristoski’s teaching with Triplet’s teaching of SME defined constraining rules in order to allow an SME to establish policy constraints that guardrail machine learning decisions toward goal-advancing outcomes (Paragraph 0002 of Triplet).
Regarding claim 6,
Kimura teaches programming a reinforcement learning (RL) computing environment comprising an RL algorithm acting as a reward-based RL agent programmed to make decisions in furtherance of a goal… (Pg. 2 Paragraph 1 of Section 3.1, “As text-based games are sequential decision-making problems, they can naturally be applied to RL. These games are partially observable… where the observation text does not include the entire information of the environment… The objective for the agent is to maximize the expected discounted reward E”);
and integrating a logical optimal action (LOA) computing framework (neuro-symbolic framework) integrated with the RL computing environment, wherein the LOA computing framework comprises (Page 1 Under the Figure Caption of the Introduction, “In order to train logical rules, a recent neuro-symbolic framework called the Logical Neural Network (LNN) (Riegel et al., 2020) has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning). The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss.”, Page 2 Paragraph 2 of Introduction, “In this paper, we propose an action knowledge acquisition method featuring a neuro-symbolic LNN framework for the RL algorithm.”
The specification of the instant case states that the logical optimal action refers to neuro-symbolic LNN that is capable of optimizing the LNN. In this case, the reference teaches using a neuro-symbolic framework using Logical Neural Network integrated with the reinforced learning environment.)
and (2) a logical neural network (LNN) that is trained on the at least one dataset… and establishes LNN policy rules based on the training (Page 1 of Introduction Under Figure 1 Caption, “The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss… At the same time, the trained LNN can extract obtained logical rules by selecting high weighted connections that represent the important rules for an action policy.”, See Figure 1,
PNG
media_image1.png
418
423
media_image1.png
Greyscale
Here, the dataset corresponds to the observation in the Figure, the constraining rules correspond to the symbolic logical rules generated from the FOL converter, and the LNN policy rules corresponds to the output of the logical neural network which provides the rules used for the agent to make decisions. The observation and the rules are fed into the LNN to generate the policy rules for the agent to perform the actions.),
(ii) rules comprising predicates and logical operations… that constrain the scope of the at least one dataset… (Pg. 1 Under Figure 1 Caption in Introduction Section, “…the Logical Neural Network (LNN)… Every neuron in the LNN has a component for a formula of weighted real-valued logics from a unique logical conjunction, disjunction, or negation nodes”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts…", Page 2 Introduction, “The method then converts first-order logical (predicates) facts from the propositional logics and categories of each word, such as ∃x ∈ {south, north},⟨find x⟩ = True and ∃x ∈ {east, west},⟨find x⟩ = False.”, Page 4 Section 4, “We attached the extracted rules from the medium level games here: ∃x ∈ W_direction (⟨find x⟩∧¬⟨visited x⟩ ∧¬⟨initial x⟩)∨ (⟨find x⟩∧⟨all are visited⟩ ∧ ⟨initial x⟩) → ⟪go x⟫,”
The rules comprising predicates and logical operations that limit the scope of the at least one dataset correspond to the symbolic logical rules used in the Logical Neural network (LNN). The FOL facts are explicitly predicates (e.g., find x, visited x, initial x) combined via logical operations (AND, OR, NOT) within the LNN’s neuron structure. The rules are constrained to logical predicate rules, thus limiting the scope of the information in the dataset based on the results of the FOL converter and the predicate logic of the LNN.)
wherein the at least one dataset and the policy rules are input into the RL computing environment where the RL agent makes decisions on the information in the at least one dataset that comply with the LNN policy rules and the RL agent is rewarded for decisions that advance the goal (Page 5 Paragraph 1 Under Discussion about ethics, “The data set used in our experiment does not contain any sensitive information.”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts… the agent understands an opened direction from the current room. The agent then retrieves the class type c of the word meaning in propositional logic l_i, t by using Concept Net… or the network of another word’s definition.”, Page 3 Paragraph 1 of Section 3.2.2, “The LNN training component is for obtaining an action policy from the given FOL logics.”, See Figure 1,
PNG
media_image1.png
418
423
media_image1.png
Greyscale
The reference teaches that observation data from the environment, in the form of natural language text, constitutes a dataset that is processed and converted into first-order logical facts and input into a Logical Neural Network (LNN), as shown in Figure 1 and described in Section 3.2. The LNN learns logical policy rules that are input into the reinforcement learning computing environment and directly governs the RL agent’s action selection. The RL agent makes decisions that comply with the LNN policy rules and is rewarded based on the outcome of those decisions.).
Kimura does not teach a system comprising: a processor set: one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to cause the processor set to perform operations, an interface for entry of (i) at least one dataset comprising information relating to existing polymers, (ii) rules… that constrain the scope of the at least one dataset to available materials and laboratory equipment, a reward-based RL agent programmed to make decisions in furtherance of a goal relating to the discovery of new polymer materials, rules defined by a subject matter expert (SME), and train[ing] on… the SME-defined constraining rules.
Ristoski, in the same field of endeavor, teaches [a] system comprising: a processor set: one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to cause the processor set to perform operations comprising: (Paragraph 9 of Ristoski, “A system, in one aspect, can include a hardware processor and a memory device coupled with the hardware processor. The hardware processor can be configured to generate a set of material candidates expected to yield materials with target properties. The hardware processor can also be configured to receive subject matter expert's decision indicating accepted and rejected material candidates from the set of material candidates. The hardware processor can also be configured to, based on the subject matter expert's input, train a machine learning model to replicate the subject matter expert's decision.”, Paragraph 12, “A computer readable storage medium storing a program of instructions executable by a machine to perform one or more methods described herein also may be provided.”)
(1) an interface for entry of (i) at least one dataset comprising information relating to existing polymers (Paragraph 0091, “For instance, at least one hardware processor 1102 may receive training data set which can include candidate material for polymerization, identify one or more desired features in the candidate material, and train a machine learning model to generate a new material having one or more of the desired features.”, Paragraph 0088, “In an embodiment, e.g., optionally, the identified desired features can be presented or caused to be presented on a user interface, e.g., for user interaction or view. In an embodiment, a user (e.g., an SME) may select from the desired features, a desired feature to include in the new material being generated.”
The reference teaches an interface or GUI that allows for the user to select desired features, which is considered an entry due to the fact that the user is entering constraints into the model.)
(ii) rules… that constrain the scope of the at least one dataset to available materials and laboratory equipment (Paragraph 0042, “The general model tries to learn general rules for synthesizing new materials that are shared across all labs and SMEs, e.g., each monomer suitable for polymerization must contains a polymerizable group and should not contain pendant groups that will be active under polymerization conditions.”, Paragraph 0050, “By way of example only, a training data set to train or build the classification model 106 can include a random sample from a combinatorial library of polymerizable components suitable for the preparation of polyimides, including dianhydrides, dicarboxylic acids, and diamines.”),
a reward-based RL agent programmed to make decisions in furtherance of a goal relating to the discovery of new polymer materials (Paragraph 0063 of Ristoski, “SeqGAN is a Sequence Generative Adversarial Network using reinforcement learning. The system includes generator and discriminator, where the generator is treated as reinforcement learning agent. In such a scenario, the generated tokens represent the state and the action is the next token to be generated… The reward can be calculated by the discriminator on a complete sequence via Monte Carlo search.”, Paragraph 0002, “Approaches have emerged, such as computational screening, inverse design, generative modeling, reinforcement learning as ways to accelerate the design of polymer materials.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura’s teaching of integrating a neuro-symbolic Logical Neural Network (LNN) framework with a reinforcement learning environment for decision making with Ristoski’s teaching of providing a user interface for entering polymer related constraints in a system in order to apply the reinforcement learning and LNN to the discovery of new polymer materials, thereby improving the efficiency and guidance of polymer discovery (Paragraph 0002 of Ristoski).
Kimura in view of Ristoski do not teach rules defined by a subject matter expert (SME) and train[ing] on… the SME-defined constraining rules.
Triplet, in the same field of endeavor, teaches rules… defined by a subject matter expert (SME) (Paragraph 0097 of Triplet, “The methods may detect compliance violations in real-time and predict remediation actions to non-compliant events, such as by using a) rules manually defined by SME to map each policy to a remediation template, b) collaborative filtering algorithms to predict which template can best remediate a given violation based on historical data, c) supervised ML to predict which template can best remediate a given violation, d) reinforcement learning algorithms for more complex scenarios, to predict a sequence of remediations actions on one or several devices, among others.”)
train[ing] on… the SME-defined constraining rules (Paragraph 0097, “The methods may detect compliance violations in real-time and predict remediation actions to non-compliant events, such as by using a) rules manually defined by SME to map each policy to a remediation template… c) supervised ML to predict which template can best remediate a given violation, d) reinforcement learning algorithms for more complex scenarios, to predict a sequence of remediations actions on one or several devices, among others.”, Paragraph 0041, “In some embodiments, AI may be used to discover these rules and policies and/or generate the rules to improve some area of rules compliance. Configuring policies and rules may be part of an ML training process.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura in view of Ristoski’s teaching with Triplet’s teaching of SME defined constraining rules in order to allow an SME to establish policy constraints that guardrail machine learning decisions toward goal-advancing outcomes (Paragraph 0002 of Triplet).
Claim 7 is an article of manufacture claim that recites identical limitations to method claim 3. Therefore, claim 7 is rejected using the same rationale as claim 3.
Claim 8 is an article of manufacture claim that recites identical limitations to method claim 4. Therefore, claim 8 is rejected using the same rationale as claim 4.
Regarding claim 9,
Kimura does not teach the SME reviews the decisions by the RL agent to eliminate decisions that do not advance the goal.
Ristoski, in the same field of endeavor, teaches the SME reviews the decisions by the RL agent to eliminate decisions that do not advance the goal (Paragraph 0002 of Ristoski, “Approaches have emerged, such as computational screening, inverse design, generative modeling, reinforcement learning as ways to accelerate the design of polymer materials. The drawback of these approaches is that they generate a large number of candidates for new molecules, which then need to be manually reviewed by subject matter experts who select only a dozen for further investigation.”, Paragraph 0063, “SeqGAN is a Sequence Generative Adversarial Network using reinforcement learning. The system includes generator and discriminator, where the generator is treated as reinforcement learning agent… the generated tokens represent the state and the action is the next token to be generated.”, Paragraph 0004, “Subject matter experts (SMEs) such as polymer chemists and synthetic organic chemists review many pages of candidates and select the ones that appear viable for further testing. With the very large number of candidates, and the fact that only a few will be selected for experiment manually”
The reinforcement learning agent generates material candidates and the outcome of the agent is used to have a SME review the decisions of the agent, and selects certain candidates for further testing).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura’s reinforcement learning system employing neuro-symbolic policy reasoning with Ristoski’s teaching of subject matter expert review of generated material candidates from the reinforcement learning agent in order to apply Ristoski’s SME to evaluate and eliminate agent decisions that do not advance the goal, thus improving the quality and relevance of selected outcomes (Paragraph 0032 of Ristoski).
Regarding claim 10,
Kimura teaches [a] computer-implemented method comprising (Page 2 Paragraph 2 in Introduction, “In this paper, we propose an action knowledge acquisition method featuring a neuro-symbolic LNN framework for the RL algorithm.”),
programming a reinforcement learning (RL) algorithm to act as a reward-based RL agent within an RL computing environment (See Algorithm 1 RL by FOL-LNN,
PNG
media_image5.png
419
389
media_image5.png
Greyscale
),
wherein the RL agent makes decisions in furtherance of a goal… (Pg. 2 Paragraph 1 of Section 3.1, “As text-based games are sequential decision-making problems, they can naturally be applied to RL. These games are partially observable… where the observation text does not include the entire information of the environment… The objective for the agent is to maximize the expected discounted reward E”);
and integrating a logical optimal action (LOA) computing framework with the RL computing environment, wherein the LOA computing framework comprises (Page 1 Under the Figure Caption of the Introduction, “In order to train logical rules, a recent neuro-symbolic framework called the Logical Neural Network (LNN) (Riegel et al., 2020) has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning). The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss.”, Page 2 Paragraph 2 of Introduction, “In this paper, we propose an action knowledge acquisition method featuring a neuro-symbolic LNN framework for the RL algorithm.”
The specification of the instant case states that the logical optimal action refers to neuro-symbolic LNN that is capable of optimizing the LNN. In this case, the reference teaches using a neuro-symbolic framework using Logical Neural Network integrated with the reinforced learning environment.)
(ii) constraining rules comprising predicates and logical operations that limit the scope of the information in the at least one dataset (Page 5 Paragraph 1 Under Discussion about ethics, “The data set used in our experiment does not contain any sensitive information.”, Pg. 1 Under Figure 1 Caption In Introduction Section, “…the Logical Neural Network (LNN)… has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning). The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss. Every neuron in the LNN has a component for a formula of weighted real-valued logics from a unique logical conjunction, disjunction, or negation nodes… the trained LNN can extract obtained logical rules by selecting high weighted connections that represent the important rules for an action policy”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts…", Page 4 Section 4, “We attached the extracted rules from the medium level games here: ∃x ∈ W_direction (⟨find x⟩∧¬⟨visited x⟩ ∧¬⟨initial x⟩)∨ (⟨find x⟩∧⟨all are visited⟩ ∧ ⟨initial x⟩) → ⟪go x⟫,”
The "constraining rules that limit the scope of the information in the at least one dataset" correspond to the symbolic logical rules used in the Logical Neural network (LNN). The symbolic logical rules are the parsed observation from the FOL converter that is used in the LNN. The rules are constrained to logical rules only, thus limiting the scope of the information in the dataset based on the effect of the FOL converter.),
and (2) a logical neural network (LNN) that is trained on the at least one dataset and the constraining rules and establishes LNN policy rules based on the training (Page 1 of Introduction Under Figure 1 Caption, “The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss… At the same time, the trained LNN can extract obtained logical rules by selecting high weighted connections that represent the important rules for an action policy.”),
wherein the at least one dataset and the LNN policy rules are input into the RL computing environment where the RL agent makes decisions on the information in the at least one dataset that comply with the LNN policy rules and the RL agent is rewarded for decisions that advance the goal (Page 5 Paragraph 1 Under Discussion about ethics, “The data set used in our experiment does not contain any sensitive information.”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts… the agent understands an opened direction from the current room. The agent then retrieves the class type c of the word meaning in propositional logic l_i,t by using Concept Net… or the network of another word’s definition.”, Page 3 Paragraph 1 of Section 3.2.2, “The LNN training component is for obtaining an action policy from the given FOL logics.”, See Figure 1,
PNG
media_image1.png
418
423
media_image1.png
Greyscale
The reference teaches that observation data from the environment, in the form of natural language text, constitutes a dataset that is processed and converted into first-order logical facts and input into a Logical Neural Network (LNN), as shown in Figure 1 and described in Section 3.2. The LNN learns logical policy rules that are input into the reinforcement learning computing environment and directly governs the RL agent’s action selection. The RL agent makes decisions that comply with the LNN policy rules and is rewarded based on the outcome of those decisions.).
Kimura does not teach a system, an interface for entry of (i) at least one dataset comprising information relating to existing polymers, and a dataset relating to the discovery of new polymer materials.
Ristoski, in the same field of endeavor, teaches relating to the discovery of new polymer materials (Paragraph 0080, “FIG. 8 is a diagram illustrating a method of polymer discovery in one embodiment… At 802, the method can include generating a set of material candidates expected to yield materials with target properties. Examples of material candidates include, but not limited to, molecules, monomers, and/or polymer repeat units.”)
(1) an interface for entry of (i) at least one dataset comprising information relating to existing polymers (Paragraph 0091, “For instance, at least one hardware processor 1102 may receive training data set which can include candidate material for polymerization, identify one or more desired features in the candidate material, and train a machine learning model to generate a new material having one or more of the desired features.”, Paragraph 0088, “In an embodiment, e.g., optionally, the identified desired features can be presented or caused to be presented on a user interface, e.g., for user interaction or view. In an embodiment, a user (e.g., an SME) may select from the desired features, a desired feature to include in the new material being generated.”
The reference teaches an interface or GUI that allows for the user to select desired features, which is considered an entry due to the fact that the user is entering constraints into the model.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura’s teaching of integrating a neuro-symbolic Logical Neural Network (LNN) framework with a reinforcement learning environment for decision making with Ristoski’s teaching of providing a user interface for entering polymer related constraints in order to apply the reinforcement learning and LNN to the discovery of new polymer materials, thereby improving the efficiency and guidance of polymer discovery (Paragraph 0002 of Ristoski).
Kimura in view of Ristoski appears to not teach SME defined constraining rules.
Triplet, in the same field of endeavor, teaches SME defined constraining rules… that limit the scope of the information in the at least one dataset (Paragraph 97 of Triplet, “The methods may detect compliance violations in real-time and predict remediation actions to non-compliant events, such as by using a) rules manually defined by SME to map each policy to a remediation template, b) collaborative filtering algorithms to predict which template can best remediate a given violation based on historical data, c) supervised ML to predict which template can best remediate a given violation, d) reinforcement learning algorithms for more complex scenarios, to predict a sequence of remediations actions on one or several devices, among others.”, Paragraph 41, “In some embodiments, AI may be used to discover these rules and policies and/or generate the rules to improve some area of rules compliance. Configuring policies and rules may be part of an ML training process.”
Triplet teaches SME defining rules constrained to compliance violations that limits the scope of violations based on historical data.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura in view of Ristoski’s teaching with Triplet’s teaching of SME defined constraining rules in order to allow an SME to establish policy constraints that guardrail machine learning decisions toward goal-advancing outcomes (Paragraph 0002 of Triplet).
Claim 11 is a process claim that recites similar limitations to method claim 2. Therefore, claim 11 is rejected using the same rationale as claim 2.
Claim 12 is a process claim that recites similar limitations to method claim 3. Therefore, claim 12 is rejected using the same rationale as claim 3.
Claim 13 is a process claim that recites similar limitations to method claim 4. Therefore, claim 13 is rejected using the same rationale as claim 4.
Claim 14 is a process claim that recites similar limitations to method claim 5. Therefore, claim 14 is rejected using the same rationale as claim 5.
Regarding claim 15,
Kimura teaches [a] computer-implemented method comprising (Page 2 Paragraph 2 in Introduction, “In this paper, we propose an action knowledge acquisition method featuring a neuro-symbolic LNN framework for the RL algorithm.”):
programming a reinforcement learning (RL) algorithm to act as a reward-based RL agent within an RL computing environment (See Algorithm 1 RL by FOL-LNN,
PNG
media_image5.png
419
389
media_image5.png
Greyscale
),
wherein the RL agent makes decisions in furtherance of a goal… (Pg. 2 Paragraph 1 of Section 3.1, “As text-based games are sequential decision-making problems, they can naturally be applied to RL. These games are partially observable… where the observation text does not include the entire information of the environment… The objective for the agent is to maximize the expected discounted reward E”);
and integrating a logical optimal action (LOA) computing framework with the RL computing environment, wherein the LOA computing framework comprises (Page 1 Under the Figure Caption of the Introduction, “In order to train logical rules, a recent neuro-symbolic framework called the Logical Neural Network (LNN) (Riegel et al., 2020) has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning). The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss.”, Page 2 Paragraph 2 of Introduction, “In this paper, we propose an action knowledge acquisition method featuring a neuro-symbolic LNN framework for the RL algorithm.”
The specification of the instant case states that the logical optimal action refers to neuro-symbolic LNN that is capable of optimizing the LNN. In this case, the reference teaches using a neuro-symbolic framework using Logical Neural Network integrated with the reinforced learning environment.)
and (2) a logical neural network (LNN) that is trained on the at least one dataset… and establishes LNN policy rules based on the training (Page 1 of Introduction Under Figure 1 Caption, “The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss… At the same time, the trained LNN can extract obtained logical rules by selecting high weighted connections that represent the important rules for an action policy.”, See Figure 1,
PNG
media_image1.png
418
423
media_image1.png
Greyscale
Here, the dataset corresponds to the observation in the Figure, the constraining rules correspond to the symbolic logical rules generated from the FOL converter, and the LNN policy rules corresponds to the output of the logical neural network which provides the rules used for the agent to make decisions. The observation and the rules are fed into the LNN to generate the policy rules for the agent to perform the actions.),
(ii) rules comprising predicates and logical operations … that constrain the scope of the at least one dataset… (Pg. 1 Under Figure 1 Caption In Introduction Section, “…the Logical Neural Network (LNN)… Every neuron in the LNN has a component for a formula of weighted real-valued logics from a unique logical conjunction, disjunction, or negation nodes”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts…", Page 2 Introduction, “The method then converts first-order logical (predicates) facts from the propositional logics and categories of each word, such as ∃x ∈ {south, north},⟨find x⟩ = True and ∃x ∈ {east, west},⟨find x⟩ = False.”, Page 4 Section 4, “We attached the extracted rules from the medium level games here: ∃x ∈ W_direction (⟨find x⟩∧¬⟨visited x⟩ ∧¬⟨initial x⟩)∨ (⟨find x⟩∧⟨all are visited⟩ ∧ ⟨initial x⟩) → ⟪go x⟫,”
The "rules comprising predicates and logical operations that constrain the scope of the information of the at least one dataset" correspond to the symbolic logical rules used in the Logical Neural network (LNN). The FOL facts are explicitly predicates (e.g., find x, visited x, initial x) combined via logical operations (AND, OR, NOT) within the LNN’s neuron structure. The rules are constrained to logical predicate rules, thus limiting the scope of the information in the dataset based on the results of the FOL converter and the predicate logic of the LNN.)
wherein the at least one dataset and the policy rules are input into the RL computing environment where the RL agent makes decisions on the information in the at least one dataset that comply with the LNN policy rules and the RL agent is rewarded for decisions that advance the goal (Page 5 Paragraph 1 Under Discussion about ethics, “The data set used in our experiment does not contain any sensitive information.”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts… the agent understands an opened direction from the current room. The agent then retrieves the class type c of the word meaning in propositional logic l_i,t by using Concept Net… or the network of another word’s definition.”, Page 3 Paragraph 1 of Section 3.2.2, “The LNN training component is for obtaining an action policy from the given FOL logics.”, See Figure 1,
PNG
media_image1.png
418
423
media_image1.png
Greyscale
The reference teaches that observation data from the environment, in the form of natural language text, constitutes a dataset that is processed and converted into first-order logical facts and input into a Logical Neural Network (LNN), as shown in Figure 1 and described in Section 3.2. The LNN learns logical policy rules that are input into the reinforcement learning computing environment and directly governs the RL agent’s action selection. The RL agent makes decisions that comply with the LNN policy rules and is rewarded based on the outcome of those decisions.).
wherein the decisions that advance the goal are used to refine the LNN policy rules (Page 3 Section 3.2.2, “The LNN trains by this fact inputs and reward; that means it forwards from input facts through LNN, calculates a loss value from the reward value, and optimizes weights in LNN.”, See Algorithm 1 on Page 3,
PNG
media_image2.png
408
381
media_image2.png
Greyscale
Kimura teaches that goal-advancing decisions are stored in the replay buffer are used to update the LNN weights via gradient updates (line 14 of Algorithm 1). The reward signal, which corresponds to decisions that advance the goal of maximizing expected discounted reward, drives the optimization of the LNN weights, which in turn refines the policy rules extracted from the LNN. Specifically, Kimura shows that the reward from goal advancing decisions is used to update the LNN weights at line 14 of Algorithm 1. The LNN weights encode the logical connections that define the LNN policy rules. Thus, updating the weights based on goal advancing decisions directly refines those policy rules.)
Kimura does not teach relating to the discovery of new polymer materials, an interface for entry of (i) at least one dataset comprising information relating to existing polymers, rules… that constrain the scope of the at least one dataset to available materials and laboratory equipment, rules defined by a subject matter expert (SME), and train[ing] on… the SME-defined constraining rules.
Ristoski, in the same field of endeavor, teaches relating to the discovery of new polymer materials (Paragraph 0080, “FIG. 8 is a diagram illustrating a method of polymer discovery in one embodiment… At 802, the method can include generating a set of material candidates expected to yield materials with target properties. Examples of material candidates include, but not limited to, molecules, monomers, and/or polymer repeat units.”)
(1) an interface for entry of (i) at least one dataset comprising information relating to existing polymers (Paragraph 0091, “For instance, at least one hardware processor 1102 may receive training data set which can include candidate material for polymerization, identify one or more desired features in the candidate material, and train a machine learning model to generate a new material having one or more of the desired features.”, Paragraph 0088, “In an embodiment, e.g., optionally, the identified desired features can be presented or caused to be presented on a user interface, e.g., for user interaction or view. In an embodiment, a user (e.g., an SME) may select from the desired features, a desired feature to include in the new material being generated.”
The reference teaches an interface or GUI that allows for the user to select desired features, which is considered an entry due to the fact that the user is entering constraints into the model.),
and (ii) rules… that constrain the scope of the at least one dataset to available materials and laboratory equipment (Paragraph 0042, “The general model tries to learn general rules for synthesizing new materials that are shared across all labs and SMEs, e.g., each monomer suitable for polymerization must contains a polymerizable group and should not contain pendant groups that will be active under polymerization conditions.”, Paragraph 0050, “By way of example only, a training data set to train or build the classification model 106 can include a random sample from a combinatorial library of polymerizable components suitable for the preparation of polyimides, including dianhydrides, dicarboxylic acids, and diamines.”),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura’s teaching of integrating a neuro-symbolic Logical Neural Network (LNN) framework with a reinforcement learning environment for decision making with Ristoski’s teaching of providing a user interface for entering polymer related constraints in order to apply the reinforcement learning and LNN to the discovery of new polymer materials, thereby improving the efficiency and guidance of polymer discovery (Paragraph 0002 of Ristoski).
Kimura in view of Ristoski do not teach rules defined by a subject matter expert (SME) and train[ing] on… the SME-defined constraining rules.
Triplet, in the same field of endeavor, teaches rules defined by a subject matter expert (SME) (Paragraph 0097 of Triplet, “The methods may detect compliance violations in real-time and predict remediation actions to non-compliant events, such as by using a) rules manually defined by SME to map each policy to a remediation template, b) collaborative filtering algorithms to predict which template can best remediate a given violation based on historical data, c) supervised ML to predict which template can best remediate a given violation, d) reinforcement learning algorithms for more complex scenarios, to predict a sequence of remediations actions on one or several devices, among others.”)
train[ing] on… the SME-defined constraining rules (Paragraph 0097, “The methods may detect compliance violations in real-time and predict remediation actions to non-compliant events, such as by using a) rules manually defined by SME to map each policy to a remediation template… c) supervised ML to predict which template can best remediate a given violation, d) reinforcement learning algorithms for more complex scenarios, to predict a sequence of remediations actions on one or several devices, among others.”, Paragraph 0041, “In some embodiments, AI may be used to discover these rules and policies and/or generate the rules to improve some area of rules compliance. Configuring policies and rules may be part of an ML training process.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura and Ristoski’s teaching with Triplet’s teaching of SME defined constraining rules in order to allow an SME to establish policy constraints that guardrail machine learning decisions toward goal-advancing outcomes (Paragraph 0002 of Triplet).
Claim 16 is a process claim that recites similar limitations to method claim 7. Therefore, claim 16 is rejected using the same rationale as claim 7.
Claim 17 is a process claim that recites similar limitations to method claim 8. Therefore, claim 17 is rejected using the same rationale as claim 8.
Claim 18 is a process claim that recites similar limitations to method claim 9. Therefore, claim 18 is rejected using the same rationale as claim 9.
Regarding claim 19,
Kimura teaches establishing a reinforcement learning (RL) computing environment comprising an RL algorithm acting as a reward-based RL agent programmed to make decisions in furtherance of a goal… (Pg. 2 Paragraph 1 of Section 3.1, “As text-based games are sequential decision-making problems, they can naturally be applied to RL. These games are partially observable… where the observation text does not include the entire information of the environment… The objective for the agent is to maximize the expected discounted reward E”);
establishing a logical optimal action (LOA) computing framework integrated with the RL computing environment, wherein the LOA computing framework comprises (Page 1 Under the Figure Caption of the Introduction, “In order to train logical rules, a recent neuro-symbolic framework called the Logical Neural Network (LNN) (Riegel et al., 2020) has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning). The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss.”, Page 2 Paragraph 2 of Introduction, “In this paper, we propose an action knowledge acquisition method featuring a neuro-symbolic LNN framework for the RL algorithm.”
The specification of the instant case states that the logical optimal action refers to neuro-symbolic LNN that is capable of optimizing the LNN. In this case, the reference teaches using a neuro-symbolic framework using Logical Neural Network integrated with the reinforced learning environment.)
(ii) constraining rules comprising predicates and logical operations, that limit the scope of the information in the at least one dataset (Page 5 Paragraph 1 Under Discussion about ethics, “The data set used in our experiment does not contain any sensitive information.”, Pg. 1 Under Figure 1 Caption In Introduction Section, “…the Logical Neural Network (LNN)… has been proposed to simultaneously provide key properties of both the neural network (learning) and the symbolic logic (reasoning). The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss. Every neuron in the LNN has a component for a formula of weighted real-valued logics from a unique logical conjunction, disjunction, or negation nodes… the trained LNN can extract obtained logical rules by selecting high weighted connections that represent the important rules for an action policy”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts…", Page 2 Introduction, “The method then converts first-order logical (predicates) facts from the propositional logics and categories of each word, such as ∃x ∈ {south, north},⟨find x⟩ = True and ∃x ∈ {east, west},⟨find x⟩ = False.”, Page 4 Section 4, “We attached the extracted rules from the medium level games here: ∃x ∈ W_direction (⟨find x⟩∧¬⟨visited x⟩ ∧¬⟨initial x⟩)∨ (⟨find x⟩∧⟨all are visited⟩ ∧ ⟨initial x⟩) → ⟪go x⟫,”
The "constraining rules comprising predicates and logical operations that limit the scope of the information in the at least one dataset" correspond to the symbolic logical rules used in the Logical Neural network (LNN). The FOL facts are explicitly predicates (e.g., find x, visited x, initial x) combined via logical operations (AND, OR, NOT) within the LNN’s neuron structure. The rules are constrained to logical predicate rules, thus limiting the scope of the information in the dataset based on the results of the FOL converter and the predicate logic of the LNN.),
and (2) a logical neural network (LNN) that is trained on the at least one dataset and the constraining rules and establishes LNN policy rules based on the training (Page 1 of Introduction Under Figure 1 Caption, “The LNN can train the symbolic rules with logical functions in the neural networks by having an end-to-end differentiable network minimizes a contradiction loss… At the same time, the trained LNN can extract obtained logical rules by selecting high weighted connections that represent the important rules for an action policy.”, See Figure 1,
PNG
media_image1.png
418
423
media_image1.png
Greyscale
Here, the dataset corresponds to the observation in the Figure, the constraining rules correspond to the symbolic logical rules generated from the FOL converter, and the LNN policy rules corresponds to the output of the logical neural network which provides the rules used for the agent to make decisions. The observation and the rules are fed into the LNN to generate the policy rules for the agent to perform the actions.),
wherein the at least one dataset and the LNN policy rules are input into the RL computing environment where the RL agent makes decisions on the information in the at least one dataset that comply with the LNN policy rules and the RL agent is rewarded for decisions that advance the goal (Page 5 Paragraph 1 Under Discussion about ethics, “The data set used in our experiment does not contain any sensitive information.”, Page 3 of Section 3.2.1, “The FOL converter converts a given natural observation text ot and observation history (ot−1, ot−2, ...) into first-order logic facts… the agent understands an opened direction from the current room. The agent then retrieves the class type c of the word meaning in propositional logic l_i,t by using ConceptNet… or the network of another word’s definition.”, Page 3 Paragraph 1 of Section 3.2.2, “The LNN training component is for obtaining an action policy from the given FOL logics.”, See Figure 1,
The reference teaches that observation data from the environment, in the form of natural language text, constitutes a dataset that is processed and converted into first-order logical facts and input into a Logical Neural Network (LNN), as shown in Figure 1 and described in Section 3.2. The LNN learns logical policy rules that are input into the reinforcement learning computing environment and directly governs the RL agent’s action selection. The RL agent makes decisions that comply with the LNN policy rules and is rewarded based on the outcome of those decisions.).
wherein the decisions that advance the goal are used to refine the LNN policy rules (Page 3 Section 3.2.2, “The LNN trains by this fact inputs and reward; that means it forwards from input facts through LNN, calculates a loss values from the reward value, and optimizes weights in LNN.”, See Algorithm 1 on Page 3,
PNG
media_image2.png
408
381
media_image2.png
Greyscale
Kimura teaches that goal-advancing decisions are stored in the replay buffer are used to update the LNN weights via gradient updates (line 14 of Algorithm 1). The reward signal, which corresponds to decisions that advance the goal of maximizing expected discounted reward, drives the optimization of the LNN weights, which in turn refines the policy rules extracted from the LNN. Specifically, Kimura shows that the reward from goal advancing decisions is used to update the LNN weights at line 14 of Algorithm 1. The LNN weights encode the logical connections that define the LNN policy rules. Thus, updating the weights based on goal advancing decisions directly refines those policy rules.).
Kimura does not teach [a] computer program product for discovery of polymers comprising: program instructions on or more computer readable storage media to cause a processor set to perform operations comprising…, relating to the discovery of new polymer materials, (1) an interface for entry of (i) at least one dataset comprising information relating to existing polymers, and (ii) Subject Matter Expert defined constraining rules… that limit the scope of the information in the at least one dataset.
Ristoski, in the same field of endeavor, teaches [a] computer program product for discovery of polymers comprising: program instructions on or more computer readable storage media to cause a processor set to perform operations comprising (Paragraph 9 of Ristoski, “A system, in one aspect, can include a hardware processor and a memory device coupled with the hardware processor. The hardware processor can be configured to generate a set of material candidates expected to yield materials with target properties. The hardware processor can also be configured to receive subject matter expert's decision indicating accepted and rejected material candidates from the set of material candidates. The hardware processor can also be configured to, based on the subject matter expert's input, train a machine learning model to replicate the subject matter expert's decision.”, Paragraph 12, “A computer readable storage medium storing a program of instructions executable by a machine to perform one or more methods described herein also may be provided.”, Paragraph 12, “A computer readable storage medium storing a program of instructions executable by a machine to perform one or more methods described herein also may be provided.”)
relating to the discovery of new polymer materials (Paragraph 0080, “FIG. 8 is a diagram illustrating a method of polymer discovery in one embodiment… At 802, the method can include generating a set of material candidates expected to yield materials with target properties. Examples of material candidates include, but not limited to, molecules, monomers, and/or polymer repeat units.”)
(1) an interface for entry of (i) at least one dataset comprising information relating to existing polymers (Paragraph 0091, “For instance, at least one hardware processor 1102 may receive training data set which can include candidate material for polymerization, identify one or more desired features in the candidate material, and train a machine learning model to generate a new material having one or more of the desired features.”, Paragraph 0088, “In an embodiment, e.g., optionally, the identified desired features can be presented or caused to be presented on a user interface, e.g., for user interaction or view. In an embodiment, a user (e.g., an SME) may select from the desired features, a desired feature to include in the new material being generated.”
The reference teaches an interface or GUI that allows for the user to select desired features, which is considered an entry due to the fact that the user is entering constraints into the model.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura’s teaching of integrating a neuro-symbolic Logical Neural Network (LNN) framework with a reinforcement learning environment for decision making with Ristoski’s teaching of providing a user interface for entering polymer related constraints used in a computer program in order to apply the reinforcement learning and LNN to the discovery of new polymer materials in a computer program, thereby improving the efficiency and guidance of polymer discovery (Paragraph 0002 of Ristoski).
Kimura in view of Ristoski does not teach ii) Subject Matter Expert defined constraining rules.
Triplet, in the same field of endeavor, teaches (ii) Subject Matter Expert defined constraining rules… that limit the scope of the information in the at least one dataset (Paragraph 0097 of Triplet, “The methods may detect compliance violations in real-time and predict remediation actions to non-compliant events, such as by using a) rules manually defined by SME to map each policy to a remediation template, b) collaborative filtering algorithms to predict which template can best remediate a given violation based on historical data, c) supervised ML to predict which template can best remediate a given violation, d) reinforcement learning algorithms for more complex scenarios, to predict a sequence of remediations actions on one or several devices, among others.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Kimura in view of Ristoski’s teaching with Triplet’s teaching of SME defined constraining rules in order to allow an SME to establish policy constraints that guardrail machine learning decisions toward goal-advancing outcomes (Paragraph 0002 of Triplet).
Claim 20 is an article of manufacture claim that recites similar limitations to method claim 2. Therefore, claim 20 is rejected using the same rationale as claim 2.
Claim 21 is an article of manufacture claim that recites similar limitations to method claim 3. Therefore, claim 21 is rejected using the same rationale as claim 3.
Claim 22 is an article of manufacture claim that recites similar limitations to method claim 4. Therefore, claim 22 is rejected using the same rationale as claim 4.
Claim 23 is an article of manufacture claim that recites similar limitations to method claim 5. Therefore, claim 23 is rejected using the same rationale as claim 5.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAJD MAHER HADDAD whose telephone number is (571)272-2265. The examiner can normally be reached Mon-Friday 8-5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.M.H./Examiner, Art Unit 2125
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
Prosecution Insights