DETAILED ACTION
The Applicant’s filing, received 01 August 2022, has been fully considered. The following rejections and/or objections constitute the complete set presently being applied to the instant application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
The preliminary amendment received 01 August 2022 has been entered.
Claims 21-40 are pending.
Claims 21-40 are rejected.
Claims 23 and 40 are objected to.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
This application is a 371 of PCT/CA2021/050103, filed 29 January 2021, which claims benefit of U.S. provisional application number 62/967,898, filed 30 January 2020, and claims benefit of U.S. provisional application number 63/076,151, filed 09 September 2020.
Therefore, the effective filing date of the claimed invention is 30 January 2020.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01 August 2022 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Drawings
Two sets of drawings were received on 01 August 2022. The set of drawings without the header information “WO 2021/151208” and “PCT/CA2021/050103” are acceptable.
Specification
Two specifications were received on 01 August 2022. The specification without the header information “WO 2021/151208” and “PCT/CA2021/050103” is entered.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) are:
a “chemical reaction prediction module” in claims 21, 35, and 36;
a “scoring function module” in claims 21 and 37;
an “approximation module” in claims 22 and 25;
at least one “actor module” in claims 23 and 34;
at least one “critic module” in claims 24, 26, 29, 34;
a “reinforcement learning module” in claim 30; and
a “retrosynthesis prediction module” in claim 38.
Because these claim limitation(s) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The written description discloses a corresponding structure for:
a “chemical reaction prediction module” in claims 21, 35, and 36, i.e., hardware, e.g., a computing device (para. [24]); and software, e.g., machine learning algorithms (paras. [10], [25], [27], [28], [29], [30], & [70]);
a “scoring function module” in claims 21 and 37, i.e., hardware, e.g., a computing device (para. [24]); and software, e.g., machine learning algorithms (paras. [10], [25], [27], [28], [29], [30], & [70]);
an “approximation module” in claims 22 and 25, i.e., hardware, e.g., a computing device (para. [24]); and software, e.g., machine learning algorithms (paras. [10], [25], [27], [28], [29], [30], & [70]);
at least one “actor module” in claims 23 and 34, i.e., hardware, e.g., a computing device (para. [24]); and software, e.g., machine learning algorithms (paras. [10], [25], [27], [28], [29], [30], & [70]);
at least one “critic module” in claims 24, 26, 29, 34, i.e., hardware, e.g., a computing device (para. [24]); and software, e.g., machine learning algorithms (paras. [10], [25], [27], [28], [29], [30], & [70]);
a “reinforcement learning module” in claim 30, i.e., hardware, e.g., a computing device (para. [24]); and software, e.g., machine learning algorithms (paras. [10], [25], [27], [28], [29], [30], & [70]); and
a “retrosynthesis prediction module” in claim 38, i.e., hardware, e.g., a computing device (para. [24]); and software, e.g., machine learning algorithms (paras. [10], [25], [27], [28], [29], [30], & [70]).
If applicant does not intend to have these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Objections
Claim 23 is objected to because of the following informalities: The word “one” should be inserted between the word “least” and the word “reaction” in line seven.
Claim 40 is objected to because of the following informalities: A hyphen should be inserted between the word “computer” in line eleven and the word “implemented” in line twelve.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 37 and 39 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 37 recites the limitation "the set of probable products" in lines two and three. There is insufficient antecedent basis for this limitation in the claim. Claim 37 is further indefinite because it is not clear as to whether the limitation “the set of probable products” is referring to “the set of probable reaction products” as recited in lines four and five of claim 21.
Claim 39 is indefinite for depending from claim 37 and for failing to remedy the indefiniteness of claim 37.
Claim 39 recites the limitation "the at least one predicted property" in line one. There is insufficient antecedent basis for this limitation in the claim, because claim 37 recites “at least one predicted or experimental property.” As alternatives and is not a necessary requirement.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 21-40 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite: (a) mathematical concepts, (e.g., mathematical relationships, formulas or equations, mathematical calculations); and (b) mental processes, i.e., concepts performed in the human mind, (e.g., observation, evaluation, judgement, opinion).
Claim Interpretations
Claims 23, 24, 25, and 40 recite the limitation “computer-implemented agent.” The broadest reasonable interpretation of this limitation is a software program.
Claims 21, 23, 31, 38, and 40 recite the limitation “artificial intelligence environment.” The broadest reasonable interpretation of this limitation is a computational framework for virtually designing molecules.
Claim 28 recites the limitation “a reinforcement learning model which is trained to imitate an output of a genetic algorithm.” This limitation is interpreted to recite a product-by-process limitation with the product being the trained model, and further interpreted to not require active steps of performing the process to train the model.
Claim 29 recites the limitation “at least one actor and/or at least one critic module is/are trained based on an output of the genetic algorithm.” This limitation is interpreted to recite a product-by-process limitation with the product being the trained model, and further interpreted to not require active steps of performing the process to train the model.
Claim 30 recites the limitation “a planning method or a reinforcement learning module trained to imitate a planning method is employed….” This limitation is interpreted to recite a product-by-process limitation with the product being the trained model, and further interpreted to not require active steps of performing the process to train the model.
Subject matter eligibility evaluation in accordance with MPEP 2106.
Eligibility Step 1: Step 1 of the eligibility analysis asks: Is the claim to a process, machine, manufacture or composition of matter?
Claims 21-39 recite a system for automated design of molecules (i.e., a machine or manufacture); and claim 40 recites a method for automated design of molecules (i.e., a process).
Therefore, these claims are encompassed by the categories of statutory subject matter, and thus, satisfy the subject matter eligibility requirements under step 1.
[Step 1: YES]
Eligibility Step 2A: First it is determined in Prong One whether a claim recites a judicial exception, and if so, then it is determined in Prong Two whether the recited judicial exception is integrated into a practical application of that exception.
Eligibility Step 2A Prong One: In determining whether a claim is directed to a judicial exception, examination is performed that analyzes whether the claim recites a judicial exception, i.e., whether a law of nature, natural phenomenon, or abstract idea is set forth or described in the claim.
Independent claim 21 recites the following steps which fall within the mental processes and/or mathematical concepts groupings of abstract ideas:
an artificial intelligence environment comprising a chemical reaction prediction module and a scoring function module (i.e., software) (i.e., mental processes and mathematical concepts);
wherein the artificial intelligence environment predicts a set of probable reaction products based on at least one reaction involving at least one reactant (i.e., mental processes and mathematical concepts); and
the artificial intelligence environment scores the set of probable reaction products based on a desired metric (i.e., mental processes and mathematical concepts).
Independent claim 40 recites the following steps which fall within the mental processes and/or mathematical concepts groupings of abstract ideas:
using an agent to generate at least one reaction involving at least one reactant (i.e., mental processes);
simulating, in the artificial intelligence environment, the at least one reaction involving at least one reactant to generate a set of at least one probable reaction product (i.e., mental processes);
scoring the set of at least one probable reaction product according to a desired property (i.e., mental processes and mathematical concepts); and
generating a set of optimal reaction products selected from the set of at least one probable reaction product and passing the set of optimal reaction products to the agent to serve as a new set of reactants (i.e., mental processes);
wherein the method is terminated when the set of optimal reaction products contains a desired final product (i.e., mental processes).
Dependent claims 22-39 further recite the following steps which fall within the mental processes and/or mathematical concepts groupings of abstract ideas, as noted below.
Dependent claim 22 further recites:
the approximation module identifies a closest set of reactants from a set of all available reactants based on a distance in a compatible metric space (i.e., mental processes and mathematical concepts).
Dependent claim 23 further recites:
the agent operates according to a reinforcement learning process and comprises at least one actor module (i.e., mental processes and mathematical concepts); and
the agent interfaces with the artificial intelligence environment through the reinforcement learning process by providing to the artificial intelligence environment the at least reaction involving at least one reactant for the purpose of simulating a reaction and/or an action in the space of reactants (i.e., mental processes and mathematical concepts).
Dependent claim 24 further recites:
the agent further comprises at least one critic module which is used to evaluate an output of the at least one actor module (i.e., mental processes).
Dependent claim 25 further recites:
the approximation module is differentiable and is part of an agent, so that the approximation module may update at least one of an actor network and a critic network based on an output of the critic network by propagating a gradient through the approximation module (i.e., mental processes and mathematical concepts).
Dependent claim 26 further recites:
an initial reactant is sampled randomly, is sampled by using a statistical metric, or is sampled by using a network whose output is evaluated by a critic module (i.e., mental processes and mathematical concepts).
Dependent claim 27 further recites:
the at least one reaction involving at least one reactant are selected through a proto-action generated by a genetic algorithm (i.e., mental processes).
Dependent claim 28 further recites:
the at least one reaction involving at least one reactant are selected by a reinforcement learning model which is trained to imitate an output of a genetic algorithm (i.e., mental processes and mathematical concepts).
Dependent claim 29 further recites:
at least one actor and/or at least one critic module is/are trained based on an output of the genetic algorithm (i.e., mental processes).
Dependent claim 30 further recites:
a planning method or a reinforcement learning module trained to imitate a planning method is employed to compute at least one action at every time step (i.e., mental processes and mathematical concepts).
Dependent claim 31 further recites:
the artificial intelligence environment further uses at least one reaction condition in predicting the set of probable reaction products (i.e., mental processes).
Dependent claim 32 further recites:
the set of probable reaction products serves as the at least one reactant of a subsequent reaction (i.e., mental processes).
Dependent claim 33 further recites:
the at least one reactant comprises a tensor in a space defined by features of a set of all available reactants (i.e., mental processes and mathematical concepts).
Dependent claim 34 further recites:
a critic module evaluates an output of the at least one actor module for the purpose of choosing a reactant (i.e., mental processes).
Dependent claim 35 further recites:
the chemical reaction prediction module predicts at least one probable reaction product on the basis of at least one of: a rule-based algorithm, a physics-based algorithm, a quantum mechanical algorithm, a machine-learning algorithm, and a hybrid quantum machine-learning algorithm (i.e., mental processes and mathematical concepts).
Dependent claim 36 further recites:
the chemical reaction prediction module predicts the set of at least one probable reaction products on the basis of an N-component transformation (i.e., mental processes and mathematical concepts).
Dependent claim 37 further recites:
the scoring function module determines a reward according to at least one predicted or experimental property of the set of probable products (i.e., mental processes and mathematical concepts).
Dependent claim 38 further recites:
the artificial intelligence environment uses a retrosynthesis prediction module based on at least one of: a rule-based algorithm, a quantum mechanical algorithm, a physics-based algorithm, a machine-learning algorithm and a hybrid quantum machine-learning algorithm to evaluate a synthesis process (i.e., mental processes and mathematical concepts).
Dependent claim 39 further recites:
the at least one predicted property is determined by at least one of: a rule-based algorithm, a quantum mechanical algorithm, a physics-based algorithm, a machine-learning algorithms, and a hybrid quantum machine-learning algorithm (i.e., mental processes and mathematical concepts).
The abstract ideas recited in the claims are evaluated under the broadest reasonable interpretation (BRI) of the claim limitations when read in light of and consistent with the specification. As noted in the foregoing section, the claims are determined to contain limitations that can practically be performed in the human mind with the aid of a pen and paper (e.g., predict a set of probable reaction products based on at least one reaction involving at least one reactant), and therefore recite judicial exceptions from the mental process grouping of abstract ideas. Additionally, the recited limitations that are identified as judicial exceptions from the mathematical concepts grouping of abstract ideas (e.g., score the set of probable reaction products based on a desired metric) are abstract ideas irrespective of whether or not the limitations are practical to perform in the human mind.
Therefore, claims 21-40 recite an abstract idea.
[Step 2A Prong One: YES]
Eligibility Step 2A Prong Two: In determining whether a claim is directed to a judicial exception, further examination is performed that analyzes if the claim recites additional elements that when examined as a whole integrates the judicial exception(s) into a practical application (MPEP 2106.04(d)). A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. The claimed additional elements are analyzed to determine if the abstract idea is integrated into a practical application (MPEP 2106.04(d)(I); MPEP 2106.05(a-h)). If the claim contains no additional elements beyond the abstract idea, the claim fails to integrate the abstract idea into a practical application (MPEP 2106.04(d)(III)).
The judicial exceptions identified in Eligibility Step 2A Prong One are not integrated into a practical application because of the reasons noted below.
Dependent claims 22 and 26-39 do not recite any elements in addition to the judicial exception, and thus are part of the judicial exception.
The additional elements in independent claim 21 include:
a system for automated design of molecules (i.e., a computer and/or computer-related components).
The additional elements in independent claim 40 include:
a computer; and
providing, by the computer-implemented agent, the at least one reaction involving at least one reactant to an artificial intelligence environment (i.e., outputting and/or inputting data).
The additional elements in dependent claims 23-25 include:
a computer (claims 23, 24, and 25).
The additional elements of a computer and/or computer-related components (claim 21); and a computer (claims 23, 24, 25, and 40); invoke a computer and/or computer-related components merely as tools for use in the claimed process, and therefore are not an improvement to computer functionality itself, or an improvement to any other technology or technical field, and thus, do not integrate the judicial exceptions into a practical application (MPEP 2106.04(d)(1)).
The additional element of outputting and/or inputting data (claim 40) is merely a pre-solution activity of gathering data for use in the claimed process – a nominal addition to the claims that does not meaningfully limit the claims, and therefore does not add more than insignificant extra-solution activity to the judicial exceptions (MPEP 2106.05(g)).
Thus, the additionally recited elements merely invoke a computer and/or computer related components as tools; and/or amount to insignificant extra-solution activity; and as such, when all limitations in claims 21-40 have been considered as a whole, the claims are deemed to not recite any additional elements that would integrate a judicial exception into a practical application, and therefore claims 21-40 are directed to an abstract idea (MPEP 2106.04(d)).
[Step 2A Prong Two: NO]
Eligibility Step 2B: Because the claims recite an abstract idea, and do not integrate that abstract idea into a practical application, the claims are probed for a specific inventive concept. The judicial exception alone cannot provide that inventive concept or practical application (MPEP 2106.05). Identifying whether the additional elements beyond the abstract idea amount to such an inventive concept requires considering the additional elements individually and in combination to determine if they amount to significantly more than the judicial exception (MPEP 2106.05A i-vi).
The claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception(s) because of the reasons noted below.
Dependent claims 22 and 26-39 do not recite any elements in addition to the judicial exception(s).
The additional elements recited in independent claims 21 and 40 and dependent claims 23, 24, and 25 are identified above, and carried over from Step 2A Prong Two along with their conclusions for analysis at Step 2B. Any additional element or combination of elements that was considered to be insignificant extra-solution activity at Step 2A Prong Two was re-evaluated at Step 2B, because if such re-evaluation finds that the element is unconventional or otherwise more than what is well-understood, routine, conventional activity in the field, this finding may indicate that the additional element is no longer considered to be insignificant; and all additional elements and combination of elements were evaluated to determine whether any additional elements or combination of elements are other than what is well-understood, routine, conventional activity in the field, or simply append well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, per MPEP 2106.05(d).
The additional elements of a computer and/or computer-related components (claim 21); a computer (claims 23, 24, 25, and 40); and outputting and/or inputting data (claim 40); are conventional computer components and/or functions (see MPEP at 2106.05(b) and 2106.05(d)(II) regarding conventionality of computer components and computer processes).
Therefore, when taken alone, all additional elements in claims 21-40 do not amount to significantly more than the above-identified judicial exception(s). Even when evaluated as a combination, the additional elements fail to transform the exception(s) into a patent-eligible application of that exception. Thus, claims 21-40 are deemed to not contribute an inventive concept, i.e., amount to significantly more than the judicial exception(s) (MPEP 2106.05(II)).
[Step 2B: NO]
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21 and 40 are rejected under 35 U.S.C. 103 as being unpatentable over Coley et al. (“Prediction of Organic Reaction Outcomes Using Machine Learning.” ACS Central Science, (2017), Vol. 3, pp. 434-443) in view of Raschka et al. (“Machine learning and AI-based approaches for bioactive ligand discovery and GPCR-ligand recognition.” arXiv:2001.06545v2 [q-bio.BM], (22 January 2020), pp. 1-24).
Independent claim 21 is broadly directed to a system for automating the design of molecules comprising an artificial intelligence environment comprising software algorithms for predicting probable chemical reaction products based on at least one reaction involving at least one reactant and then scoring the probable reaction products based in a desired metric.
Independent claim 40 is broadly directed to a method for automating the design of molecules comprising using a computer-implemented agent to generate at least one reaction involving at least one reactant, providing the at least one reaction involving at least one reactant to an artificial intelligence environment, simulating, in the artificial intelligence environment, the at least one reaction involving at least one reactant to generate a set of at least one probable reaction product, scoring the set of at least one probable reaction product according to a desired property, generating a set of optimal reaction products selected from the set of at least one probable reaction product and passing the set of optimal reaction products to the computer implemented agent to serve as a new set of reactants, wherein the method is terminated when the set of optimal reaction products contains a desired final product.
Coley et al. is directed to the prediction of organic reaction outcomes using machine learning, and report a model framework for anticipating reaction outcomes that combines the traditional use of reaction templates with the flexibility in pattern recognition afforded by neural networks. Candidate reactions are represented using a representation that emphasizes the fundamental transformation from reactants to products, rather than the constituent molecules’ overall structure (Abstract).
Raschka et al. is directed to a review of AI-based research for GPCR bioactive ligand discovery with a particular focus on the most recent achievements and research trends, including deep learning methods and reinforcement learning-based models for molecular design (Abstract; and throughout; and in particular Section 5.2.).
Regarding independent claims 21 and 40, Coley et al. shows a model framework that predicts the outcome of a chemical reaction in a two-step manner: (1) applying overgeneralized forward reaction templates to a pool of reactants to generate a set of chemically plausible products, and (2) estimating which candidate product is the major product as a multiway classification problem using machine learning (page 435, col. 2, para. 2; and Figure 1); a neural network design motivated by the likelihood of a reaction being a function of the atom/bond changes that are required for it to occur (page 437, col. 1, para. 3); the neural network model can evaluate any candidate reaction, even if the corresponding template and/or substrates have never been seen before (page 441, col. 2, para. 1); and model architecture for scoring candidate reactions (Figure 3).
Regarding independent claims 21 and 40, Coley et al. does not show using an artificial intelligence environment for automating the design of molecules based on a desired metric; wherein the method is terminated when the set of optimal reaction products contains a desired final product.
Regarding independent claims 21 and 40, Raschka et al. shows reinforcement learning-based models for molecular design using artificial intelligence environments (page 16, Section 5.2., paras. 1 & 2); and a representation of a basic reinforcement learning paradigm with a simple molecular example comprising an agent and an artificial intelligence environment that repeats until the episode (i.e., the series of iterations) terminates (page 17, Figure 5).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method shown by Coley et al. by incorporating methods for using reinforcement learning-based models for molecular design using artificial intelligence environments, as shown by Raschka et al. and discussed above. One of ordinary skill in the art would have been motivated to combine the methods of Coley et al. with the methods of Raschka et al., because Raschka et al. shows using reinforcement learning models for generating both novel and chemically valid compounds (page 16, Section 5.2., para. 2). This modification would have had a reasonable expectation of success given that both Coley et al. and Raschka et al. disclose using machine learning methods for molecular prediction and discovery.
Claims 22-39 are rejected under 35 U.S.C. 103 as being unpatentable over Coley et al. in view of Raschka et al. as applied to claims 21 and 40 above, and further in view of Stahl et al. (“Deep Reinforcement Learning for Multiparameter Optimization in de novo Drug Design.” Journal of Chemical Information and Modeling, (2019), Vol. 59, pp. 3166-3176).
Dependent claims 22-39 further define the steps of the claimed process that generally utilize reinforcement learning to teach an AI agent to perform actions that will maximize a cumulative reward over a series of iterations, i.e., determining a desirable product corresponding to the maximum reward, and choosing this product (e.g., by using this product as the at least one reactant of a subsequent reaction).
Stahl et al. is directed to deep reinforcement learning for multiparameter optimization in de novo drug design using an actor-critic model (Abstract).
Regarding dependent claims 22-39, Coley et al. in view of Raschka et al. as applied to claims 21 and 40 above, does not show specific aspects of using a computer-implemented agent in a reinforcement learning-based artificial intelligence environment, in particular the use of an actor-critic model.
Regarding dependent claims 22-39, Stahl et al. shows using deep reinforcement learning for multiparameter optimization in de novo drug design (Title); and further shows using a reinforcement learning approach based on an actor-critic model for the generation of novel molecules with optimal properties (Abstract). Stahl et al. further shows that reinforcement learning is a subfield of machine learning concerned with how agents take actions in an environment to maximize some notion of reward (page 3168, col. 1, para. 6); and further shows that actor-critic models make use of temporal difference learning by having a critic model that evaluates the behavior of the agent and suggests whether the agent performed well or not (page 3168, col. 2, para. 6). Stahl et al. further shows applying the actor-critic framework to a fragmented molecule (the current state) with the actor examining all fragments, deciding which of them to replace and with what fragment, with the new state being scored with a reward depending on how well it satisfies all constraints, and then the critic subsequently examines the difference between the reward added to the value of the new state and the value of the current state, with this difference, the TD-Error, being given to the actor. If this error is positive, the action of the actor will be reinforced, and if negative, the action will be discouraged. The current state is then replaced by the new state, and the process is repeated a given number of times (Figure 3).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method shown by Coley et al. in view of Raschka et al. as applied to claims 21 and 40 above, by incorporating an actor-critic model, as shown by Stahl et al. and discussed above. One of ordinary skill in the art would have been motivated to combine the methods of Coley et al. in view of Raschka et al. as applied to claims 21 and 40 above, with the methods of Stahl et al., because Stahl et al. shows a reinforcement learning approach to de novo drug design that is based on an actor-critic model for the generation of novel molecules (page 3167, col. 2, para. 2). This modification would have had a reasonable expectation of success given that both Coley et al. in view of Raschka et al. as applied to claims 21 and 40 above, and Stahl et al. disclose using machine learning methods, in particular deep reinforcement learning, that utilize an agent, with the capability to learn, that interacts with the environment.
Conclusion
No claims are allowed.
This Office action is a Non-Final action. A shortened statutory period for reply to this action is set to expire THREE MONTHS from the mailing date of this application.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN W. BAILEY whose telephone number is (571)272-8170. The examiner can normally be reached Mon - Fri. 1000 - 1800.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KARLHEINZ SKOWRONEK can be reached at (571) 272-9047. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.W.B./Examiner, Art Unit 1687
/Joseph Woitach/Primary Examiner, Art Unit 1687