DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are under examination.
This application claims priority to a US Provisional application, filed 11/3/2022. The effective filing date for the claims under examination is 11/3/2022.
This application has published as US PG Pub 2024/0152668 A1.
The Drawings as filed are suitable for examination.
The listing of references in the specification is not a proper information disclosure statement. 37 CFR 1.98(b) requires a list of all patents, publications, or other information submitted for consideration by the Office, and MPEP § 609.04(a) states, "the list may not be incorporated into the specification but must be submitted in a separate paper." Therefore, unless the references have been cited by the examiner on form PTO-892, they have not been considered.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation (BRI) using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of mental steps, mathematic concepts, organizing human activity, or a natural law without significantly more.
Applicant is directed to MPEP 2106 for the most current and complete guidelines in the analysis of patent- eligible subject matter. The current MPEP is the primary source for the USPTO’s patent eligibility guidance.
With respect to step (1): YES, the claims are drawn to statutory categories: Claims 1-19 are drawn to methods, claim 20 is drawn to a non-transitory computer readable media comprising instructions.
With respect to step (2A) (1): YES, the claims recite an abstract idea, law of nature and/or natural phenomenon. The claims explicitly recite elements that, individually and in combination, constitute one or more judicial exceptions (JE).
Mathematic concepts, Mental Processes or Elements in Addition (EIA) in the claim(s) include:
Claim 1. A method for generating a three-dimensional (3D) conformation of a molecule, the method comprising:
(Preamble indicating a method, and the goal of the method.)
obtaining a two-dimensional (2D) graph of the molecule;
(EIA- a step of data gathering and a description of the data gathered. MPEP 2106.05g)
generating a first tensor comprising molecular graph information and a second tensor comprising graph, coordinate, and distance information of atoms of the molecule;
(Mathematic concept: generating a tensor is a step of data transformation from a 2D or chemical structure into a multidimensional numerical array. MPEP 2106.04(a)(2).)
generating a first set of feature vectors corresponding to the first tensor and a second set of feature vectors corresponding to the second tensor;
(Mathematic concept: generating feature vectors from a tensor is a mathematic process of data transformation from the multidimensional array of the previous step to a mathematic vector. MPEP 2106.04(a)(2))
feeding the first set of feature vectors to a first encoder of a generative model to generate a first output, and
(Mathematic concept of applying the numerical vector to an encoder, which is a step of data transformation. MPEP 2106.04(a)(2).)
feeding the second set of feature vectors to a second encoder of the generative model to generate a second output; and
(Mathematic concept of applying the numerical vector to an encoder, which is a step of data transformation. MPEP 2106.04(a)(2).)
combining the first output and the second output in a decoder of the generative model to generate the 3D conformation of the molecule.
(Mathematic Concept: Applying the output of the encoders to a decoder is a mathematic step of data transformation, MPEP 2106.04(a)(2).)
2. The method of claim 1, wherein generating the first tensor comprises adding an additional dimension to an adjacency matrix including one or more atom features for the atoms of the molecule and one or more bond features for bonds, corresponding to atom pairs, included in the molecule. (Mathematic concept: generating a tensor is a step of data transformation from a chemical structure into a multidimensional numerical array. MPEP 2106.04(a)(2).)
3. The method of claim 2, wherein a diagonal section of the first tensor holds the one or more atom features for the atoms of molecule, and off-diagonal sections of the first tensor hold the one or more bond features for the bonds included in the molecule. (Mathematic concept: generating a tensor is a step of data transformation from a chemical structure into a multidimensional numerical array. MPEP 2106.04(a)(2).)
4. The method of claim 2, wherein the one or more atom features for an atom included in the molecule comprise one or more of an atom type, atom charge, or atom chirality for the atom. (EIA- a step of data gathering and a description of the data gathered. MPEP 2106.05g)
5. The method of claim 2, wherein the one or more bond features for a bond included in the molecule comprise one or more of a bond type, a bond stereochemistry type, an associated ring size, or a normalized bond length for the bond. (EIA- a step of data gathering and a description of the data gathered. MPEP 2106.05g)
6. The method of claim 5, wherein the bond type comprises a virtual bond. (EIA- a step of data gathering and a description of the data gathered. MPEP 2106.05g)
7. The method of claim 6, wherein the first tensor represents a fully connected symmetric tensor in which virtual bonds are extended across all atoms of the molecule. (Mathematic concept: generating a tensor is a step of data transformation from a chemical structure into a multidimensional numerical array. MPEP 2106.04(a)(2).)
8. The method of claim 2, wherein generating the first set of feature vectors comprises generating a set of atom feature vectors for the atoms of the molecule and a set of bond feature vectors for the bonds included in the molecule. (Mathematic concept: generating feature vectors from a tensor is a mathematic process of data transformation from the multidimensional array of the previous step to a mathematic vector. MPEP 2106.04(a)(2))
9. The method of claim 8, wherein an atom feature vector for an atom is generated by stacking the one or more atom features for the atom included in the molecule, wherein each of the one or more atom features for the atom is one-hot encoded. (Mathematic concept: generating feature vectors from a tensor is a mathematic process of data transformation from the multidimensional array of the previous step to a mathematic vector. MPEP 2106.04(a)(2))
10. The method of claim 8, wherein a bond feature vector for a bond is generated by:summing atom feature vectors for a pair of atoms associated with the bond to generate a new vector; and stacking the new vector with one-hot encoded bond type vector, normalized bond length, and one-hot encoded ring size vector associated with the bond. (Mathematic concept: generating feature vectors from a tensor is a mathematic process of data transformation from the multidimensional array of the previous step to a mathematic vector. MPEP 2106.04(a)(2))
11. The method of claim 8, wherein generating the second tensor comprises adding a coordinate channel to a respective atom feature vector and adding a Euclidean distance channel to a respective bond feature vector. (Mathematic concept: generating a tensor is a step of data transformation from a chemical structure into a multidimensional numerical array. MPEP 2106.04(a)(2).)
12. The method of claim 1, wherein generating the first set of feature vectors comprises encoding the first tensor using a one-dimensional (1D) convolution operation, and generating the second set of feature vectors comprises encoding the second tensor using the 1D convolution operation. (Mathematic concept: generating feature vectors from a tensor is a mathematic process of data transformation from the multidimensional array of the previous step to a mathematic vector. MPEP 2106.04(a)(2))
13. The method of claim 12, wherein using the 1D convolution operation comprises adjusting a kernel height or width of a kernel to equal a height or width of the first tensor or the second tensor during the 1D convolution operation. (Mathematic concept: convolution operations are mathematic data transformation. MPEP 2106.04(a)(2))
14. The method of claim 13, wherein the kernel height or width of the first tensor or the second tensor remains unchanged and equals a kernel size of the kernel during the 1D convolution operation. (Mathematic concept: convolution operations are mathematic data transformation. MPEP 2106.04(a)(2))
15. The method of claim 14, wherein the kernel size of the kernel after adjusting the kernel length is Nx3 kernel or Nx4 kernel, wherein N is the length of the first tensor or the second tensor. (Mathematic concept: convolution operations are mathematic data transformation. MPEP 2106.04(a)(2))
16. The method of claim 1, wherein combining the first output and the second output in the decoder of the generative model to generate the 3D conformation of the molecule comprises: computing, in a first multi-head attention layer of the decoder, query and key matrices based on the first output. (Mathematic Concept: Applying the output of the encoders to a decoder is a mathematic step of data transformation, MPEP 2106.04(a)(2).)
17. The method of claim 1, wherein combining the first output and the second output in the decoder of the generative model to generate the 3D conformation of the molecule 31 comprises: reparametrizing the second output to generate value matrices for a first multi-head attention layer of the decoder. (Mathematic Concept: Applying the output of the encoders to a decoder is a mathematic step of data transformation, MPEP 2106.04(a)(2).)
18. The method of claim 1, wherein the first encoder, the second encoder, and the decoder included in the generative model are arranged in a conditional variational autoencoder framework. (EIA- a data structure of a conditional variational autoencoder. MPEP 2106.05a)
19. The method of claim 1, wherein the generative model comprises a convolutional neural network. (EIA- a data structure of a CNN. MPEP 2106.05a)
20. A non-transitory computer readable medium comprising instructions that, when executed by a processor, cause the processor to:
(Preamble indicating a CRM, with instructions to carry out the method of claim 1. EIA- General purpose CRM, MPEP 2106.05h)
The rest of the analysis is the same as for claim 1.
With respect to step 2A (2): NO, the claims do not integrate the JE into a practical application (MPEP 2106.04(d)):
“Examiners evaluate integration into a practical application by: (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception(s); and (2) evaluating those additional elements individually and in combination to determine whether they integrate the exception into a practical application, using one or more of the considerations introduced in subsection I supra, and discussed in more detail in MPEP §§ 2106.04(d)(1), 2106.04(d)(2), 2106.05(a) through (c) and 2106.05(e) through (h).”
Claim(s) 1, 4-6, 20 recite the additional non-abstract element(s) of data gathering, or a description of the data gathered.
Data gathering steps are not an abstract idea, they are extra-solution activity, as they collect the data necessary to carry out the JE. MPEP 2106.05(g).
The data gathering does not impose any meaningful limitation on the JE, or how the JE is performed. MPEP 2106.05(g).
The data gathering steps constitute a general link to a technological environment: the trait prediction methods are intended to be applied to plant populations. (MPEP 2106.05(h), citing Mayo, Bilski, electric Power Group, Genetic Techs Ltd v Merial LLC.)
The additional limitation (data gathering) must have more than a nominal or insignificant relationship to the identified judicial exception to provide integration into a practical application. (MPEP 2106.05(g) citing Mayo, PerkinElmer, Inc. v. Interna Ltd, Intellectual Ventures LLC v. Erie Indem. Co., Electric Power Group LLC v. Alstom S.A.).
Claim(s) 18-20 recite the additional non-abstract element (EIA) of a general-purpose computer system or parts thereof.
The claims do not provide any details of how specific structures of the computer elements are used to implement the JE. MPEP 2106.05(a), contrasting decisions identifying how the computer implements an abstract idea, such as in McRo to decisions which found no specific interaction with the computer, such as in Affinity Labs of Tex v. DirecTV, LLC.
The computer elements of the claims do not provide improvements to the functioning of the computer itself. MPEP 2106.05(a) I, contrasting decisions indicating an improvement to the computer, such as DDR Holdings, LLC v. Hotels.com LP, with decisions that did not identify an improvement to the computer, such as FairWarning IP, LLC v. Iatrix Sys.
The computer elements of the claims do not provide improvements to any other technology or technical field. MPEP 2106.05(a) II: contrasting decisions indicating an improvement to the technology, such as Diamond v. Diehr, Trading Techs. Int’l v. CQG Inc, or Intellectual Ventures I v. Symantec Corp, with decisions that did not identify an improvement to the technology, such as Alice Corp, Versata Dev. Group, Inc. v. SAP AM. Inc, or TLI Communications.
The computer elements of the claims do not utilize a particular machine. MPEP 2106.05(b): contrasting decisions wherein a particular machine was identified, such as MacKay Radio & Tel. Co. v. Radio Corp. of America, Eibel Process Co. v. Minn. & Ont. Paper Co., with decisions where a general-purpose computer does not qualify as a particular machine, such as Ultramercial, Inc. v. Hulu, LLC, TLI communications, or Eon Corp. IP holdings LLC v. AT&T Mobility LLC.
Hence, these are mere instructions to apply the JE using a computer, and therefore the claim does not recite integrate that JE into a practical application.
Dependent claim(s) 2, 3, 7-17 recite(s) an abstract limitation to the JE reciting additional mathematic concepts, or mental processes. Additional abstract limitations cannot provide a practical application of the JE as they are a part of that JE.
In combination, the limitations of data gathering, for the purpose of carrying out the JE, using a general-purpose computer merely provide extra-solution activity, and fail to integrate the JE into a practical application.
With respect to step 2B: NO, the claims do not recite a specific inventive concept. The judicial exception alone cannot provide that inventive concept or practical application (MPEP 2106.05).
“… an "inventive concept" is furnished by an element or combination of elements that is recited in the claim in addition to (beyond) the judicial exception, and is sufficient to ensure that the claim, as a whole, amounts to significantly more than the judicial exception itself. Alice Corp…”
With respect to claim(s) 1, 4-6, 20: The limitation(s) identified above as non-abstract elements (EIA) related to data gathering do not rise to the level of significantly more than the judicial exception.
Duan, J. (2010) Analysis and comparison of 2D fingerprints: Insights into database screening performance using eight fingerprint methods. Journal of Molecular Graphics and Modeling, vol 29, p157-170. Duan provides 2D graphs of molecules in various fingerprinting types.
Stierand, K et al. (2010) Drawing the PDB: Protein-ligand complexes in two dimensions. ACS Medicinal Chemistry Letters, vol 1, p540-545. Stierand provides 2D graphs of various molecules and complexes.
O’Boyle (2012) Towards a Universal SMILES representation- A standard method to generate canonical SMILES based on the InChI. Journal of Cheminformatics vol 4:22 14 pages. O’Boyle obtains 2D graphs of various molecules in InChi and SMILES.
David, L. et al. (2020) Molecular representations in AI-driven drug discovery: a review and practical guide. Journal of Cheminformatics, vol 12: 56, 22 pages. David provides databases comprising 2D graphs of various molecules.
Toti, D. et al. (2022) FGDB: a comprehensive graph database of ligand fragments from the Protein Data Bank. Database, vol 2022, baac044, p1-12. Toti provides 2D graphs of various molecules in SMILES format.
These elements meet the BRI of the identified data gathering limitations. As such, the prior art recognizes that this data gathering element is routine, well understood and conventional in the art. MPEP 2106.05(d): “If, however, the additional element (or combination of elements) is no more than well-understood, routine, conventional activities previously known to the industry, which is recited at a high level of generality, then this consideration does not favor eligibility.”
In the specification at [0023] it is disclosed that the steps identified as data gathering can be met by publicly available third party databases such as protein data bank.
Data gathering steps are not an abstract idea, they are extra-solution activity, as they collect the data necessary to carry out the JE. MPEP 2106.05(g).
The data gathering does not impose any meaningful limitation on the JE, or how the JE is performed. MPEP 2106.05(g).
The additional limitation (data gathering) must have more than a nominal or insignificant relationship to the identified judicial exception to provide an inventive concept. (MPEP 2106.05(g) citing Mayo, PerkinElmer, Inc. v. Interna Ltd, Intellectual Ventures LLC v. Erie Indem. Co., Electric Power Group LLC v. Alstom S.A.)
The data gathering steps constitute a general link to a technological environment: the trait prediction methods are intended to be applied to plant populations. (MPEP 2106.05(h), citing Mayo, Bilski, electric Power Group, Genetic Techs Ltd v Merial LLC.)
Therefore, simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception are insufficient to provide significantly more (as discussed in Alice Corp.,).
With respect to claim(s) 18-20: the limitations identified above as non-abstract elements (EIA) related to general-purpose computer systems do not rise to the level of significantly more than the judicial exception.
Claims 18-19 set forth generic data structures: variational autoencoders and convolutional neural networks. Claim 20 requires non-transitory compute readable media.
Feinberg et al. (WO 2019/173401 A1) discloses VAE and CRM in methods of processing 2D graphs of molecules.
Bucher et al. (US 11,256,995 B2) discloses VAE, CNN and CRM in methods of producing 3D visualizations.
Hamp et al. (US 11,515,010 B2) discloses VAE, CNN and CRM in methods of predicting 3D protein structures.
As such, the prior art recognizes that these computing elements are routine, well understood and conventional in the art.
The specification, at [0055-0063] discloses the use of routine general-purpose computers for carrying out the invention, and/or the use of commercially available computer system elements.
The claims do not provide any details of how specific structures of the computer elements are used to implement the JE. MPEP 2106.05(a), contrasting decisions identifying how the computer implements an abstract idea, such as in McRo to decisions which found no specific interaction with the computer, such as in Affinity Labs of Tex v. DirecTV, LLC.
The computer elements of the claims do not provide improvements to the functioning of the computer itself. MPEP 2106.05(a) I, contrasting decisions indicating an improvement to the computer, such as DDR Holdings, LLC v. Hotels.com LP, with decisions that did not identify an improvement to the computer, such as FairWarning IP, LLC v. Iatrix Sys.
The computer elements of the claims do not provide improvements to any other technology or technical field. MPEP 2106.05(a) II: contrasting decisions indicating an improvement to the technology, such as Diamond v. Diehr, Trading Techs. Int’l v. CQG Inc, or Intellectual Ventures I v. Symantec Corp, with decisions that did not identify an improvement to the technology, such as Alice Corp, Versata Dev. Group, Inc. v. SAP AM. Inc, or TLI Communications.
The computer elements of the claims do not utilize a particular machine. MPEP 2106.05(b): contrasting decisions wherein a particular machine was identified, such as MacKay Radio & Tel. Co. v. Radio Corp. of America, Eibel Process Co. v. Minn. & Ont. Paper Co., with decisions where a general-purpose computer does not qualify as a particular machine, such as Ultramercial, Inc. v. Hulu, LLC, TLI communications, or Eon Corp. IP holdings LLC v. AT&T Mobility LLC.
Hence, these are mere instructions to apply the JE using a computer, and therefore the claim does not provide significantly more.
Dependent claim(s) 2, 3, 7-17 each recite a limitation requiring additional mathematic concepts or mental processes. Additional abstract limitations cannot provide significantly more than the JE as they are a part of that JE (MPEP 2106.05).
In combination, the data gathering steps providing the information required to be acted upon by the JE, performed in a generic computer or generic computing environment fail to rise to the level of significantly more than that JE. The data gathering steps provide the data for the JE, which is carried out by the general-purpose computers. No non-routine step or element has clearly been identified.
The claims have all been examined to identify the presence of one or more judicial exceptions. Each additional limitation in the claims has been addressed, alone and in combination, to determine whether the additional limitations integrate the judicial exception into a practical application. Each additional limitation in the claims has been addressed, alone and in combination, to determine whether those additional limitations provide an inventive concept which provides significantly more than those exceptions. For these reasons, the claims, when the limitations are considered individually and as a whole, are rejected under 35 USC § 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-12, 18-20 is/are rejected under 35 U.S.C. 102a1 as being anticipated by Feinberg (2019).
Feinberg, E. N. et al. Systems and methods for spatial graph convolutions with applications to drug discovery and molecular simulation. (WO 2019/173401 A1) published 9/12/2019.
Feinberg is directed to:
“… the method includes performing a first set of graph convolutions with a spatial graph representation of a set of molecules, wherein the first set of graph convolutions are based on bonds between the set of molecules, performing a second set of graph convolutions with the spatial graph representation, wherein the second set of graph convolutions are based on at least a distance between each atom and other atoms of the set of molecules, performing a graph gather with the spatial graph representation to produce a feature vector, and predicting a set of one or more characteristics for the set of molecules based on the feature vector.” (Abstract)
With respect to claim 1 and: “Claim 1. A method for generating a three-dimensional (3D) conformation of a molecule, the method comprising: obtaining a two-dimensional (2D) graph of the molecule;”
Feinberg obtains 2D graphs of a variety of molecules, as set forth at least at: [0070, 0071, 0075, et al.].
“[0075] … Process 500 receives (505) atomic information for one or more molecules in a system. In a variety of embodiments, the molecules can include a source (e.g., ligand) and a target molecule. Atomic information in accordance with several embodiments of the invention includes various information regarding each molecule, including (but not limited to) distance information and chemical properties. In numerous embodiments, atom types— the 1 x .sub.j¾ per-atom feature maps— are derived from the same initial features for both ligand and protein atoms. Process 500 builds (510) a spatial graph representation of the ligand and the target molecule.”
With respect to claim 1 and “generating a first tensor comprising molecular graph information and a second tensor comprising graph, coordinate, and distance information of atoms of the molecule;”
Feinberg generates tensors, as set forth at least at [0007, 0052, 0056-57, 0081, 0092, 0125 et al.]
“[0092] In a variety of embodiments, building the spatial graph representation can comprise generating a distance matrix and an adjacency tensor wherein the distance matrix denotes distances between atoms of the set of molecules and the adjacency tensor indicates a plurality of different edge types between atoms. Edge types can include but are not limited to: covalent bonds, ionic bonds, polar bonds, metallic bonds, non-covalent bonds (e.g. p— p stacking), salt bridge, distance bins (hard cutoff and/or expanded in a Gaussian and/or other basis set), and/or hydrogen bonds.”
“[0125] … In some embodiments, a neural network is trained. Neural networks in accordance with several embodiments of the invention can take as input initial feature tensors (x,A,R) and compute the parameters in (16) to most accurately compute potential energy, where x is an N x /)„ matrix where each of the N atoms is represented by a row vector of f.sub.jn features; A is an N x N x N.sub.et tensor of adjacency matrices among N.sub.et edge types (bond types, noncovalent interactions, etc.), and R is an N x N distance matrix.”
With respect to claim 1 and “generating a first set of feature vectors corresponding to the first tensor and a second set of feature vectors corresponding to the second tensor;”
Feinberg generates feature vectors corresponding to the tensors, at least at [0004, 0056-0057, 0060, 0077, 0125 et al.]
“[0077] In a number of embodiments, each graph convolution stage produces feature vectors for each atom, learning features for each atom based on the inputs to each stage. Feature vectors in accordance with a variety of embodiments of the invention can be used as inputs to a subsequent graph convolution stage. Each graph convolution stage in accordance with various embodiments of the invention can utilize a set of neural networks. In numerous embodiments, each a separate neural network is trained for each edge type. In many embodiments, neural networks can include, but are not limited to, fully connected neural networks, convolutional networks, and recurrent networks. Neural networks for a particular edge type can be used across the different stages in accordance with various embodiments of the invention in order to tie the weights of the different stages together. However, in many embodiments, a different neural network is trained and utilized for each edge type and stage.”
With respect to claim 1 and “feeding the first set of feature vectors to a first encoder of a generative model to generate a first output, and feeding the second set of feature vectors to a second encoder of the generative model to generate a second output; and”
Feinberg uses feature vectors as input to a generative model, which can be a variational autoencoder, to generate outputs, at least at [0052-0053, 0081, 0120, 0136]
With respect to claim 1 and “combining the first output and the second output in a decoder of the generative model to generate the 3D conformation of the molecule.”
Feinberg gathers and combines the various outputs in graph gather step and generates a 3D conformation. [0057, 0061, 0078, 0090 et al.]
“[0057] … In various embodiments, a graph gather is performed in a third stage (3). Processes in accordance with numerous embodiments of the invention perform the graph gather solely on the ligand atoms.”
“[0062] … In addition, other types of graph gather neural networks can be used in accordance with some embodiments of the invention, such as replacing the ReLU nonlinearity above with other nonlinearities like sigmoid, Leaky ReLU, tanh, etc., and other operations besides fully connected neural networks can be used, such as recurrent neural networks”
“[0078] Process 500 determines (520) whether there are more stages to perform. In many embodiments, a specified number of stages are performed. When the process determines that there are more stages to perform, the process returns to step 515. When all of the stages have been completed, process 500 performs (525) a graph gather to aggregate the information after the graph con volution stages. In a number of embodiments, the graph gather is only performed on the atoms of the ligand molecule. In a variety of embodiments, the graph gather step is performed using a fully connected neural network that is trained to predict a classification of the ligand molecule. Classifications can include (but are not limited to) a predicted ability to bind with the target molecule.”
Feinberg carries out the methods using programmed general-purpose computers [0063-0067] which comprise CRM.
As such, Claims 1 and 20 are anticipated.
With respect to claim 2, Feinberg includes adjacency matrices, including atom features, and bond features. [0004, 0020, 0045, 0052, et al.]
“[0004] … One embodiment includes a method for predicting characteristics for molecules, wherein the method includes performing a first set of graph convolutions with a spatial graph representation of a set of molecules, wherein the first set of graph convolutions are based on bonds between the set of molecules, performing a second set of graph convolutions with the spatial graph representation, wherein the second set of graph convolutions are based on at least a distance between each atom and other atoms of the set of molecules,”
“[0045] In several embodiments, a given graph that contains N nodes, f.sub.m features per node, and a single edge type, can be represented as consisting of node features x and symmetric adjacency matrix A, which designates whether a pair of nodes belong to each other’s neighbor sets N. Turning now to the drawings, a visual depiction of a Gated Graph Neural Network (GGNN) is illustrated in Figure 1. GGNN 100 is illustrated with nodes (1 10-120), representing atoms, and edges { 130- 140), representing bonds. In this example, the small molecule propanamide is chosen to illustrate the propagation of information among the different update layers of the network. For the carbonyl carbon (110), the relevant row' of the feature matrix x might be [1,0,0] to represent its element, and the corresponding row of the adjacency matrix A might be [0, 1, 0, 1, l ! to indicate its bonds to three neighbor atoms.”
With respect to claim 3, Diagonal blocks hold atom features and off diagonal sections hold bond/ edge features. [0052-0053]
“[0052] Thus far, the concept of adjacency, as encoded in a symmetric matrix A, has been restricted to chemical bonds. However, adjacency can also encompass a wider range of neighbor types to include non-covalent interactions (e.g., p— p stacking, hydrogen bonds, hydrophobic contact). Adjacency need not require domain expertise. In a variety of embodiments, pairwise distances below a threshold value can also be used. Regardless of particular scheme, the distance matrix R motivates the construction of an expanded version of A. In this framework, A becomes a tensor of shape N x N x N.sub.et, where N.sub.et represents the number of edge types.”
“[0053] If the rows are ordered by the membership of atom,: to either the protein or ligand, both A and R can be viewed as block matrices, where the diagonal blocks are self-edges (i.e., bonds and non-covalent interactions) from one ligand atom to another ligand atom or from one protein atom to another protein atom. Off-diagonal block matrices can encode edges from the protein to the ligand and from ligand to protein.”
With respect to claim 4 and atom features, Feinberg includes atom type, charge and chirality. [0091, 0104, etc.]
“[0091] … The atomic information can be but is not limited to: bond lengths within a molecule, bond strengths within a molecule, bond angles within a molecule, dihedral angles within a molecule, dihedral angles between two and/or more molecules, bond angles between two and/or more molecules , interatomic distances between pairs of atoms of the same or of two and/or more different molecules, interatomic angles between triples of atoms of the same or two and/or more different molecules), bond distances between two and/or more molecules, the charge of a molecule, electronegativity of a molecule, the dipole of a given pair of atoms, the dipole of a molecule , the dipole of a set of one or more molecules and/or forces between two and/or more molecules… The atomic information of an atom or atoms within a molecule can be include but is not limited to: the chemical element, atomic number, number of protons, number of electrons, approximate mass, electric charge, diameter, shape, orbital shape, size, energy levels, valency, magnetic moment, and/or isotope.”
With respect to claim 5-6 and bond features, Feinberg includes bond type, bond length, et al. [0009-0013, 0039, 0045, 0052, 0056, 0070-0072, 0076, 0091-0092, 0094, et al.]
“[0091] … The atomic information can be but is not limited to: bond lengths within a molecule, bond strengths within a molecule, bond angles within a molecule, dihedral angles within a molecule, dihedral angles between two and/or more molecules, bond angles between two and/or more molecules , interatomic distances between pairs of atoms of the same or of two and/or more different molecules, interatomic angles between triples of atoms of the same or two and/or more different molecules), bond distances between two and/or more molecules, the charge of a molecule, electronegativity of a molecule, the dipole of a given pair of atoms, the dipole of a molecule , the dipole of a set of one or more molecules and/or forces between two and/or more molecules. Forces between two and/or more molecules can be but are not limited to: electrostatic, ionic attraction, intermediate dipole-dipole, weak London dispersion, hydrophobic interaction, hydrophilic interaction, van der Waal’s, hydrogen bonding, covalent bonding metallic bonding, magnetic, and/or physical…”
With respect to claim 7 and the use of fully connected tensors, Feinberg uses fully connected neural networks at least at [0062, 0067, 0073, 0077-0078 et al.].
With respect to claim 8 and feature vectors, Feinberg creates bond and atom feature vectors at least at [0044-0047, 0056-0057, 0071, 0077, 0080, et al.]
“[0071] Bond featurizers in accordance with numerous embodiments of the invention can generate feature matrices for each atom of a molecular system based on the bonds of the atom to each of its neighbors. In numerous embodiments, bond featurizers can operate on different types of bonds at different stages of the modeling and prediction process. In a variety of embodiments, bond fea- turizers can determine a first set of feature matrices for each atom based on covalent bonds in a first stage, and can be used to determine a second set of feature matrices for the atoms based on other types of bonds. Spatial featurizers in accordance with some embodiments of the invention can generate feature matrices for each atom of a molecular system based on spatial distances between the different atoms, in addition to or in place of the bond types. Bond featurizers in accordance with various embodiments of the invention can operate in conjunction with spatial featurizers in some stages of the process in order to incorporate both bond and spatial information in the feature matrices for each atom. Bond and spatial featurizers can implement a number of neural networks for each bond type and/or for each stage of a process.”
With respect to claim 9, Feinberg discloses stacking of atom features at least at [0010, 0052, 0085, 0088, 0092 et al.] and one-hot encoding at [0133].
“[0133] Featurizing each conformation of a molecule can entail (a) atom-level features, which could be as simple as one-hot features of each element and its hybridization type, (b) adjacency tensor of bonds, and (c) interatomic distances.”
With respect to claim 11 and coordinate data, and Euclidean distance information- Feinberg discloses these at least at [0053].
With respect to claim 12 and 1D convolution, Feinberg discloses convolution stages at least at [0076-80, 0096-0098]
With respect to claim 18 and VAE, Feinberg discloses VAE at [0120].
With respect to claim 19 and CNN, convolutional neural networks are disclosed throughout.
Claim(s) 1-12, 18-20 is/are rejected under 35 U.S.C. 102a1 as being anticipated by Bucher et al (2022).
Bucher, A. et al. System and method for prediction of protein-ligand bioactivity using point-cloud machine learning. US 11,256,995 B1, published 2/22/2022, with priority to at least 4/22/2021.
Bucher is directed to:
“A system and method that predicts whether a given protein-ligand pair is active or inactive, the ground-truth protein-ligand complex crystalline-structure similarity, and an associated bioactivity value. The system and method further produce 3-D visualizations of previously unknown protein-ligand pairs that show directly the importance assigned to protein-ligand interactions, the positive/negative-ness of the saliencies, and magnitude.” (Abstract)
With respect to Claim 1 and: “A method for generating a three-dimensional (3D) conformation of a molecule, the method comprising: obtaining a two-dimensional (2D) graph of the molecule;”
Bucher obtains molecular graph structures of molecules, at least at column 2, first and second full paragraphs; col 7, sixth full paragraph; the paragraph bridging cols 9-10; col 13 first full paragraph, et al.
““SMILES” as used herein means any version of the “simplified molecular-input line-entry system,” which is form of chemical notation for describing the structure of molecules using short text (e.g., ASCII) strings.” Col 7
“Of particular importance is recognition of standardized biochemistry naming conventions including, but not limited to, stock nomenclature, International Union of Pure and Applied Chemistry (IUPAC) conventions, and simplified molecular-input line-entry system (SMILES) and FASTA text-based molecule representations.” Col 10
“FIG. 7 is a diagram illustrating an exemplary graph-based representation of molecules as simple relationships between atoms using a matrix of adjacencies 700, wherein atoms are represented as nodes and bonds between the atoms are represented as edges. Representation of molecules as a graph is useful because it provides a molecular structure which can be processed by graph-based machine learning algorithms like GNNs. Further, the graph-based representation of a molecule can be stated in terms of two matrices, one for the node features (e.g., type of atom and its available bonds) and one for the edges (i.e., the bonds between the atoms). The combination of the nodes (atoms) and edges (bonds) represents the molecule. Each molecule represented in the matrix comprises a dimensionality and features that describe the type of bond between the atoms. According to one embodiment, all bonds within the graph hold the same value, e.g., 1. However, in other embodiments, bonds may be differentiated such as hydrogen bonds having a value of 3, or by having the bond feature dimension exist in each cell.” Col 13
With respect to claim 1 and “generating a first tensor comprising molecular graph information and a second tensor comprising graph, coordinate, and distance information of atoms of the molecule;”
Bucher creates tensors as set forth at column 17,
“Latent samples are passed through a sequence of dense layers, after which the two different matrices (node feature matrix, N and edge feature tensor) are used to reconstruct the node feature and edge feature matrices.”
With respect to claim 1 and “generating a first set of feature vectors corresponding to the first tensor and a second set of feature vectors corresponding to the second tensor;
Bucher creates feature vectors corresponding to the molecular graph information, and coordinate/distance information at least at [col 2, first full paragraph; col 3, description of Fig 9; col 10; col 15 first full paragraph; col 17, second full paragraph; col 22 description of Fig 25, et al.]
“Biochemical databases 214 or similar sources of information may be used to supplement the graph with known properties of proteins, molecules, physiological effects, etc. Separately from the knowledge graph 215, vector representations of proteins, molecules, interactions, and other information may be represented as vectors 216, which may either be extracted from the knowledge graph 215 or may be created directly from data received from the data extraction engine 212. The link between the knowledge graph 215 and the data analysis engine 220 is merely an exemplary abstraction. The knowledge graph 215 does not feed into the models directly but rather the data contained in a knowledge graph structured database is used to train the models. The same exemplary abstraction applies between the vector extraction and embedding 216 and the data analysis engine 220.” Col 10.
With respect to claim 1 and “feeding the first set of feature vectors to a first encoder of a generative model to generate a first output, and feeding the second set of feature vectors to a second encoder of the generative model to generate a second output; and”
Bucher applies the feature vectors to encoders to generate outputs [Col 4, descriptions of Fig 17 and 21; the paragraph bridging cols 15-16; col 17, col 21 et al.]. The Encoders can be VAE as set forth at least at [col 16, last paragraph; Col 17, second and third full paragraphs; col 19, third full paragraph; et al.]
With respect to claim 1 and “combining the first output and the second output in a decoder of the generative model to generate the 3D conformation of the molecule.”
Bucher uses the outputs of the decoder in a generative model to generate 3D conformations, at least at: [col 15, last paragraph; col 17, second full paragraph; col 20, second full paragraph; the paragraph bridging col 20-21, col 21 first full paragraph; the paragraph bridging col 21-22, et al.]
“The neural networks build a model from the training data. In the case of using an autoencoder (or a variational autoencoder), the encoder portion of the neural network reduces the dimensionality of the input molecules, learning a model from which the decoder portion recreates the input molecule. The significance of outputting the same molecule as the input is that the decoder may then be used as a generative function for new molecules. One aspect of a generative decoder module is that the learned model (i.e., protein-ligand atom-features according to one embodiment) lies in a latent space 1404. Sampled areas of the latent space are then interpolated and perturbed 1405 to alter the model such that new and unique latent examples 1406 may be discovered. Other ways to navigate the latent space exist, Gaussian randomization as one example, that may be used in other embodiments of the invention. Furthermore, libraries, other trained models, and processes exist that may assist in the validation of chemically viable latent examples within the whole of the latent space; processing the candidate set of latent examples through a bioactivity model, as one example 1407.” Col 15
“Following this optimization, the obtained minima can be decoded back into a real molecule by taking the decoder output and transforming the atom-densities into the best-matching molecular structure. During optimization of the latent space, it is likely that some constraints must be applied to the latent space to avoid ending up in areas that decode to nonsensical atom densities.” Col 21
All of Bucher’s methods are carried out using programmed computer systems comprising computer readable media [col 34-35]. As such both claims 1 and 20 are anticipated.
With respect to claim 2, Bucher includes adjacency matrices, including atom features, and bond features. [col 12,
“nodes are assigned to each entity identified in each category and attributes of the entity are assigned to the node 601a-f. Attributes of the nodes/entity are information describing the characteristics of the nodes/entity. For example, in some embodiments, attributes of nodes related to molecules are in the form of an adjacency matrix which represents the molecule as relationships between the atoms of the molecule. After nodes have been assigned to all identified entities 601a-f, the relationships between entities are assigned, both within the category of knowledge and between all other categories of knowledge 602a-f. As a simple example of the process, assume that a certain molecule 306 is identified during data extraction. A node is created for the molecule and attributes are assigned to the molecule/node in the form of an adjacency matrix representing the molecule as a series of relationships between the atoms of the molecule.” Col 12
“The relationships between the atoms in the molecule are defined by the adjacency matrix 730. The top row of the adjacency matrix 731 shows all of the atoms in the molecule, and the left column of the matrix 732 shows a list of all possible atoms that can be represented by the matrix for a given set of molecules. In this example, the top row 731 and left column 732 contain the same list of atoms, but in cases where multiple molecules are being represented in the system, the left column may contain other atoms not contained in the particular molecule being represented. The matrix shows, for example, that the hydrogen atom 711 is connected to the carbon atom 712 (a “1” at the intersection of the rows and columns for H and C) and that the carbon atom 712 is connected to the nitrogen atom 713 (a “1” at the intersection of the rows and columns for C and N). In this example, each atom is also self-referenced (a “1” at the intersection of the rows and columns for H and H, C and C, and N and N), but in some embodiments, the self-referencing may be eliminated. In some embodiments, the rows and columns may be transposed (not relevant where the matrix is symmetrical, but relevant where it is not).
FIG. 8 is a diagram illustrating an exemplary graph-based representation of molecules as relationships between atoms using a matrix of adjacencies 800, wherein atoms are represented as nodes and bonds between the atoms are represented as edges, and wherein the type and number of bonds are distinguished. Representation of molecules as a graph is useful because it provides a molecular structure which can be processed by graph-based machine learning algorithms like GNNs. Further, the graph-based representation of a molecule can be stated in terms of two matrices, one for the node features (e.g., type of atom and its available bonds) and one for the edges (i.e., the bonds between the atoms). The combination of the nodes (atoms) and edges (bonds) represents the molecule.” Col 13-14
With respect to claim 3, Diagonal blocks hold atom features and off diagonal sections hold bond/ edge features. [col 22, second full paragraph]
With respect to claim 4-6 and atom / bond features, Bucher includes atom type, charge and chirality. [the paragraph bridging col 6-7; col 13 et al.], bond type, bond length and stereochemistry are provided.
“FIG. 7 is a diagram illustrating an exemplary graph-based representation of molecules as simple relationships between atoms using a matrix of adjacencies 700, wherein atoms are represented as nodes and bonds between the atoms are represented as edges. Representation of molecules as a graph is useful because it provides a molecular structure which can be processed by graph-based machine learning algorithms like GNNs. Further, the graph-based representation of a molecule can be stated in terms of two matrices, one for the node features (e.g., type of atom and its available bonds) and one for the edges (i.e., the bonds between the atoms). The combination of the nodes (atoms) and edges (bonds) represents the molecule. Each molecule represented in the matrix comprises a dimensionality and features that describe the type of bond between the atoms. According to one embodiment, all bonds within the graph hold the same value, e.g., 1. However, in other embodiments, bonds may be differentiated such as hydrogen bonds having a value of 3, or by having the bond feature dimension exist in each cell.” Col 13
With respect to claim 7 and the use of fully connected tensors, Bucher uses fully connected neural networks at least at [col 18].
With respect to claim 8 and feature vectors, Bucher creates bond and atom feature vectors at least at [col 13, col 17]
“The encoder of this embodiment consists of a message passing neural network, which given node and edge features is designed to learn a hidden representation of a molecule (i.e., a readout vector). This is done by continuously aggregating neighboring node and edge information through a process called message passing. The readout vector is subsequently split into the mean and variance vectors which serve and as the parameters of the posterior distribution from the sampling. The model may learn a latent distribution that governs molecular properties and provide a decoder which can construct chemically valid molecules from samples of the prior 1505. Latent samples are passed through a sequence of dense layers, after which the two different matrices (node feature matrix, N and edge feature tensor) are used to reconstruct the node feature and edge feature matrices.” Col 17
With respect to claim 9, Bucher discloses stacking of atom features at least at [col 22 description of fig 25] and one-hot encoding at [col 3, description of Fig 9; col 14, last full paragraph to col 15; col 17].
With respect to claim 10, summing vectors [col 21, desc of fig 21; col 27-28;] and stacking [col 22 description of fig 25] are disclosed.
With respect to claim 11 and coordinate data, and Euclidean distance information- Bucher discloses these at least at [col 20, last full paragraph; description of fig 30].
With respect to claim 12 and 1D convolution, Bucher discloses convolution stages at least at [col 8, last paragraph; col 18, desc of fig 16; et al.]
With respect to claim 18 and VAE, Bucher discloses VAE as set forth above.
With respect to claim 19 and CNN, convolutional neural networks are disclosed throughout.
Claim(s) 1-8, 20 is/are rejected under 35 U.S.C. 102a1 as being anticipated by Xu, M. (2021).
Xu, M. et al. (March 31, 2021) Learning neural generative dynamics for molecular conformation generation. arXiv:2102.10240v3, 17 pages.
Xu is directed to generating 3D structures from a molecular graph.
“Inspired by the recent progress in deep generative models, in this paper, we propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph. We propose a method combining the advantages of both flow-based and energy-based models, enjoying:
(1) a high model capacity to estimate the multimodal conformation distribution; (2) explicitly capturing the complex long-range dependencies between atoms in the observation space.” (abstract)
With respect to Claim 1 and: “A method for generating a three-dimensional (3D) conformation of a molecule, the method comprising: obtaining a two-dimensional (2D) graph of the molecule;”
Xu obtains 2D graphs of a variety of molecules, as set forth at least at: Section 4.1 Experiment setup.
“Benchmarks: We use the recent proposed GEOM-QM9 and GEOM-Drugs (Axelrod & Gomez Bombarelli, 2020) datasets for conformation generation task and ISO17 dataset (Simm & Hernandez-Lobato, 2020) for distances modeling task. The choice of different datasets is because of their distinct properties. Specifically, GEOM datasets consist of stable conformations, which is suitable to evaluate the conformation generation task. By contrast, ISO17 contains snapshots of molecular dynamics simulations, where the structures are not equilibrium conformations but can reflect the density around the equilibrium state. Therefore, it is more suitable for the assessment of similarity between the model distribution and the data distribution around equilibrium states. More specifically, GEOM-QM9 is an extension to the QM9 (Ramakrishnan et al., 2014) dataset: it contains multiple conformations for most molecules while the original QM9 only contains one. This dataset is limited to 9 heavy atoms (29 total atoms), with small molecular mass and few rotatable bonds. We randomly draw 50000 conformation-molecule pairs from GEOM-QM9 to be the training set, and take another 17813 conformations covering 150 molecular graphs as the test set. GEOM-Drugs dataset consists of much larger drug molecules, up to a maximum of 181 atoms (91 heavy atoms). It also contains multiple conformations for each molecule, with a larger variance in structures, e.g., there are the 6.5 rotatable bonds in average. We randomly take 50000 conformation molecule pairs from GEOM-Drugs as the training set, and another 9161 conformations (covering 100 molecular graphs) as the test split. ISO17 dataset is also built upon QM9 datasets, which consists of 197 molecules, each with 5000 conformations. Following Simm & Hernandez-Lobato (2020), we also split ISO17 into the training set with 167 molecules and the test set with another 30 molecules.”
With respect to claim 1 and “generating a first tensor comprising molecular graph information and a second tensor comprising graph, coordinate, and distance information of atoms of the molecule;”
Xu generates tensors which describe the molecular graph information, as well coordinate and distance data, as shown in fig 1. Tensors are algebraic objects which describe a multilinear relationship between sets of algebraic objects associated with a vector space. Section 3.1-3.2 describes taking coordinates and distances and taking molecular graph information and creating tensors which map the data into a vector space.
With respect to claim 1 and “generating a first set of feature vectors corresponding to the first tensor and a second set of feature vectors corresponding to the second tensor;
Xu creates feature vectors corresponding to the molecular graph information, and coordinate/distance information at least at Section 3.2-3.3.
“MPNN takes node attributes, edge attributes and the bonds lengths d(t) as input to compute the node and edge embeddings.” P5
With respect to claim 1 and “feeding the first set of feature vectors to a first encoder of a generative model to generate a first output, and feeding the second set of feature vectors to a second encoder of the generative model to generate a second output; and”
Xu feeds the feature vectors to the relevant encoder to generate outputs. Fig 1, Section 3.2 et al.
“Each message passing layer updates the node embeddings by aggregating the information from neighboring nodes according to its hidden vectors of respective nodes and edges. Final features are fed into a neural network to compute the value of the dynamic fθ for all distances independently.” P5.
With respect to claim 1 and “combining the first output and the second output in a decoder of the generative model to generate the 3D conformation of the molecule.”
Xu generates three-dimensional conformations of the molecules as set forth in section 3.2-3.4.
“Closed-form p(R|d,G). The generated pair-wise distances can be converted into 3D structures through postprocessing methods such as the classic Euclidean Distance Geometry (EDG) algorithm. In this paper, we adopt an alternative way by defining the conformations as a conditional distribution: [EQ 8] where Z is the partition function to normalize the probability and {αuv} are parameters that control the variance of desired Cartesian coordinates, which can be either learned or manually designed according to the graph structure G. With the probabilistic formulation, we can conduct either sampling via MCMC or searching the local optimum with optimization methods. This simple function is fast to calculate, making the generation procedure very efficient with a negligible computational cost.” P5
“The continuous-filter convolutional layers in SchNet allow each atom to aggregate the representations of all single, pairwise, and higher order interactions between the atoms through non-linear functions. The final atomic representations are pooled to a single vector and then passed into a network to produce the scalar output.” P5
“We employ a two-stage dynamic system to synthesize a possible conformation given the molecular graph representation G. In the first stage, we first draw a latent variable ˆz from the Gaussian prior N(0,I), and then pass it through the continuous deterministic dynamics model Fθ defined in Eq. 6 to get ˆd0 = Fθ(ˆz0,G). Then an optimization procedure such as stochastic gradient descent is employed to search the realistic conformations R with local maximum probability of p(R|d,G) (defined in Eq. 8). By doing this, an initial conformation R(0) can be generated. In the second stage, we further refine the initial conformation R(0) with the energy-based model defined in Eq. 5 with K steps of Langevin dynamics: [EQ 10] where denotes the step size. The second integration term in Eθ,φ can be estimated through approximate methods. In practice, we use Monte Carlo Integration to conduct the approximation, which is simple yet effective with just a few distance samples from the CGCF model pθ(d|G).” p5-6
See also Fig 2 and its description in the text, and Appendices A-K.
All of Xu’s methods are carried out using programmed computer systems comprising computer readable media. As such both claims 1 and 20 are anticipated.
With respect to claim 2-7, the tensors add adjacency matrices comprising atom features and bond features. Xu provides these matrices and features as set forth at Appendix B, Appendix I, Appendix K.
“we also generate the confirmations by firstly predicting all the pairwise distances, which enjoys the invariant property to rotation and translation. Since the bonds existing in the molecular graph are not sufficient to characterize a conformation, we pre-process the graphs by extending auxiliary edges. Specifically, the atoms that are 2 or 3 hops away are connected with virtual bonds, labeled differently from the real bonds of the original graph. These extra edges contribute to reducing the degrees of freedom in the 3D coordinates, with the edges between second neighbors helping to fix the angles between atoms, and those between third neighbors fixing dihedral angles.” Appendix B,.
Bond features include: “the energy expression is constituted by seven terms: bond stretching, angle bending, stretch-bend, out-of-plane bending, torsional, van der Waals and electrostatic.” Atom features include: “the first step is to assign correct types to each atom. At the second step, atom-centered partial charges are computed according to the MMFF charge model (Halgren, 1996b). Then, all bonded and non-bonded interactions in the molecular system under study, depending on its structure and connectivity, are loaded into the energy expression.” (Appendix I)
With respect to claim 8, vectors are created for atom and bond features as set forth above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 13-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Feinberg as applied to claims 1-12, 18-20 above, in view of Han (US 2021/0406646 A1).
Feinberg, E. N. et al. Systems and methods for spatial graph convolutions with applications to drug discovery and molecular simulation. (WO 2019/173401 A1) published 9/12/2019.
Han, S. et al. Method, accelerator and electronic device with tensor processing. (US 2021/0406646 A1) published 12/30/2021, having priority to at least 11/2020.
Feinberg teaches the limitations of the independent claims, as set forth above:
With respect to claim 1 and: “Claim 1. A method for generating a three-dimensional (3D) conformation of a molecule, the method comprising: obtaining a two-dimensional (2D) graph of the molecule;”
Feinberg obtains 2D graphs of a variety of molecules, as set forth at least at: [0070, 0071, 0075, et al.].
With respect to claim 1 and “generating a first tensor comprising molecular graph information and a second tensor comprising graph, coordinate, and distance information of atoms of the molecule;”
Feinberg generates tensors, as set forth at least at [0007, 0052, 0056-57, 0081, 0092, 0125 et al.]
With respect to claim 1 and “generating a first set of feature vectors corresponding to the first tensor and a second set of feature vectors corresponding to the second tensor;”
Feinberg generates feature vectors corresponding to the tensors, at least at [0004, 0056-0057, 0060, 0077, 0125 et al.]
With respect to claim 1 and “feeding the first set of feature vectors to a first encoder of a generative model to generate a first output, and feeding the second set of feature vectors to a second encoder of the generative model to generate a second output; and”
Feinberg uses feature vectors as input to a generative model, which can be a variational autoencoder, to generate outputs, at least at [0052-0053, 0081, 0120, 0136]
With respect to claim 1 and “combining the first output and the second output in a decoder of the generative model to generate the 3D conformation of the molecule.”
Feinberg gathers and combines the various outputs in graph gather step and generates a 3D conformation. [0057, 0061, 0078, 0090 et al.]
Feinberg carries out the methods using programmed general-purpose computers [0063-0067] which comprise CRM.
Claims 13-15 recite elements regarding adjusting kernel height or width of one or both tensors.
Feinberg does not address kernel height or width of the tensors, in generating the feature vectors from tensors.
In the field of improving tensor processing, Han is directed to the following:
“… receiving a request to process a neural network including a normalization layer by an accelerator; and generating an instruction executable by the accelerator in response to the request, wherein, by executing the instruction, the accelerator is configured to determine an intermediate tensor corresponding to a result of a portion of operations of the normalization layer, by performing, in a channel axis direction, a convolution based on an input tensor and a kernel, wherein the input tensor is of the normalization layer and includes a plurality of channels, a number of input channels of the kernel is determined based on the input tensor, and scaling values of elements of the kernel are determined based on the number of input channels.” (abstract)
Han describes computer systems comprising processors, memory and CRM that accelerate neural network / tensor processing and response to requests to process information [0054]. Neural processing units (NPU) and tensor processing units [TPU] are described at length, beginning at [0057]. As illustrated by Fig 6, and detailed in Figs 1-5, the kernel generator accepts information from the channels to be convoluted, which in Feinberg would be a coordinate channel and a Euclidean distance channel from the information provided by the 2D molecular graph.
“[0063] To implement the average operation and/or the average subtraction operation. which may not be directly supported by the accelerator 140, the average operation and/or the average subtraction operation may be replaced with a convolution operation performed by the accelerator 140. A kernel used for the convolution operation may be generated based on the data to be normalized. Through this, the accelerator 140 may obtain output data identical to outputs that would be generated by directly performing operations specified by a normalization layer based on the convolution operation, even when the convolution operation does not identically match the operations specified by the normalization layer…”
“The first normalization layer may be based on layer normalization, wherein convolution may be performed to decrease a size of a tensor in a channel axis direction.” [0066]
“[0067] An input tensor of FIG. 2 may be data input into a first normalization layer. In the example of FIG. 2, the input tensor has a (1, 3, 1, 4) shape, and the value of each element may be represented by any one of a to l. In the example of FIG. 2, the tensor structure of FIG. 2 is expressed as (batch, channel, height, width). In an example, the input tensor may correspond to output data of one layer (e.g., a hidden layer) of a neural network.”
“[0068] A kernel shown in FIG. 2 may have a number (e.g., a total number) of input channels determined based on the input tensor. For example, the number of input channels of the kernel may be equal to a number of channels of the input tensor. A number of output channels of the kernel that is applied to a convolution operation for obtaining an average operation result may be “1”. In the example of FIG. 2, the kernel has a (1, 1, 3, 1) shape, where the kernel shape is expressed as (height, width, input channel, output channel). Further, scaling values of the elements included in the kernel may correspond to an inverse of the number of channels of the input tensor. In the example of FIG. 2, the scaling values are ⅓. In the present disclosure, a scaling value may also be referred to as a weight value for ease of description…
[0069] … In other words, a number of channels may be reduced through the convolution operation without reducing a number of batches, a height, and a width. By determining the average tensor as described above with reference to FIG. 2, the average value of the input tensor may be determined in the accelerator 140 through a single convolution operation, …”
“[0072] … In the example of FIG. 3, the kernel may have a (1, 1, 3, 3) shape, where the kernel shape is expressed as (height, width, input channel, output channel). Further, the diagonal elements and the remaining elements in the kernel may have different scaling values. For example, the diagonal elements may have scaling values of 1−1/n (here, n denotes the number of channels of the input tensor), and the remaining elements may have scaling values of −1/n. When the number of channels n of the input tensor is “3” in the example of FIG. 3, the scaling values of the diagonal elements may be ⅔, and the scaling values of the remaining elements may be −⅓.”
Han details methods of increasing or decreasing tensor size in a channel axis direction, utilizing changes in tensor shape, kernel shape and parameter values in the descriptions of Figs 2-5 and 6.
“[0092] The kernel generator 620 may be a hardware device that generates a kernel (e.g., any one or more of the kernels described above with reference FIGS. 1-5) for convolution (e.g., any one or more of the convolutions described above with reference to FIGS. 1-5). The kernel generator 620 may receive a number of channels to be convoluted n, and a flag. The number of channels n may be determined by an input tensor (e.g., any one or more of the input tensors described above with reference to FIGS. 1-5), and may correspond to, for example, a number of channels of the input tensor in the case of layer normalization, and may correspond to, for example, a number of channels of a transposed input tensor (e.g., any one or more of the transposed input tensors described above with reference to FIGS. 1-5) in the case of instance normalization. The flag may be an indicator indicating whether a normalization operation to be replaced with convolution is an average operation or an average subtraction operation. For example, the flag having a value of “0” may indicate that an average operation is to be implemented as convolution, and the flag having a value of “1” may indicate that an average subtraction operation is to be implemented as convolution…”
Han discloses at length, the use of the tensor processing in accelerating neural network performance, including transformer networks beginning at [0098].
In KSR Int 'l v. Teleflex, the Supreme Court, in rejecting the rigid application of the teaching, suggestion, and motivation test by the Federal Circuit, indicated that “The principles underlying [earlier] cases are instructive when the question is whether a patent claiming the combination of elements of prior art is obvious. When a work is available in one field of endeavor, design incentives and other market forces can prompt variations of it, either in the same field or a different one. If a person of ordinary skill can implement a predictable variation, § 103 likely bars its patentability.” KSR Int'l v. Teleflex lnc., 127 S. Ct. 1727, 1740 (2007).
Applying the KSR standard of obviousness to Feinberg and Hahn, the Examiner concluded that adding the input channel and tensor processing as disclosed by Han to the 3D conformation generation model and system disclosed by Feinberg represents the use of a known technique to improve similar methods. The nature of the problem to be solved may lead inventors to look at references relating to possible solutions to that problem. Generation of valid 3D models from 2D molecular graphs requires a significant amount of data processing in the generation of the tensors, and time in the use of multiple encoders and decoders to provide the final 3D conformation. Han provides a way to speed up tensor generation and channel processing, by use of their tensor processing operations. Han’s method also has the improvement of ensuring associated channel information is analyzed and not lost, improving the final outputs. One of skill in the art of bioinformatics would have had a reasonable expectation of success in making the combination and the improvement, as the tensor processing of Han is compatible with the tensor generation of Feinberg. Therefore, it would have been prima facie obvious to one of ordinary skill in the art at the time of filing to have used the tensor processing of Han in the 3D conformation generation methods of Feinberg, absent evidence to the contrary. Such a combination is merely a "predictable use of prior art elements according to their established functions." KSR Int’l 7, 127 S. Ct. at 1740.
Claim(s) 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Feinberg as applied to claims 1-12, 18-20 above, in view of Prat (US 2022/0198286 A1).
Feinberg, E. N. et al. Systems and methods for spatial graph convolutions with applications to drug discovery and molecular simulation. (WO 2019/173401 A1) published 9/12/2019.
Prat, A. et al. System and method for molecular reconstruction from molecular probability distributions. US 2022/0128286 A1, published 6/23/2022, filed 12/1/2021.
Feinberg teaches the limitations of the independent claims, as set forth above:
With respect to claim 1 and: “Claim 1. A method for generating a three-dimensional (3D) conformation of a molecule, the method comprising: obtaining a two-dimensional (2D) graph of the molecule;”
Feinberg obtains 2D graphs of a variety of molecules, as set forth at least at: [0070, 0071, 0075, et al.].
With respect to claim 1 and “generating a first tensor comprising molecular graph information and a second tensor comprising graph, coordinate, and distance information of atoms of the molecule;”
Feinberg generates tensors, as set forth at least at [0007, 0052, 0056-57, 0081, 0092, 0125 et al.]
With respect to claim 1 and “generating a first set of feature vectors corresponding to the first tensor and a second set of feature vectors corresponding to the second tensor;”
Feinberg generates feature vectors corresponding to the tensors, at least at [0004, 0056-0057, 0060, 0077, 0125 et al.]
With respect to claim 1 and “feeding the first set of feature vectors to a first encoder of a generative model to generate a first output, and feeding the second set of feature vectors to a second encoder of the generative model to generate a second output; and”
Feinberg uses feature vectors as input to a generative model, which can be a variational autoencoder, to generate outputs, at least at [0052-0053, 0081, 0120, 0136]
With respect to claim 1 and “combining the first output and the second output in a decoder of the generative model to generate the 3D conformation of the molecule.”
Feinberg gathers and combines the various outputs in graph gather step and generates a 3D conformation. [0057, 0061, 0078, 0090 et al.]
Feinberg carries out the methods using programmed general-purpose computers [0063-0067] which comprise CRM.
Claims 13-15 recite elements regarding adjusting kernel height or width of one or both tensors.
Feinberg does not address using multi-head attention in a generative model to generate 3D conformations.
In the field of identifying and generating 3D conformations of a molecule from 2D molecular graphs, Prat addresses the use of multi-head attention layers [0189], which act on Query and Key matrices.
“A system and method comprising a transmoler that identifies common substructures of a given 3D conformer and predicts its structural information.” (abstract)
Figure 13 of Prat illustrates the workflow, beginning with SMILES and 2D molecular graph construction, which includes adjacency matrices, and node features, followed by application to a transformer based neural network which contains multi-head attention layers. Fig 12 and its description provide exemplary molecular attention assignment steps to analyze molecular structure.
“[0117] … A first approach begins with preprocessing all the SMILES representations for reactants and products to convert to canonical form (SMILES to Mol & Mol to SMILES through a cheminformatics toolkit), remove duplicates & clean the data, augmenting SMILE equivalents via enumeration. Then, transformer models are used with multiple attention heads and a k-beam search is set up.”
“[0180] Once the molecule graph construction 1013 is completed, the node features matrices 1012 and adjacency matrices 1011 are passed to a message passing neural network (MPNN) 1020, wherein the processing is parallelized by distributing groups 1021 nodes of the graph amongst a plurality of processors (or threads) for processing. Each processor (or thread) performs attention assignment 1022 on each node, increasing or decreasing the strength of its relationships with other nodes, and outputs of the node and signals to other neighboring nodes 1023 (i.e., nodes connected by edges) based on those attention assignments are determined. Messages are passed 1024 between neighboring nodes based on the outputs and signals, and each node is updated with the information passed to it. Messages can be passed between processors and/or threads as necessary to update all nodes. In some embodiments, this message passing (also called aggregation) process is accomplished by performing matrix multiplication of the array of node states by the adjacency matrix to sum the value of all neighbors or divide each column in the matrix by the sum of that column to get the mean of neighboring node states. This process may be repeated an arbitrary number of times.”
To generate 3D conformations, the multi-head attention results can be applied to a variational autoencoder. The autoencoder comprises both encoder and decoder elements.
“[0176] … A node level mechanism may then perform attention on these to predict the molecule DeepSMILES one-hot encoded representations 4304. Beam search and re-conformerization can also be performed to improve the models performance. In particular, re-conformarization involves reconstructing a molecule using our generative 3D restricted variational autoencoder and saving the gaussian representation (Xr′). Using Xr′ to predict the set of box attributes (i.e., prediction boxes (Pb) attributes) using the substructure processing module 4004. Using Pb to generate various DeepSMILES using beam search (standard deep learning sequence prediction procedure). Then use re-conformerization of the predicted molecules to determine which of these better match Xr′ via a registration. Since set sequence models suffer from over generation, this process can be very useful, ensuring or motivating the decoded molecule to actually represent its gaussian representation Xr′. Irrespective of the various approaches previously laid forth for molecular reconstruction, the transmoler 3900 output may be a SMILES representation 4304 given an input of a gaussian distribution representative 3014 of a molecule.”
“[0192] The outputs of the multi-head attention mechanism 1207 are then concatenated 1214, and optionally sent to a drawing program for display of the outputs in graphical form 1213.”
In KSR Int 'l v. Teleflex, the Supreme Court, in rejecting the rigid application of the teaching, suggestion, and motivation test by the Federal Circuit, indicated that “The principles underlying [earlier] cases are instructive when the question is whether a patent claiming the combination of elements of prior art is obvious. When a work is available in one field of endeavor, design incentives and other market forces can prompt variations of it, either in the same field or a different one. If a person of ordinary skill can implement a predictable variation, § 103 likely bars its patentability.” KSR Int'l v. Teleflex lnc., 127 S. Ct. 1727, 1740 (2007).
Applying the KSR standard of obviousness to Feinberg and Prat, the Examiner concluded that processing the output in a decoder using multi-head attention to then generate a 3D conformation as disclosed by Prat to the 3D conformation generation model and system disclosed by Feinberg represents the use of a known technique to improve similar methods. The nature of the problem to be solved may lead inventors to look at references relating to possible solutions to that problem. Use of multi-head attention and combining outputs in the decoder as disclosed by Prat allows for the integration of multiple sources of information, to identify structural interactions between differing parts of the molecular graph, locally and globally. The use of multi-head attention can improve interpretability of the results of the process. One of skill in the art of bioinformatics would have had a reasonable expectation of success in making the combination and the improvement, as the use of multi-head attention in the analysis of drugs, compounds, or molecules was well known at the time of filing and widely applied to the same neural networks of Feinberg. Therefore, it would have been prima facie obvious to one of ordinary skill in the art at the time of filing to have used the multi-head attention of Prat in the 3D conformation generation methods of Feinberg, absent evidence to the contrary. Such a combination is merely a "predictable use of prior art elements according to their established functions." KSR Int’l 7, 127 S. Ct. at 1740.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARY K ZEMAN whose telephone number is 5712720723. The examiner can normally be reached on 8am-2pm M-F. Email may be sent to mary.zeman@uspto.gov if the appropriate permissions have been filed.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Larry Riggs can be reached on 571 270-3062. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARY K ZEMAN/ Primary Examiner, Art Unit 1686
Prosecution Insights