DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-9 are pending and under examination.
This application is a National Stage application of PCT KR2021/017765, filed 11/29/2021, which claims priority to a KR priority document filed 2/1/2021. The copy of the certified priority document has been provided by the IB. However no certified translation of this priority document is of record, thus the claims are afforded the effective filing date of 11/29/2021.
The examiner has reviewed all PCT documentation.
This application has published as US 2024-0079098 A1.
The IDS filed 7/26/2023 has been entered and considered.
The drawings are objected to because:
In Figure 1, the topmost part appears upside down with respect to the middle part of the figure, and the bottom part of the figure is sideways with respect to the middle part of the figure. It is entirely unclear how to connect these three portions as drawn. Figure 1 appears to describe a polypeptide sequence which should be annotated with a SEQ ID NO in the Brief Description of the Drawings, and a corresponding sequence in a sequence listing is required.
Figure 2 appears to describe a polypeptide sequence which should be annotated with a SEQ ID NO in the Brief Description of the Drawings, and a corresponding sequence in the sequence listing is required.
Figure 3 appears to describe two polypeptide sequences at element (11) which should each be annotated with a SEQ ID NO in the Brief Description of the Drawings, and corresponding sequences in the sequence listing are required.
Figure 5 appears to describe two polypeptide sequences which should each be annotated with a SEQ ID NO in the Brief Description of the Drawings, and corresponding sequences in the sequence listing are required.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Summary of Requirements for Patent Applications Filed On Or After July 1, 2022, That Have Sequence Disclosures
37 CFR 1.831(a) requires that patent applications which contain disclosures of nucleotide and/or amino acid sequences that fall within the definitions of 37 CFR 1.831(b) must contain a “Sequence Listing XML”, as a separate part of the disclosure, which presents the nucleotide and/or amino acid sequences and associated information using the symbols and format in accordance with the requirements of 37 CFR 1.831-1.835. This “Sequence Listing XML” part of the disclosure may be submitted:
1. In accordance with 37 CFR 1.831(a) using the symbols and format requirements of 37 CFR 1.832 through 1.834 via the USPTO patent electronic filing system (see Section I.1 of the Legal Framework for Patent Electronic System (https://www.uspto.gov/PatentLegalFramework), hereinafter “Legal Framework”) in XML format, together with an incorporation by reference statement of the material in the XML file in a separate paragraph of the specification (an incorporation by reference paragraph) as required by 37 CFR 1.835(a)(2) or 1.835(b)(2) identifying:
a. the name of the XML file
b. the date of creation; and
c. the size of the XML file in bytes; or
2. In accordance with 37 CFR 1.831(a) using the symbols and format requirements of 37 CFR 1.832 through 1.834 on read-only optical disc(s) as permitted by 37 CFR 1.52(e)(1)(ii), labeled according to 37 CFR 1.52(e)(5), with an incorporation by reference statement of the material in the XML format according to 37 CFR 1.52(e)(8) and 37 CFR 1.835(a)(2) or 1.835(b)(2) in a separate paragraph of the specification identifying:
a. the name of the XML file;
b. the date of creation; and
c. the size of the XML file in bytes.
SPECIFIC DEFICIENCIES AND THE REQUIRED RESPONSE TO THIS NOTICE ARE AS FOLLOWS:
Specific deficiency - This application fails to comply with the requirements of 37 CFR 1.831-1.834 because it does not contain a “Sequence Listing XML” as a separate part of the disclosure. A “Sequence Listing XML” is required because the drawings set forth polypeptide sequences falling within the guidelines for this requirement.
Required response - Applicant must provide:
• A “Sequence Listing XML” part of the disclosure, as described above in item 1. or 2.; together with
o A statement that indicates the basis for the amendment, with specific references to particular parts of the application as originally filed, as required by 37 CFR 1.835(a)(3);
o A statement that the “Sequence Listing XML” includes no new matter as required by 37 CFR 1.835(a)(4)
AND
• A substitute specification in compliance with 37 CFR 1.52, 1.121(b)(3), and 1.125 inserting the required incorporation by reference paragraph as required by 37 CFR 1.835(a)(2), consisting of:
o A copy of the previously-submitted specification, with deletions shown with strikethrough or brackets and insertions shown with underlining (marked-up version);
o A copy of the amended specification without markings (clean version); and
o A statement that the substitute specification contains no new matter.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation (BRI) using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of mental steps, mathematic concepts, organizing human activity, or a natural law without significantly more.
Applicant is directed to MPEP 2106 for the most current and complete guidelines in the analysis of patent- eligible subject matter. The current MPEP is the primary source for the USPTO’s patent eligibility guidance.
With respect to step (1): YES, the claims are drawn to statutory categories: computer-implemented processes.
With respect to step (2A) (1): YES, the claims recite an abstract idea, law of nature and/or natural phenomenon. The claims explicitly recite elements that, individually and in combination, constitute one or more judicial exceptions (JE).
Mathematic concepts, Mental Processes or Elements in Addition (EIA) in the claim(s) include:
[Claim 1] A method for predicting a binding region or drug-target interaction by using a self-attention-based deep neural network, the method being performed by a control unit including one or more processors and a memory, the method comprising:
(Preamble, indicating a method, the goal of the method, and the use of a general-purpose computer. The general-purpose computer is an element in addition (EIA) MPEP 2106.05(f))
training a transformer network by a drug fingerprint and a protein sequence database;
(EIA: a generically stated transformer network, a data structure of a neural network that uses attention mechanisms. MPEP 2106.05(f). EIA of a generically stated training database, an element of data gathering. Training a transformer network is the application of training data to a neural network to teach it how to predict missing or upcoming data values and optimization, a mathematic concept MPEP 2106.04(a)(2) section 1.)
transforming the drug fingerprint into a drug token by passing the drug fingerprint through a dense layer;
(Mathematic concept of data transformation or tokenization wherein data is converted into vectors that capture semantic meaning MPEP 2106.04(a)(2) section 1. AND an EIA: A “dense layer” is a part of a neural network where every neuron connects to every neuron in the previous layer. However, the transformer network of (a) does not recite any particular layers, or how they are connected MPEP 2106.05(f).)
transforming a protein sequence into a protein grid encoding by performing a convolution operation on the protein sequence,
(Mathematic concept sliding a filter or a kernel across a protein sequence to extract local patterns, by multiplying internal weights and calculating a numerical feature MPEP 2106.05(a)(2) section 1)
dividing the protein sequence into predetermined unit grids, and
(Mathematic concept of dividing a string of data into predetermined units. MPEP 2106.05(a)(2) section 1)
then performing max pooling thereon;
(Mathematic concept of extracting the highest numerical value representing the strongest biochemical signal or feature from a segment of a protein sequence. MPEP 2106.05(a)(2) section 1)
concatenating the drug token to the protein grid encoding;
(Mathematic concept of combining tokens and encoded data into a single combined input. MPEP 2106.05(a)(2) section 1)
inputting the drug token and the protein grid encoding, which are concatenated to each other, to the transformer network; and
(Mathematic concept of applying the concatenated single combined input to the (trained) transformer network, which transforms the input into numerical vectors. MPEP 2106.05(a)(2) section 1).
predicting an interaction between a drug and a target protein or a binding region where the drug binds to the target protein by an output of the transformer network.
(Mental process of observing the output of the transformer network and making a judgement as to whether that output describes an interaction. MPEP 2106.05(a)(2) section 3).
[Claim 2] The method of claim 1, wherein the drug fingerprint is a Morgan fingerprint hashed by a Morgan algorithm.
(EIA- an element of data gathering or describing data required to be used in the method. MPEP 2106.05(g)).
[Claim 3] The method of claim 1, wherein the drug fingerprint and the protein sequence database in the step (a) comprise a three- dimensional structure and binding information of the drug and the protein.
(EIA- an element of data gathering or describing data required to be used in the method. MPEP 2106.05(g)).
[Claim 4] The method of claim 3, wherein, in the step (a), the transformer network is trained by transforming a binding site of the binding information into a binding region including up to a sequence adjacent to the binding site.
(Mathematic concept of transforming binding site information into a vector describing the binding site. Training a transformer network is the application of training data to a neural network to teach it how to predict missing or upcoming data values and optimization, a mathematic concept MPEP 2106.04(a)(2) section 1.)
[Claim 5] The method of claim 1, wherein the step (c) comprises performing a convolutional operation on the protein sequence by using a Convolution Neural Network (CNN).
(Mathematic concept of performing a convolutional operation on a data string using a generically stated neural network. MPEP 2106.04(a)(2) section 1; sliding a filter or a kernel across a protein sequence to extract local patterns, by multiplying internal weights and calculating a numerical feature MPEP 2106.04(a)(2) section 1).
[Claim 6] The method of claim 1, wherein the drug token and the unit grid have a same length.
(Mathematic concept modification, defining a part of the transformation. MPEP 2106.04(a)(2) section 1)
[Claim 7] The method of claim 1, wherein the step (e) comprises transforming the drug token and the protein grid encoding, which are concatenated to each other, into Q (query), K (key), and V (value) vectors and inputting the Q (query), K (key), and V (value) vectors to the transformer network.
(Mathematic concept of data transformation by transforming the concatenated data into 3 types of vectors. MPEP 2106.04(a)(2) section 1)
[Claim 8] The method of claim 1, wherein the transformer network comprises two or more transformer networks.
(EIA describing an aspect of the structure of the transformer network. MPEP 2106.05(f).)
[Claim 9] The method of claim 1, wherein the step (f) comprises predicting a relationship between the drug and the protein by using an attention score between the drug and the protein grid encoding.
(Mental process modification specifying the comparison of an attention score and making a judgement as to any interaction. MPEP 2106.04(a)(2) section 3).
With respect to step 2A (2): NO, the claims do not integrate the JE into a practical application (MPEP 2106.04(d)):
“Examiners evaluate integration into a practical application by: (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception(s); and (2) evaluating those additional elements individually and in combination to determine whether they integrate the exception into a practical application, using one or more of the considerations introduced in subsection I supra, and discussed in more detail in MPEP §§ 2106.04(d)(1), 2106.04(d)(2), 2106.05(a) through (c) and 2106.05(e) through (h).”
Claim(s) 1-3 recite the additional non-abstract element(s) of data gathering, or a description of the data gathered.
Data gathering steps are not an abstract idea, they are extra-solution activity, as they collect the data necessary to carry out the JE. MPEP 2106.05(g).
The data gathering does not impose any meaningful limitation on the JE, or how the JE is performed. MPEP 2106.05(g).
The data gathering steps constitute a general link to a technological environment: the prediction methods are intended to be applied to drug-target interactions (MPEP 2106.05(h), citing Mayo, Bilski, electric Power Group, Genetic Techs Ltd v Merial LLC.)
The additional limitation (data gathering) must have more than a nominal or insignificant relationship to the identified judicial exception to provide integration into a practical application. (MPEP 2106.05(g) citing Mayo, PerkinElmer, Inc. v. Interna Ltd, Intellectual Ventures LLC v. Erie Indem. Co., Electric Power Group LLC v. Alstom S.A.).
Claim(s) 1 and 8 recite the additional non-abstract element (EIA) of a general-purpose computer system and parts thereof.
The claims do not provide any details of how specific structures of the computer elements are used to implement the JE. MPEP 2106.05(a), contrasting decisions identifying how the computer implements an abstract idea, such as in McRo to decisions which found no specific interaction with the computer, such as in Affinity Labs of Tex v. DirecTV, LLC.
The computer elements of the claims do not provide improvements to the functioning of the computer itself. MPEP 2106.05(a) I, contrasting decisions indicating an improvement to the computer, such as DDR Holdings, LLC v. Hotels.com LP, with decisions that did not identify an improvement to the computer, such as FairWarning IP, LLC v. Iatrix Sys.
The computer elements of the claims do not provide improvements to any other technology or technical field. MPEP 2106.05(a) II: contrasting decisions indicating an improvement to the technology, such as Diamond v. Diehr, Trading Techs. Int’l v. CQG Inc, or Intellectual Ventures I v. Symantec Corp, with decisions that did not identify an improvement to the technology, such as Alice Corp, Versata Dev. Group, Inc. v. SAP AM. Inc, or TLI Communications.
The computer elements of the claims do not utilize a particular machine. MPEP 2106.05(b): contrasting decisions wherein a particular machine was identified, such as MacKay Radio & Tel. Co. v. Radio Corp. of America, Eibel Process Co. v. Minn. & Ont. Paper Co., with decisions where a general-purpose computer does not qualify as a particular machine, such as Ultramercial, Inc. v. Hulu, LLC, TLI communications, or Eon Corp. IP holdings LLC v. AT&T Mobility LLC.
Hence, these are mere instructions to apply the JE using a computer, and therefore the claim does not recite integrate that JE into a practical application.
Dependent claim(s) 4-7, 9 recite(s) an abstract limitation to the JE reciting additional mathematic concepts, or mental processes. Additional abstract limitations cannot provide a practical application of the JE as they are a part of that JE.
In combination, the limitations of data gathering, for the purpose of carrying out the JE, using a general-purpose computer merely provide extra-solution activity, and fail to integrate the JE into a practical application.
With respect to step 2B: NO, the claims do not recite a specific inventive concept. The judicial exception alone cannot provide that inventive concept or practical application (MPEP 2106.05).
“… an "inventive concept" is furnished by an element or combination of elements that is recited in the claim in addition to (beyond) the judicial exception, and is sufficient to ensure that the claim, as a whole, amounts to significantly more than the judicial exception itself. Alice Corp…”
With respect to claim(s) 1-3: The limitation(s) identified above as non-abstract elements (EIA) related to data gathering do not rise to the level of significantly more than the judicial exception.
Lee et al. (2019: pto-1449) discloses databases of protein sequences and drug fingerprints as shown in Fig 1 “training dataset generation” DrugBank, KEGG, IUPHAR, et al.
Ji et al. (2020; PTO-1449) discloses databases of protein sequences and drug fingerprints as set forth at page 2/13, and Fig 1.
Chen et al. (2020; PTO-1449) discloses databases of proteins and drug fingerprints, as set forth at page 4, section 2.2.1 Public Datasets.
Fokoue-Nkoutche et al. (US 2019/0303535 A1) discloses databases of proteins and drug fingerprints as set forth at [0026, 0032-0033, 0036, 0049-0051 et al. biomedical entity pairs, gene sequences, protein sequences, SMILES representations].
Quan et al. (CN110289050 A, 9/27/2019) discloses databases of proteins and drug fingerprints, including MORGAN fingerprints.
These elements meet the BRI of the identified data gathering limitations. As such, the prior art recognizes that this data gathering element is routine, well understood and conventional in the art. MPEP 2106.05(d): “If, however, the additional element (or combination of elements) is no more than well-understood, routine, conventional activities previously known to the industry, which is recited at a high level of generality, then this consideration does not favor eligibility.”
Data gathering steps are not an abstract idea, they are extra-solution activity, as they collect the data necessary to carry out the JE. MPEP 2106.05(g).
The data gathering does not impose any meaningful limitation on the JE, or how the JE is performed. MPEP 2106.05(g).
The additional limitation (data gathering) must have more than a nominal or insignificant relationship to the identified judicial exception to provide an inventive concept. (MPEP 2106.05(g) citing Mayo, PerkinElmer, Inc. v. Interna Ltd, Intellectual Ventures LLC v. Erie Indem. Co., Electric Power Group LLC v. Alstom S.A.)
The data gathering steps constitute a general link to a technological environment: the trait prediction methods are intended to be applied to plant populations. (MPEP 2106.05(h), citing Mayo, Bilski, electric Power Group, Genetic Techs Ltd v Merial LLC.)
Therefore, simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception are insufficient to provide significantly more (as discussed in Alice Corp.,).
With respect to claim(s) 1 and 8: the limitations identified above as non-abstract elements (EIA) related to general-purpose computer systems and parts thereof, do not rise to the level of significantly more than the judicial exception.
With respect to the general-purpose computer that can host a neural network:
Each of Li, Ji, Chen, Fokoue-Nkoutche and Quan cited above provide general purpose computers comprising processors and memory and can host neural networks.
With respect to the “transformer network”:
Feala (WO2020/167667 A1) discloses transformer networks, used to transform biological sequence data and drug fingerprint data. General purpose computers are also disclosed.
Wang et al. (US 2019/0266246 A1) discloses transformer networks, used to transform biological sequence data. General purpose computers are also disclosed.
Wu et al. (CN 110853704 A 2/28/2020) discloses transformer networks, used to transform biological sequence data. General purpose computers are also disclosed.
Brown et al. (WO2020/049293 A1, 12 March 2020) discloses transformer networks for transforming biological sequence data. General purpose computers are also disclosed.
As such, the prior art recognizes that these computing elements are routine, well understood and conventional in the art.
The claims do not provide any details of how specific structures of the computer elements are used to implement the JE. MPEP 2106.05(a), contrasting decisions identifying how the computer implements an abstract idea, such as in McRo to decisions which found no specific interaction with the computer, such as in Affinity Labs of Tex v. DirecTV, LLC.
The computer elements of the claims do not provide improvements to the functioning of the computer itself. MPEP 2106.05(a) I, contrasting decisions indicating an improvement to the computer, such as DDR Holdings, LLC v. Hotels.com LP, with decisions that did not identify an improvement to the computer, such as FairWarning IP, LLC v. Iatrix Sys.
The computer elements of the claims do not provide improvements to any other technology or technical field. MPEP 2106.05(a) II: contrasting decisions indicating an improvement to the technology, such as Diamond v. Diehr, Trading Techs. Int’l v. CQG Inc, or Intellectual Ventures I v. Symantec Corp, with decisions that did not identify an improvement to the technology, such as Alice Corp, Versata Dev. Group, Inc. v. SAP AM. Inc, or TLI Communications.
The computer elements of the claims do not utilize a particular machine. MPEP 2106.05(b): contrasting decisions wherein a particular machine was identified, such as MacKay Radio & Tel. Co. v. Radio Corp. of America, Eibel Process Co. v. Minn. & Ont. Paper Co., with decisions where a general-purpose computer does not qualify as a particular machine, such as Ultramercial, Inc. v. Hulu, LLC, TLI communications, or Eon Corp. IP holdings LLC v. AT&T Mobility LLC.
Hence, these are mere instructions to apply the JE using a computer, and therefore the claim does not provide significantly more.
Dependent claim(s) 4-7, 9 each recite a limitation requiring additional mathematic concepts or mental processes. Additional abstract limitations cannot provide significantly more than the JE as they are a part of that JE (MPEP 2106.05).
In combination, the data gathering steps providing the information required to be acted upon by the JE, performed in a generic computer or generic computing environment fail to rise to the level of significantly more than that JE. The data gathering steps provide the data for the JE, which is carried out by the general-purpose computers. No non-routine step or element has clearly been identified.
The claims have all been examined to identify the presence of one or more judicial exceptions. Each additional limitation in the claims has been addressed, alone and in combination, to determine whether the additional limitations integrate the judicial exception into a practical application. Each additional limitation in the claims has been addressed, alone and in combination, to determine whether those additional limitations provide an inventive concept which provides significantly more than those exceptions. For these reasons, the claims, when the limitations are considered individually and as a whole, are rejected under 35 USC § 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
As set forth above, the claims are afforded the effective filing date of 11/29/2021.
Claim(s) 1-6, and 9 is/are rejected under 35 U.S.C. 102a1 as being anticipated by Truong Jr. (2020).
Truong Jr., T. F. (May 2020) Interpretable deep learning framework for binding affinity prediction. Thesis, Massachusetts Institute of Technology, 76 pages. (Herein: Truong)
Truong provides methods of predicting binding regions or a drug-target interaction (DTI) using attention-based transformer networks. (Abstract)
“The model combines recently developed learned protein sequence embeddings that encode structural information with compound fingerprints using a Transformer architecture.”
“Potential/Transformer model Proteins are represented using previously developed learned
protein embeddings [2] which encode structural information. Briefly, the embeddings were learned by training a deep bidirectional LSTM on two protein-only side tasks – predicting protein structural similarity as defined by the SCOP classification hierarchy, and predicting contact maps derived from the protein's structure. The LSTM takes as input the one hot encoding of the protein sequence, and produces a sequence of 100 dimensional embeddings with the same length as the protein.” P26
“Potential/Transformer model Compounds are represented using ECFP fingerprints [32] with diameter 4 and 8192 bits. The compound encoder network was again the identity function” p28
The transformer network is trained with data from a database comprising protein sequence information and drug fingerprint information. BindingDB provides the database of DTI with both drug information and target (protein) information. Truong generates training and test set data (random-split test set and held-out-protein-class test set) from BindingDB as set forth at p24.
Training is discussed at p29:
“Models composed of Transformers are trained for 100 epochs using Adam with a learning rate of le - 4 and otherwise default parameters. After training, an ensemble is created by combining the models from the top 10 epochs based on Pearson's r on the validation set, where the predictions from the models of the ensemble are combined via a simple mean. Additionally, we explored ensembling models trained with different weight initializations, also via a simple mean, which further improved results.”
The drug fingerprint data is tokenized as set forth at Fig 2-1 “compound representation”, as well as at p28 as cited above, meeting step b) of claim 1.
The protein sequence data is transformed, using a convolution, cut into grids followed by a pooling layer, as set forth at p25-26 as cited above, meeting step c) of claim 1.
Each transformed set is concatenated, joined, aggregated, or combined as set forth at Fig 2-1, step 5 (combine), meeting step d) of claim 1. This concatenation is further discussed at p28-29:
“Potential/Transformer model To combine the learned protein sequence embeddings ξR lx100 (for a protein of length l) and ECFP fingerprints E {0, 1 }81 92 , we concatenate the two representations by replicating the ECFP fingerprint for each residue in the protein sequence and concatenating along the protein sequence dimension, producing a representation E R1x(rno+si92). The concatenated representation is further processed by a Transformer [40] with 3 layers and a hidden dimension of 256. Larger Transformers did not improve performance. The output of the Transformer is a sequence E R1x256 , which is projected by an MLP to a sequence with one dimension E R1x1 . The sum of this sequence is taken as the binding affinity prediction. The rationale for computing the affinity as a sum is that each term of the sum conceptually represents the affinity of a residue in the protein sequence to the compound.”
The combined dataset is applied to the transformer model, as set forth at p17-18, meeting step e) of claim 1:
“In order to use structural information in a sequence-based model, we leverage recently developed learned protein sequence embeddings that encode structural information [2]. Structural information is encoded in the embeddings by training the embeddings to predict structual features such as protein structural similarity and protein contact maps. Our deep learning model processes the learned protein sequence embeddings using a Transformer [40], and computes the binding affinity as a sum of potentials, where each potential conceptually represents the affinity of a residue of the protein to the input compound… we additionally train the model to predict ligand (i.e. compound) binding residues…Then, we combine the two models by training a model to predict both binding affinity and ligand binding residues. We show that the per residue potentials predicted by the combined model are also predictive of ligand binding residues, demonstrating the interpretability of the model.” P17-18
The output of the trained model is an interaction, or a binding region, meeting step f) of claim 1. Tables 2.2-2.5 p31-32 illustrate the results.
Truong further provides transformer models specifically to identify ligand binding sites/ regions, in Chapter 3.
“Here we present the DeepLiBRe family of models, which combines learned protein residue embeddings with existing sequence and structure derived features in a novel deep learning framework to improve ligand binding residue prediction. First, to predict ligand binding residues in the absence of an experimentally solved protein structure, DeepLiBRe-sequenceonly uses a Transformer [40] to integrate heterogeneous sequence derived features, including learned protein embeddings, conservation, and predictions from template-based methods. Second, to predict ligand binding residues when structures are available, DeepLiBRe-withstructure leverages the structure by using a convolutional neural network (CNN) [22] to combine the protein's contact map with the aforementioned per residue sequence derived features. We can also incorporate per residue structure-derived features from existing geometry-based methods [14][8]. Third, to predict ligand-specific binding residues for 5 ligands, CA, MG, MN, ATP, and HEME, DeepSLiBRe-with-structure adapts DeepLiBRe-with-structure to include the ligand's molecular fingerprint [31] and ligand-specific predictions from template based methods as features. In comprehensive benchmarks, DeepLiBRe outperforms the state of the art methods for both ligand-specific and non-ligand-specific binding residue prediction. In many cases, DeepLiBRe discovers novel binding pockets not found by other methods.” P37
This transformer model is trained with a database comprising drug and protein information: BioLiP (p38). This database comprises 3D structure information and binding information, meeting dependent claim 3. The training using the database meets the limitations of dependent claim 4.
“The BioLiP database is used as the source of ground truth ligand binding residues. BioLiP
collects all protein-ligand complexes from the PDB and annotates residues as ligand binding or not. It includes only ligands that are biologically relevant e.g. molecules used as additives for solving protein structures are excluded. BioLiP is available as a redundant set and a non-redundant set (at 90% sequence identity) which are updated weekly. The non-ligand-specific models (DeepLiBRe-sequence-only and DeepLiBRe-with-structure) are tested on the COACH test set [44]. It is derived from BioLiP, and is composed of 500 non-redundant proteins. For training, we use the latest non-redundant BioLiP dataset as of January 03, 2019. To test the generalizability of the models, none of the training sequences had more than 30% sequence identity to any sequence in the test set. The 30% threshold has been used by other studies and is standard for testing the generalizability of ligand binding
residue prediction models [44][42].
The ligand-specific model (DeepSLiBRe-with-structure) is tested on five ligand-specific datasets that are derived from BioLiP and were curated by the DELIA authors [42]. The five ligands include three metal ions (CA, MG, and MN), and two biologically relevant molecules (ATP and HEME), and their corresponding test sets comprise 515, 651, 144, 41, and 96 non-redundant proteins each. For training, we use the latest non-redundant BioLiP dataset as of April 04, 2020. Five training sets were created, one for each test set, by removing sequences with more than 30% sequence identity to any sequence in the corresponding test set. The BioLiP dataset used to train the ligand-specific model is more up to date than the BioLiP dataset used to train the non-ligand-specific models because the ligand-specific test sets include data from a later version of BioLiP (i.e. later than January 03, 2019).”
Protein sequence processing for this model is discussed at pages 39-42.
Prediction of binding regions of proteins is discussed beginning at page 42.
“For our models, we use the predictions from Con Cavity [8]. Con Cavity normally combines pocket predictions from LigSite [14] with the degree of conservation of each residue. However, we run ConCavity without including the conservation data, rationalizing that it can be derived from the PSFM, which is another input to the model. For simplicity, we perform pocket predictions on per protein chain structures, which are readily available from BioLiP, rather than protein structures with experimentally determined or predicted quaternary structure. We call this specific method LigSiteC -tertiary.
As each protein sequence can be associated with multiple structures, we represent the LigSiteC -tertiary predictions as the maximum and mean prediction per residue i.e. C = (c1, c2, ... , en) where ci1 is the maximum prediction and ci2 is the mean prediction for the ith residue.”
Truong uses Morgan fingerprints for the ligand/ drug information, which can be RDkit. P43. This meets dependent claim 2.
Section 3.2.3, beginning at page 44 discloses the transformer model architecture for the binding region predictions.
“All 3 models utilize a Transformer as one of the components of the model. The architecture is the same as that of the encoder in [40], except that it consists of 3 layers instead of 6, and no positional encoding is used. Specifically, each multi-headed attention module has dimension 512 and 8 heads each, with feedforward networks of dimension 2048 between attention modules. These were the optimal hyperparameters determined based on validation set performance and a grid search over number of layers E {3, 6}, number of heads E { 4, 8}, and attention dimension E {256,512}. In each case, the feedforward dimension was set to 4 times the attention dimension. The models are trained using ADAM with a learning rate of le - 4, and otherwise default parameters provided by PyTorch 1.2. A variable batch size is used, where each batch contains up to 12000 tokens when sequences are zero-padded to the length of the longest sequence in the batch.
An ensemble of models is used to form the final prediction by averaging the predictions of 5 seperately trained models with the same hyperparameters, where the predictions for each model were formed by averaging the predictions of the top 5 epochs for that model. The top 5 epochs were decided based on the performance on a validation set of 100 sequences.” P44-45
As such, claim 1 is anticipated.
With respect to claim 5, Truong discloses use of CNN in protein sequence embedding and at page 46.
“The CNN is composed of two layers - the first layer projects the input to 50 channels via a convolution with 1 x 1 kernels, and the second layer applies a 3 x 3 kernel with 100 output channels. To form the final prediction, the output of the CNN E Rn xn xlOO is projected to a lD representation E Rnx 200 by mean and max pooling”
With respect to claim 6, the token and the grid are the same length, as set forth above, for concatenation and at p46.
“To incorporate the Morgan fingerprint, which does not have a per residue representation, the Morgan fingerprint is broadcasted along the protein sequence dimension of the input.”
With respect to claim 9, Attention scores are utilized in the predicting as set forth at p47:
“Specifically, each multi-headed attention module has dimension 512 and 8 heads each, with feedforward networks of dimension 2048 between attention modules.”
Claim(s) 1, 3-5, 7-9 is/are rejected under 35 U.S.C. 102a2 as being anticipated by Liu et al (2022).
Liu, K. et al. Attention-based neural network to predict peptide binding, presentation and immunogenicity. US 2022/0122690 A1, 4/21/2022, filed 7/16/2021, having priority to 7/17/2020.
Liu is directed to attention-based transformer networks, to predict binding regions of polypeptides, and to predict binding affinity/ target interactions. (Abstract). All of Liu’s disclosure is carried out using systems comprising processors and memory. [0065]. The architecture of the transformer network model are disclosed beginning at [0090-0091].
“[0003] This present disclosure generally relates to using machine-learning models (e.g., that include an attention mechanism) to generate predictions relating to whether peptides (e.g., mutant peptide) of interest will experience a target interaction(s) with an immunoprotein complex (IPC) (e.g., be bound to an MEW molecule, presented by an MEW molecule, be bound to a TCR, etc.), the affinity associated with such a target interaction(s), and/or the ability of the peptides to trigger an immune response.”
Training the transformer network is accomplished using peptide and MHC/TCR database information and drug fingerprint information. Immunoprotein complex information is also used. [0083-0085]
Liu obtains polypeptide sequences from a database, comprising sequence information and structure information, for possible vaccine peptides, TCR sequences and MHC molecule sequences. [0068-0070, 0083-0085]
The MHC and TCR representations meet the BRI of the “drug” compositions which are also embedded or processed as desired, including one hot encoding, SMILES, BLOSUM, et al. [0112]. Dense layers are disclosed as fully connected layers and blocks.
Liu transforms the protein sequence into a grid, using a convolution layer and max pooling. [0090-0097, 0103-0106]
“[0092] Machine-learning model 132 may include one or more encoders configured to, for example, transform an input (e.g., a sequence representation representing, for example, an amino acid sequence, a nucleic acid sequence, a codon sequence, etc.) into a higher dimensional space. An encoder may be a transformer encoder. The encoder may be configured to implement an attention-based technique and/or to include one or more attention layers (e.g., one or more self-attention layers).
[0093] In some embodiments, machine-learning model 132 may use or may omit a convolutional layer, long-short term memory unit, recurrent structure, and/or recurrent component.”
Liu concatenates the transformed data sets, as set forth at [0107] to apply to the trained transformer network, to obtain a binding region prediction, or interaction, or affinity.
“[0107] Composite subsystem 306 receives the transformed representations (e.g., transformed peptide representation 316, transformed IPC representation 322, transformed N-flank representation 328, transformed C-flank representation 334, transformed TCR representation 340, or combination thereof) that are output from initial attention subsystem 304 and performs one or more operations to generate composite representation 342. Composite representation 342 may be, for example, an aggregate of the transformed representations that are output from initial attention subsystem 304. In one or more embodiments, composite representation may include a concatenation layer that concatenates the transformed representations that are output from initial attention subsystem 304. In some embodiments, composite representation 342 includes one or more additional feature vectors (e.g., which may be added to a beginning or end of a transformed representation).”
“[0128] Interaction output 466 may include, for example, set of interaction predictions 470, set of interaction affinity predictions 472, or both with respect to one or more target interactions. An interaction prediction may include, for example, a prediction for a corresponding peptide-IPC (e.g., peptide-MHC, peptide-TCR) combination of whether the IPC (e.g., MHC, TCR) will bind to the peptide. An interaction prediction may include, for example, a prediction for a corresponding peptide-IPC (e.g., peptide-MHC) combination of whether the IPC (e.g., MHC) will present the peptide at a cell surface. Further, an interaction affinity prediction may include, for example, a prediction of an affinity for a target interaction for a corresponding peptide-IPC (e.g., peptide-MHC, peptide-TCR) combination. The target interaction may be, for example, the binding of the peptide and the IPC. The affinity for the target interaction, which may be, for example, a binding affinity, indicates a strength, tendency, and/or stability of the binding between the peptide and the IPC..”
As such, claim 1 is anticipated.
With respect to claim 3, 3D and binding information can be part of the database.
With respect to claim 4, the training uses binding region information.
With respect to claim 5 convolutional operations are disclosed as set forth above.
With respect to claim 7, modifying step e) of claim 1, Liu discloses transformation of the processed data into Query, Key and Value vectors for input into the trained network, as set forth at [0152]:
[0152] Step 304 includes determining a key vector, a value vector, and a query vector for each element in the sequence representation using a set of key weights, a set of value weights, and a set of query weights, respectively. If, for example, a sequence represented in the sequence representation includes, e.g., 20 amino acids, 20 key vectors, 20 value vectors, and 20 query vectors may be generated. An element in the sequence representation may correspond to, for example, a row or column in a 2-dimensional sequence representation (e.g., where a first dimension represents different amino acids in a sequence and a second dimension represents, for example, different components characterizing individual amino acids).
[0153] In some embodiments, the set of key weights are in the form of a key weight matrix. The key weight matrix for a particular element may have a size equal to a length of the element by a length that the key vector is to be. For example, the element may have a length of 20 (e.g, each value corresponding to a binary indication as to whether the amino acid in the sequence is the same as a specific 1 of 21 amino acids), and if a length of a key vector is to be 5 (e.g., representing 5 components or features), the key weight matrix can have a size of [5, 21]. The key weight matrix can be learned during training (e.g., and randomly initialized at the start of training).
[0154] The value vector for an element may have the same size as the key vector for the element. The value vector can be determined using a set of value weights, which may be learned during training and which may be included within a value weight matrix. The value weight matrix for a given element can have a size of the key weight matrix and/or may have a size defined based on a length of that element and a length that the value vector is to be.
[0155] The query vector for an element may have a same size as the key vector and/or the value vector for the element. The query vector can be determined using a set of query weights, which may be learned during training and which may be included within a query weight matrix. The query weight matrix for an element can have a size of the key weight matrix and/or the value weight matrix and/or may have a size defined based on a length of the element and a length that the query vector is to be.”
With respect to claim 8, multiple transformer networks are contemplated throughout.
With respect to claim 9, attention scores are used to predict the binding, interaction or relationship between the peptide and the TCR or MHC.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Maragakis, P. et al. (2020) A deep=learning view of chemical space designed to facilitate drug discovery. JCOM, vol 60, p4487-4496.
Huang, K. (28 August, 2021) Therapeutics Data Commons: Machine learning datasets and tasks for drug discovery and development. arXiv: 2012:09548v2, 48 pages.
Hu, F. et al. (2020) Structure Enhanced Protein-Drug interaction prediction using transformer and graph embedding. IEEE Int Conf on Bioinform and Biomed (BIBM), 1010-1015.
Yuyou-Weng, C.L. et al. (2019) Drug target interaction prediction using multi-task learning and co-attention. IEEE Int Conf on Bioinform and biomed (BIBM) p528-533.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARY K ZEMAN whose telephone number is 5712720723. The examiner can normally be reached on 8am-2pm M-F. Email may be sent to mary.zeman@uspto.gov if the appropriate permissions have been filed.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Larry Riggs can be reached on 571 270-3062. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARY K ZEMAN/ Primary Examiner, Art Unit 1686
Prosecution Insights