DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of the Claims
Claims 9-11, 13-14, 20-21, 23-24, 26 and 28 are canceled.
Claims 1-8, 12, 15-19, 22, 25, 27 and 29-31 are pending.
Claims 1, 3, 8, 18, 25 and 27 are objected to.
Claims 1-8, 12, 15-19, 22, 25, 27 and 29-31 are rejected.
Priority
This application US 18/279,526 (08/30/2023) is a 371 of PCT/IB2022/051740 (02/28/2022) and claims benefit of US Application 63/155,506 (03/02/2021) as reflected in the filing receipt mailed on 02/01/2024. The claims to the benefit of priority are acknowledged and the effective filing date of claims 1-8, 12, 15-19, 22, 25, 27 and 29-31 is 03/02/2021.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 08/30/2023 was considered.
Nucleotide and/or Amino Acid Sequence Disclosures
Summary of Requirements for Patent Applications Filed On Or After July 1, 2022, That Have Sequence Disclosures
37 CFR 1.831(a) requires that patent applications which contain disclosures of nucleotide and/or amino acid sequences that fall within the definitions of 37 CFR 1.831(b) must contain a “Sequence Listing XML”, as a separate part of the disclosure, which presents the nucleotide and/or amino acid sequences and associated information using the symbols and format in accordance with the requirements of 37 CFR 1.831-1.835. This “Sequence Listing XML” part of the disclosure may be submitted:
1. In accordance with 37 CFR 1.831(a) using the symbols and format requirements of 37 CFR 1.832 through 1.834 via the USPTO patent electronic filing system (see Section I.1 of the Legal Framework for Patent Electronic System (https://www.uspto.gov/PatentLegalFramework), hereinafter “Legal Framework”) in XML format, together with an incorporation by reference statement of the material in the XML file in a separate paragraph of the specification (an incorporation by reference paragraph) as required by 37 CFR 1.835(a)(2) or 1.835(b)(2) identifying:
a. the name of the XML file
b. the date of creation; and
c. the size of the XML file in bytes; or
2. In accordance with 37 CFR 1.831(a) using the symbols and format requirements of 37 CFR 1.832 through 1.834 on read-only optical disc(s) as permitted by 37 CFR 1.52(e)(1)(ii), labeled according to 37 CFR 1.52(e)(5), with an incorporation by reference statement of the material in the XML format according to 37 CFR 1.52(e)(8) and 37 CFR 1.835(a)(2) or 1.835(b)(2) in a separate paragraph of the specification identifying:
a. the name of the XML file;
b. the date of creation; and
c. the size of the XML file in bytes.
SPECIFIC DEFICIENCIES AND THE REQUIRED RESPONSE TO THIS NOTICE ARE AS FOLLOWS:
Specific deficiency - This application fails to comply with the requirements of 37 CFR 1.831-1.834 because it does not contain a “Sequence Listing XML” as a separate part of the disclosure. A “Sequence Listing XML” is required because the 08/30/2023 Drawings present unlabeled nucleotide and/or amino acid sequences that qualify for sequence identifiers( Figs. 3A-C and 12). Said required sequence identifiers must be accompanied by a “Sequence Listing XML” as a separate part of the disclosure.
Required response - Applicant must provide:
• A “Sequence Listing XML” part of the disclosure, as described above in item 1. or 2.; together with
o A statement that indicates the basis for the amendment, with specific references to particular parts of the application as originally filed, as required by 37 CFR 1.835(a)(3);
o A statement that the “Sequence Listing XML” includes no new matter as required by 37 CFR 1.835(a)(4)
AND
• A substitute specification in compliance with 37 CFR 1.52, 1.121(b)(3), and 1.125 inserting the required incorporation by reference paragraph as required by 37 CFR 1.835(a)(2), consisting of:
o A copy of the previously-submitted specification, with deletions shown with strikethrough or brackets and insertions shown with underlining (marked-up version);
o A copy of the amended specification without markings (clean version); and
o A statement that the substitute specification contains no new matter.
Specific deficiency - Sequences appearing in the specification (Figs 3A-C and 12) are not identified by sequence identifiers (i.e., “SEQ ID NO:X” or the like) in accordance with 37 CFR 1.831(c).
Required response – Applicant must provide:
A substitute specification in compliance with 37 CFR 1.52, 1.121(b)(3), and 1.125 inserting the required sequence identifiers, consisting of:
• A copy of the previously-submitted specification, with deletions shown with strikethrough or brackets and insertions shown with underlining (marked-up version);
• A copy of the amended specification without markings (clean version); and
• A statement that the substitute specification contains no new matter.
Specific deficiency - Sequences appearing in the drawings are not identified by sequence identifiers in accordance with 37 CFR 1.831(c). Sequence identifiers for sequences (i.e., “SEQ ID NO:X” or the like) must appear either in the drawings or in the Brief Description of the Drawings.
Required response – Applicant must provide:
Amended drawings in accordance with 37 CFR 1.121(d) inserting the required sequence identifiers;
AND/OR
A substitute specification in compliance with 37 CFR 1.52, 1.121(b)(3), and 1.125 inserting the required sequence identifiers (i.e., “SEQ ID NO:X” or the like) into the Brief Description of the Drawings, consisting of:
• A copy of the previously-submitted specification, with deletions shown with strikethrough or brackets and insertions shown with underlining (marked-up version);
• A copy of the amended specification without markings (clean version); and
• A statement that the substitute specification contains no new matter.
This application contains sequence disclosures in accordance with the definitions for nucleotide and/or amino acid sequences set forth in 37 CFR 1.831(a) and 1.831(b). However, this application fails to comply with the requirements of 37 CFR 1.831-1.834. The examiner has noted that nucleotide and/or amino acid sequences appearing in the drawings in Figs. 3A-C and 12 are not identified by sequence identifiers in accordance with 37 CFR 1.821(d). Sequence identifiers for nucleotide and/or amino acid sequences must appear either in the drawings or in the Brief Description of the Drawings.
Applicant must provide:
• A replacement “Sequence Listing XML” part of the disclosure, as described above in item 1. or 2., as well as
• A statement that identifies the location of all additions, deletions, or replacements of sequence information in the “Sequence Listing XML” as required by 1.835(b)(3);
• A statement that indicates support for the amendment in the application, as filed, as required by 37 CFR 1.835(b)(4);
• A statement that the “Sequence Listing XML” includes no new matter in accordance with 1.835(b)(5); and
• A substitute specification in compliance with 37 CFR 1.52, 1.121(b)(3), and 1.125 inserting the required incorporation by reference paragraph as required by 37 CFR 1.835(b)(2), consisting of:
o A copy of the previously-submitted specification, with deletions shown with strikethrough or brackets and insertions shown with underlining (marked-up version);
o A copy of the amended specification without markings (clean version); and
A statement that the substitute specification contains no new matter.
Drawings
The drawings are objected to as failing to comply with 37 CPR 1.84(p)(5) because they include illegible labels in Figs. 12 B-C and 19.
Corrected drawing sheets in compliance with 37 CPR 1.121 (d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as "amended." If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either "Replacement Sheet" or "New Sheet" pursuant to 3 7 CPR 1.121 (d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification Objections
The disclosure is objected to because it contains an embedded hyperlink and/or other form of browser-executable code in pg. 2 line 12 and pg. 50 line 7. Applicant is required to delete the embedded hyperlink and/or other form of browser-executable code; references to websites should be limited to the top-level domain name without any prefix such as http:// or other browser-executable code. See MPEP § 608.01.
Claim objections
Claims 5, 12, 17, 22, and 29 are objected to because of the following informality: the recited "TCR" should be spelled out. Appropriate correction is required.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION —The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-8, 12, 15-19, 22, 25, 27 and 29-31 are rejected under 35 U.S.C. 112(b)as being indefinite for failing to particularly point out and distinctly claim the subject matter the invention. Dependent claims are rejected similarly, unless otherwise noted below. The following issues cause the respective claims to be rejected under 112(b) as indefinite:
Claim 1 recites "to predict biophysiochemical properties of an amino acid sequence" in the preamble and "predict a biophysiochemical property" in the body of the claim. The scope of the claim is unclear because it is unclear if the claim requires one or more biophysiochemical properties. To overcome this rejection, the claim must be amended to clarify the number of properties required by the claim.
Claims 8, 18 and 25 repeat the lack of clarity above while additionally reciting "displaying, … one or more biophysiochemical properties" (last claim element).
In claim 4, the recited "relative robustly" is a term of relative or vague degree or form of association, neither defined in the specification (pg. 48 lines 1-3) nor having a well-known and sufficiently particular definition in the art and in the instant context. The disclosure is not interpreted as a definition. MPEP 2173.05(b) pertains. Although claims are interpreted in light of the specification, examples from the specification are not imported into the claims as limitations absent a clearly limiting definition in the specification. MPEP 2173.05(b) pertains.
In claims 7 and 31, the recited "about" is a term of relative or vague degree or form of association, neither defined in the specification (pg. 27 para. 1-2) nor having a well-known and sufficiently particular definition in the art and in the instant context. The disclosure is not interpreted as a definition. MPEP 2173.05(b) pertains. Although claims are interpreted in light of the specification, examples from the specification are not imported into the claims as limitations absent a clearly limiting definition in the specification. MPEP 2173.05(b) pertains.
Claim 8 recites “providing a trained… system, wherein the trained… system is generated by…” It is unclear whether the wherein clause is intended to require training the NLP system within the metes and bounds of the claimed invention, or if it is only further limiting the type of NLP system utilized in the invention such that training the system is not required within the metes and bounds of the invention. As set forth in MPEP 2111.04.I, “wherein” clauses raise the question as to the limiting effect of the language in a claim. As the claims do not recite an active step of training the system, the metes and bounds of the claims are unclear. For compact examination, it is assumed that the assay is not required to be performed. The rejection may be overcome by clarifying what steps are required to be performed.
Claims 18 and 25 repeat the issue above.
In claim 31, the recited "wherein about 10 - 20% (preferably 12-17% or 15%)" is indefinite because it recites both a broad and narrow range in the same claim. A broad range or limitation together with a narrow range or limitation that falls within the broad range or limitation (in the same claim) may be considered indefinite if the resulting claim does not clearly set forth the metes and bounds of the patent protection desired. See MPEP § 2173.05(c). In the present instance, claim 31 recites the broad recitation "wherein about 10 - 20%", and the claim also recites "(preferably 12-17% or 15%)" which is the narrower statement of the range/limitation. The claim(s) are considered indefinite because there is a question or doubt as to whether the feature introduced by such narrower language is (a) merely exemplary of the remainder of the claim, and therefore not required, or (b) a required feature of the claims.
In claim 31, the recited "preferably" is indefinite because it recites exemplary claim language (see MPEP 2173.05(d)).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless -
(a)(l) the claimed invention was patented, described in a printed publication, or in public use, on sale,
or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2 and 4-7 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Filipavicius ("Pre-training protein language models with label-agnostic binding pairs enhances performance in downstream tasks." arXiv preprint arXiv:2012.03084 (2020)), as cited on the attached Form PTO-892.
Claim 1 recites:
in a first phase, training the predictive protein language NLP system comprising a first neural network on a diversified protein sequence dataset in a self-supervised manner, wherein the first neural network comprises one or more transformers with attention and in a second phase, training the predictive protein language NLP system in a supervised manner to predict a biophysiochemical property, wherein the predictive protein language NLP system comprises features from the first phase of training
• Filipavicius teaches a Natural Language Processing model that uses self-supervised learning network to learn representations from unlabeled text using an attention-based context-aware Transformer model – a modification to the RoBERTa model (i.e. in a first phase, training the predictive protein language NLP system comprising a first neural network on a diversified protein sequence dataset in a self-supervised manner, wherein the first neural network comprises one or more transformers with attention) (pg. 1 Abstract) and, in the second step, the model is fine-tuned specifically for each downstream task in a supervised manner with labeled datasets (i.e. in a second phase, training the predictive protein language NLP system in a supervised manner to predict a biophysiochemical property, wherein the predictive protein language NLP system comprises features from the first phase of training) (pg. 2 para. 2).
Claim 2 recites:
wherein the first neural network comprises a first transformer with attention and a second transformer with attention, wherein the first transformer is trained on a first tokenized masked dataset and the second transformer is trained on a second dataset
• Filipavicius teaches two separate transformer models are applied during the pre-training and fine-tuning stages (i.e. first and second transformers) (pg. 6 para. 1); wherein said pretraining in the first step relies solely on Masked Language Modeling objective (i.e. first transformer is trained on a first tokenized masked dataset) (pg. 1 Abstract).
Claim 4 recites:
wherein the first transformer or the second transformer comprises a robustly optimized bidirectional encoder representations from transformers model
• Filipavicius teaches a transformer - Longformer - as an auto-encoding transformer which learns bidirectional contexts for sequence pair tasks like binding prediction (pg. 2 para. 8); wherein a fine-tune (i.e. robustly optimized) method is used for protein-protein binding prediction, TCR-epitope binding prediction, cellular-localization and remote homology classification tasks (i.e. wherein the first transformer or the second transformer comprises a robustly optimized bidirectional encoder representations from transformers model) (pg. 1 Abstract). The art teaches using RoBERTa (pg. 1 Abstract), which is the example provided in the spec of a robustly optimized model [0181].
Claim 5 recites:
wherein the biophysiochemical property is binding affinity of a TCR to an epitope
• Filipavicius teaches a fine-tune neural network NLP method used for protein-protein binding prediction, TCR-epitope binding prediction, cellular-localization and remote homology classification tasks (i.e. wherein the biophysiochemical property is binding affinity of a TCR to an epitope) (pg. 1 Abstract).
Claim 6 recites:
further comprising: training, in the first phase, the predictive protein language NLP system using a protein sequence dataset that has undergone individual amino acid-level tokenization, n-mer tokenization, or sub-word tokenization of respective protein sequences
• Filipavicius teaches a Byte-Pair Encoding tokenization algorithm creates a predefined size vocabulary from subwords by merging the most frequently occurring subword pairs in a bottom-up fashion, until the vocabulary size is reached, allowing efficient compression of sequences in a dataset (i.e. sub-word tokenization of respective protein sequences) (pg. 5 para. 2).
Claim 7 recites:
further comprising: training, in the first phase, the predictive protein language NLP system using a diversified protein sequence dataset, wherein about 10 - 20% of individual amino acids in the diversified protein sequence dataset are masked
• Filipavicius teaches the self-supervised pre-training network using sequences from Pfam, String, StringLF and SwissProt databases (i.e. training, in the first phase, the predictive protein language NLP system using a diversified protein sequence dataset) (pg. 3 para. 2); wherein pretraining occurs only with the masked language modeling objective, by masking out 15% of input tokens at random (i.e. wherein about 10 - 20% of individual amino acids in the diversified protein sequence dataset are masked) (pg. 3 para. 2).
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
A. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Filipavicius as applied to the 102 rejection above further in view of Rao ("Evaluating protein transfer learning with TAPE." Advances in neural information processing systems 32 (2019)), as cited on the attached Form PTO-892.
Claim 3 recites:
further comprising generating concatenated sequence and categorical embeddings from the first phase of training and providing the concatenated sequence and categorical embeddings to the second neural network for the second phase of training
• Filipavicius does not teach the recitation above. However, Rao teaches a machine learning approach applied to natural language processing of protein sequences – Tasks Assessing Protein Embeddings (TAPE) - where features are learned by self-supervised pretraining (pg. 1 Abstract); wherein, in addition to self-supervised algorithms, TAPE further supervised model increases the performance on secondary structure prediction (pg. 3 para. 2); wherein TAPE comprises three architectures: a Long Short-Term Memory (LSTM) model, a Transformer and a dilated residual network, where LSTM consists of layers with hidden units corresponding to the forward and backward language models whose output sequences are concatenated in the final layer (pg. 7 para. 3) and unlabeled sequences (i.e. first phase – self-supervised) are clustered into evolutionarily-related groups called families (i.e. categorical embeddings) (pg. 4 para. 2); wherein tasks are LSTM is tied to the residual network for the prediction of secondary structure (i.e. providing the concatenated sequence and categorical embeddings to the second neural network for the second phase of training) (pg. 9 para. 3).
Rationale for combining (MPEP §2142-2143)
Regarding claim 3, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, in the course of routine experimentation and with a reasonable expectation of success, the methods of Filipavicius in view of Rao because all references disclose machine learning methods applied to protein sequences. The motivation would have been to improve performance to outperform features learned via self-supervision on contact prediction (pg. 2 para. 5 Rao).
Therefore it would have been obvious to one of ordinary skill in the art to substitute the machine learning methods applied to protein sequences of Filipavicius to the methods by Rao because such a substitution is no more than the simple substitution of one known element for another. One of ordinary skill in the art would be able to motivated to combine the teachings in these references with a reasonable expectation of success since the described teachings pertain to machine learning methods applied to protein sequences.
B. Claims 8, 12, 15, 17-19, 22, 25 and 29-31 are rejected under 35 U.S.C. 103(a) as being unpatentable over Filipavicius ("Pre-training protein language models with label-agnostic binding pairs enhances performance in downstream tasks." arXiv preprint arXiv:2012.03084 (2020)) in view of Tenney ("The language interpretability tool: Extensible, interactive visualizations and analysis for NLP models." Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations. 2020)), as cited on the attached Form PTO-892.
Claim 8 recites:
A computer-implemented method for predicting biophysiochemical properties of an amino acid sequence using natural language processing (NLP) comprising:
providing a trained predictive protein language NLP system, wherein the trained predictive protein language NLP system is generated by: in a first phase, training a predictive protein language NPL system comprising a first neural network on one or more protein sequence datasets in a self-supervised manner, wherein the first neural network comprises one or more transformers with attention; and in a second phase, training the predictive protein language NLP system with a protein sequence dataset in a supervised manner to predict a biophysiochemical property, wherein the predictive protein language NLP system comprises features from the first phase of training
• Filipavicius teaches a Natural Language Processing model that uses self-supervised learning network to learn representations from unlabeled text using an attention-based context-aware Transformer model (i.e. in a first phase, training the predictive protein language NLP system comprising a first neural network on a diversified protein sequence dataset in a self-supervised manner, wherein the first neural network comprises one or more transformers with attention) (pg. 1 Abstract) and, in the second step, the model is fine-tuned specifically for each downstream task in a supervised manner with labeled datasets (i.e. in a second phase, training the predictive protein language NLP system in a supervised manner to predict a biophysiochemical property, wherein the predictive protein language NLP system comprises features from the first phase of training) (pg. 2 para. 2); wherein computational model RoBERTa utilizes a predefined length attention window within which the expensive quadratic complexity self-attention is computed (i.e. computer implemented method) (pg. 3 para. 4); wherein the fine-tuned neural network NLP method is used for protein-protein binding prediction, TCR-epitope binding prediction, cellular-localization and remote homology classification tasks (i.e. for predicting biophysiochemical properties of an amino acid sequence using natural language processing (NLP)) (pg. 1 Abstract).
receiving an input query, from a user interface device coupled to the trained predictive protein language NLP system, comprising a candidate amino acid sequence;
generating, by the trained predictive protein language NLP system, a prediction including one or more biophysiochemical properties for the candidate amino acid sequence; and
displaying, on a display screen of a device, the predicted one or more biophysiochemical properties for the candidate amino acid sequence
• Filipavicius teach "generating, by the trained predictive protein language NLP system, a prediction including one or more biophysiochemical properties for the candidate amino acid sequence" as a task that predicts for a given T-cell Receptor (TCR) the most likely peptide fragments that bind the TCR (peptide fragments from a larger antigen epitope that’s fragmented/digested by the T-cell and subsequently presented on its surface) where TCRs are represented by their very short CDR3 regions which are only 5-20 aa with 124,486 TCR and binding epitope pairs, and 13,832 pairs in evaluation dataset (pg. 12 para. 2).
• Filipavicius does not teach "receiving an input query, from a user interface device coupled to the trained predictive protein language NLP system, comprising a candidate amino acid sequence … and displaying, on a display screen of a device, the predicted one or more biophysiochemical properties for the candidate amino acid sequence." However, Tenney teaches Language Interpretability Tool (LIT), a toolkit and browser based user interface for NLP model understanding (pg. 107 col. 2 para. 3); wherein the tool allows dynamic datapoint generation, and an array of interactive visualizations, metrics, and modules that respond to user input (i.e. receiving step) (pg. 108 col. 2 para. 1); wherein the tool displays model predictions (i.e. displaying step), including classification, text generation, language model probabilities, and a graph visualization for structured prediction tasks (pg. 109 Table 1).
Claim 12 recites:
wherein the biophysiochemical property is binding affinity of a TCR to an epitope
• Filipavicius teaches a fine-tune neural network NLP method used for protein-protein binding prediction, TCR-epitope binding prediction, cellular-localization and remote homology classification tasks (i.e. wherein the biophysiochemical property is binding affinity of a TCR to an epitope) (pg. 1 Abstract).
Claim 15 recites:
wherein the trained predictive protein language NLP system comprises a salience module, further comprising:
generating, for display on the display screen, information from the salience module that indicates a contribution of respective amino acids to the prediction of the binding affinity
• Filipavicius does not teach "generating, for display on the display screen, information from the salience module that indicates a contribution of respective amino acids to the prediction of the binding affinity." However, Tenney teaches Language Interpretability Tool (LIT), a toolkit and browser based user interface for NLP model understanding (pg. 107 col. 2 para. 3); wherein a salience module shows heatmaps (i.e. wherein the trained predictive protein language NLP system comprises a salience module) for token-based feature attribution for a selected datapoint using techniques like local gradients and LIME (i.e. generating, for display on the display screen, information from the salience module that indicates a contribution of respective amino acids to the prediction of the binding affinity) (pg. 109 Table 1).
Claim 17 recites:
further comprising: receiving a plurality of candidate amino acid sequences; analyzing the candidate amino acid sequences; and predicting whether a candidate epitope binds to a TCR
• Filipavicius teaches T-cell receptor binding prediction where 13,832 pairs were evaluated and 124,486 TCR and binding epitope pairs were predicted (pg. 12 para. 2).
Claim 18 recites:
A system or apparatus to predict biophysiochemical properties of an amino acid sequence comprising one or more processors for executing instructions to:
provide a trained predictive protein language NLP system, wherein: in a first phase, the trained predictive protein language NLP system comprising a first neural network is trained on a protein sequence dataset in a self-supervised manner, wherein the first neural network comprises one or more transformers with attention; and in a second phase, the predictive protein language NLP system is trained in a supervised manner to predict a biophysiochemical property, wherein the predictive protein language NLP system comprises features from the first phase of training
• Filipavicius teaches a Natural Language Processing model that uses self-supervised learning network to learn representations from unlabeled text using an attention-based context-aware Transformer model (i.e. in a first phase, training the predictive protein language NLP system comprising a first neural network on a diversified protein sequence dataset in a self-supervised manner, wherein the first neural network comprises one or more transformers with attention) (pg. 1 Abstract) and, in the second step, the model is fine-tuned specifically for each downstream task in a supervised manner with labeled datasets (i.e. in a second phase, training the predictive protein language NLP system in a supervised manner to predict a biophysiochemical property, wherein the predictive protein language NLP system comprises features from the first phase of training) (pg. 2 para. 2); wherein computational model RoBERTa utilizes a predefined length attention window within which the expensive quadratic complexity self-attention is computed (i.e. system comprising one or more processors for executing instructions) (pg. 3 para. 4); wherein the fine-tuned neural network NLP method is used for protein-protein binding prediction, TCR-epitope binding prediction, cellular-localization and remote homology classification tasks (i.e. for predicting biophysiochemical properties of an amino acid sequence using natural language processing (NLP)) (pg. 1 Abstract).
receive an input query, from a user interface device coupled to the trained predictive protein language NLP system, comprising a candidate amino acid sequence;
generate, by the trained predictive protein language NLP system, a prediction including one or more biophysiochemical properties for the candidate amino acid sequence; and
display, on a display screen of a device, the predicted one or more biophysiochemical properties for the candidate amino acid sequence
• Filipavicius teach "generating, by the trained predictive protein language NLP system, a prediction including one or more biophysiochemical properties for the candidate amino acid sequence" as a task that predicts for a given T-cell Receptor (TCR) the most likely peptide fragments that bind the TCR (peptide fragments from a larger antigen epitope that’s fragmented/digested by the T-cell and subsequently presented on its surface) where TCRs are represented by their very short CDR3 regions which are only 5-20 aa with 124,486 TCR and binding epitope pairs, and 13,832 pairs in evaluation dataset (pg. 12 para. 2).
• Filipavicius does not teach "receiving an input query, from a user interface device coupled to the trained predictive protein language NLP system, comprising a candidate amino acid sequence … and displaying, on a display screen of a device, the predicted one or more biophysiochemical properties for the candidate amino acid sequence." However, Tenney teaches Language Interpretability Tool (LIT), a toolkit and browser based user interface for NLP model understanding (pg. 107 col. 2 para. 3); wherein the tool allows dynamic datapoint generation, and an array of interactive visualizations, metrics, and modules that respond to user input (i.e. receiving step) (pg. 108 col. 2 para. 1); wherein the tool displays model predictions (i.e. displaying step), including classification, text generation, language model probabilities, and a graph visualization for structured prediction tasks (pg. 109 Table 1).
Claim 19 recites:
wherein the first neural network comprises a first transformer with attention and a second transformer with attention, wherein the first transformer is trained on a first tokenized masked dataset and the second transformer is trained on a second dataset
• Filipavicius teaches a Natural Language Processing model that uses self-supervised learning network to learn representations from unlabeled text using an attention-based context-aware Transformer model (pg. 1 Abstract) and, in the second step, the model is fine-tuned specifically for each downstream task in a supervised manner with labeled datasets (i.e. second transformer using a second dataset) (pg. 2 para. 2); wherein two separate transformer models are applied during the pre-training and fine-tuning stages (i.e. first and second transformers) (pg. 6 para. 1); wherein said pretraining in the first step relies solely on Masked Language Modeling objective (i.e. first transformer is trained on a first tokenized masked dataset) (pg. 1 Abstract).
Claim 22 recites:
wherein the biophysiochemical property is binding affinity of a TCR to an epitope
• Filipavicius teaches a fine-tune neural network NLP method used for protein-protein binding prediction, TCR-epitope binding prediction, cellular-localization and remote homology classification tasks (i.e. wherein the biophysiochemical property is binding affinity of a TCR to an epitope) (pg. 1 Abstract).
Claim 25 recites:
A computer program product for predicting biophysiochemical properties of an amino acid sequence, the computer program product comprising a computer readable storage medium having instructions corresponding to a predictive protein language NLP system embodied therewith, the instructions executable by one or more processors to cause the processors to:
provide a trained predictive protein language NLP system, wherein: in a first phase, a predictive protein language NLP system comprising a first neural network is trained on a diversified protein sequence dataset in a self-supervised manner, wherein the first neural network comprises one or more transformers with attention; and in a second phase, the predictive protein language NLP system is trained with an annotated protein sequence dataset in a supervised manner to predict a biophysiochemical property, wherein the predictive protein language NLP system comprises features from the first phase of training
• Filipavicius teaches a Natural Language Processing model that uses self-supervised learning network to learn representations from unlabeled text using an attention-based context-aware Transformer model (i.e. in a first phase, training the predictive protein language NLP system comprising a first neural network on a diversified protein sequence dataset in a self-supervised manner, wherein the first neural network comprises one or more transformers with attention) (pg. 1 Abstract) and, in the second step, the model is fine-tuned specifically for each downstream task in a supervised manner with labeled datasets (i.e. in a second phase, training the predictive protein language NLP system in a supervised manner to predict a biophysiochemical property, wherein the predictive protein language NLP system comprises features from the first phase of training) (pg. 2 para. 2); wherein the fine-tune neural network NLP method is used for protein-protein binding prediction, TCR-epitope binding prediction, cellular-localization and remote homology classification tasks (i.e. for predicting biophysiochemical properties of an amino acid sequence using natural language processing (NLP)) (pg. 1 Abstract); wherein computational model RoBERTa utilizes a predefined length attention window within which the expensive quadratic complexity self-attention is computed (i.e. computer program product for predicting biophysiochemical properties of an amino acid sequence, the computer program product comprising a computer readable storage medium having instructions corresponding to a predictive protein language NLP system embodied therewith, the instructions executable by one or more processors to cause the processors) (pg. 3 para. 4).
receive an input query, from a user interface device coupled to the trained predictive protein language NLP system, comprising a candidate amino acid sequence;
generate, by the trained predictive protein language NLP system, a prediction including one or more biophysiochemical properties for the candidate amino acid sequence; and
display, on a display screen of a device, the predicted one or more biophysiochemical properties for the candidate amino acid sequence
• Filipavicius teach "generating, by the trained predictive protein language NLP system, a prediction including one or more biophysiochemical properties for the candidate amino acid sequence" as a task that predicts for a given T-cell Receptor (TCR) the most likely peptide fragments that bind the TCR (peptide fragments from a larger antigen epitope that’s fragmented/digested by the T-cell and subsequently presented on its surface) where TCRs are represented by their very short CDR3 regions which are only 5-20 aa with 124,486 TCR and binding epitope pairs, and 13,832 pairs in evaluation dataset (pg. 12 para. 2).
• Filipavicius does not teach "receiving an input query, from a user interface device coupled to the trained predictive protein language NLP system, comprising a candidate amino acid sequence … and displaying, on a display screen of a device, the predicted one or more biophysiochemical properties for the candidate amino acid sequence." However, Tenney teaches Language Interpretability Tool (LIT), a toolkit and browser based user interface for NLP model understanding (pg. 107 col. 2 para. 3); wherein the tool allows dynamic datapoint generation, and an array of interactive visualizations, metrics, and modules that respond to user input (i.e. receiving step) (pg. 108 col. 2 para. 1); wherein the tool displays model predictions (i.e. displaying step), including classification, text generation, language model probabilities, and a graph visualization for structured prediction tasks (pg. 109 Table 1).
Claim 29 recites:
wherein the biophysiochemical property is binding affinity of a TCR to an epitope
• Filipavicius teaches a fine-tune neural network NLP method used for protein-protein binding prediction, TCR-epitope binding prediction, cellular-localization and remote homology classification tasks (i.e. wherein the biophysiochemical property is binding affinity of a TCR to an epitope) (pg. 1 Abstract)
Claim 30 recites:
further comprising: training, in the first phase, the predictive protein language NLP system using a diversified protein sequence dataset that has undergone individual amino acid-level tokenization, n-mer tokenization or sub-word tokenization of respective protein sequences
• Filipavicius teaches a Byte-Pair Encoding tokenization algorithm creates a predefined size vocabulary from subwords by merging the most frequently occurring subword pairs in a bottom-up fashion, until the vocabulary size is reached, allowing efficient compression of sequences in a dataset (i.e. sub-word tokenization of respective protein sequences) (pg. 5 para. 2).
Claim 31 recites:
further comprising: training, in the first phase, the predictive protein language NLP system using the diversified protein sequence dataset, wherein about 10 - 20% (preferably 12-17% or 15%) of the individual amino acids in the diversified protein sequence dataset are masked.
• Filipavicius teaches the self-supervised pre-training network using sequences from Pfam, String, StringLF and SwissProt databases (i.e. training, in the first phase, the predictive protein language NLP system using a diversified protein sequence dataset) (pg. 3 para. 2); wherein pretraining occurs only with the masked language modeling objective, by masking out 15% of input tokens at random (i.e. wherein about 10 - 20% (preferably 12-17% or 15%) of the individual amino acids in the diversified protein sequence dataset are masked) (pg. 3 para. 2).
Rationale for combining (MPEP §2142-2143)
Regarding claims 8, 12, 15, 17-19, 22, 25 and 29-31, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, in the course of routine experimentation and with a reasonable expectation of success, the methods of Filipavicius in view of Tenney because all references disclose the investigation of NLP models. The motivation would have been to allow users to seamlessly hop between them to test local hypotheses and validate them over a dataset (pg. 107 col. 2 para. 3 Tenney).
Therefore it would have been obvious to one of ordinary skill in the art to substitute the investigation of NLP models of Filipavicius to the methods by Tenney because such a substitution is no more than the simple substitution of one known element for another. One of ordinary skill in the art would be able to motivated to combine the teachings in these references with a reasonable expectation of success since the described teachings pertain to the investigation of NLP models.
C. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Filipavicius and Tenney as applied to claim 8 above further in view of Gyori ("From word models to executable models of signaling networks using automated assembly." Molecular systems biology 13.11 (2017): MSB177651 (2017)), as cited on the attached Form PTO-892.
Claim 16 recites:
wherein the trained predictive protein language NLP system is compiled into an executable file
• Filipavicius teaches "a trained predictive protein language NLP system" as a Natural Language Processing model that uses self-supervised learning network to learn representations from unlabeled text using an attention-based context-aware Transformer model (pg. 1 Abstract).
• Neither Filipavicius or Tenney teach a "protein language NLP system compiled into an executable file." However, Gyori teaches processing natural language to identify grammatical relationships among words in a sentence, recognize named entities such as proteins, amino acids, small molecules, cell lines, etc., and link these entities to appropriate database identifiers (pg. 4 col. 1 Box 1); wherein molecular mechanisms described in simple English are read by natural language processing algorithms, converted into an intermediate representation, and assembled into executable models (pg. 1 Abstract).
Rationale for combining (MPEP §2142-2143)
Regarding claim 16, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, in the course of routine experimentation and with a reasonable expectation of success, the methods of Filipavicius and Tenney in view of Gyori because all references disclose the investigation of NLP models. The motivation would have been to incorporate a natural language executable code and develop a model more efficient with increased model transparency, thereby promoting collaboration with the broader biology community (pg. 1 Abstract Gyori).
Therefore it would have been obvious to one of ordinary skill in the art to substitute the to the investigation of NLP models of Filipavicius and Tenney to the methods by Gyori because such a substitution is no more than the simple substitution of one known element for another. One of ordinary skill in the art would be able to motivated to combine the teachings in these references with a reasonable expectation of success since the described teachings pertain to the investigation of NLP models.
D. Claim 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over Filipavicius and Tenney as applied to claim 25 above further in view of Rao as cited on the attached Form PTO-892.
Claim 27 recites:
further comprising generating concatenated representations of sequence and categorical feature embeddings from the first phase of training and providing the concatenated representations of sequence and categorical feature embeddings to the second neural network for the second phase of training
• Neither Filipavicius or Tenney teach the recitation above. However, Rao teaches a machine learning approach applied to natural language processing of protein sequences – Tasks Assessing Protein Embeddings (TAPE) - where features are learned by self-supervised pretraining (pg. 1 Abstract); wherein, in addition to self-supervised algorithms, TAPE further supervised model increases the performance on secondary structure prediction (pg. 3 para. 2); wherein TAPE comprises three architectures: a Long Short-Term Memory (LSTM) model, a Transformer and a dilated residual network, where LSTM consists of layers with hidden units corresponding to the forward and backward language models whose output sequences are concatenated in the final layer (pg. 7 para. 3) and unlabeled sequences (i.e. first phase – self-supervised) are clustered into evolutionarily-related groups called families (i.e. categorical embeddings) (pg. 4 para. 2); wherein tasks are LSTM is tied to the residual network for the prediction of secondary structure (i.e. providing the concatenated sequence and categorical embeddings to the second neural network for the second phase of training) (pg. 9 para. 3).
Rationale for combining (MPEP §2142-2143)
Regarding claim 27, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, in the course of routine experimentation and with a reasonable expectation of success, the methods of Filipavicius and Tenney in view of Rao because all references disclose the investigation of NLP models. The motivation would have been to improve performance to outperform features learned via self-supervision on contact prediction (pg. 2 para. 5 Rao).
Therefore it would have been obvious to one of ordinary skill in the art to substitute the investigation of NLP models of Filipavicius and Tenney to the methods by Rao because such a substitution is no more than the simple substitution of one known element for another. One of ordinary skill in the art would be able to motivated to combine the teachings in these references with a reasonable expectation of success since the described teachings pertain to the investigation of NLP models.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the "right to exclude" granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 4-8, 12, 15-18, 22, 25 and 29-31 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 4-7, 10, 12-18 and 22-23 of copending Application No 18/321,044. This is a provisional nonstatutory double patenting rejection.
• Reference claim 1 teaches instant claim 1.
• Reference claim 7 teaches instant claim 4.
• Reference claims 12 and 22 teach instant claims 5, 12, 22 and 29.
• Reference claim, 4-5 and 13 teach instant claims 6 and 30.
• Reference claims 6 and 14 teach instant claims 7 and 31.
• Reference claim 10 teaches instant claim 8.
• Reference claim 15 teaches instant claim 15.
• Reference claim 16 teaches instant claim 16.
• Reference claim 17 teaches instant claim 17.
• Reference claim 18 teaches instant claim 18.
• Reference claim 23 teaches instant claim 25.
Claims 2 and 19 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over copending Application No 18/321,044 as applied to claims 1 and 18 above further in view of Filipavicius ("Pre-training protein language models with label-agnostic binding pairs enhances performance in downstream tasks." arXiv preprint arXiv:2012.03084 (2020)). This is a provisional nonstatutory double patenting rejection.
• Application No 18/321,044 does not teach instant claims 2 and 19. However, Filipavicius teaches two separate transformer models are applied during the pre-training and fine-tuning stages (i.e. first and second transformers) (pg. 6 para. 1); wherein said pretraining in the first step relies solely on Masked Language Modeling objective (i.e. wherein the first neural network comprises a first transformer with attention and a second transformer with attention, wherein the first transformer is trained on a first tokenized masked dataset and the second transformer is trained on a second dataset) (pg. 1 Abstract).
Rationale for combining (MPEP §2142-2143)
Regarding claims 2 and 19, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, in the course of routine experimentation and with a reasonable expectation of success, the methods of Application No 18/321,044 in view of Filipavicius because all references disclose machine learning methods applied to protein sequences. The motivation would have been to:
• extract useful biological information from massive unlabeled datasets (pg. 2 para. 3 Filipavicius).
Therefore it would have been obvious to one of ordinary skill in the art to substitute the machine learning methods applied to protein sequences of Application No 18/321,044 to the methods by Filipavicius because such a substitution is no more than the simple substitution of one known element for another. One of ordinary skill in the art would be able to motivated to combine the teachings in these references with a reasonable expectation of success since the described teachings pertain to machine learning methods applied to protein sequences.
Claims 3 and 27 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over copending Application No 18/321,044 as applied to claims 1 and 25 above further in view of Rao ("Evaluating protein transfer learning with TAPE." Advances in neural information processing systems 32 (2019)). This is a provisional nonstatutory double patenting rejection.
• Neither Application No 18/321,044 or Filipavicius teach instant claims 3 and 27. However, Rao teaches a machine learning approach applied to natural language processing of protein sequences – Tasks Assessing Protein Embeddings (TAPE) - where features are learned by self-supervised pretraining (pg. 1 Abstract); wherein, in addition to self-supervised algorithms, TAPE further supervised model increases the performance on secondary structure prediction (pg. 3 para. 2); wherein TAPE comprises three architectures: a Long Short-Term Memory (LSTM) model, a Transformer and a dilated residual network, where LSTM consists of layers with hidden units corresponding to the forward and backward language models whose output sequences are concatenated in the final layer (pg. 7 para. 3) and unlabeled sequences (i.e. first phase – self-supervised) are clustered into evolutionarily-related groups called families (i.e. categorical embeddings) (pg. 4 para. 2); wherein tasks are LSTM is tied to the residual network for the prediction of secondary structure (i.e. providing the concatenated sequence and categorical embeddings to the second neural network for the second phase of training) (pg. 9 para. 3).
Rationale for combining (MPEP §2142-2143)
Regarding claims 3 and 27, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, in the course of routine experimentation and with a reasonable expectation of success, the methods of Application No 18/321,044 in view of Rao because all references disclose machine learning methods applied to protein sequences. The motivation would have been to:
• improve performance to outperform features learned via self-supervision on contact prediction (pg. 2 para. 5 Rao).
Therefore it would have been obvious to one of ordinary skill in the art to substitute the machine learning methods applied to protein sequences of Application No 18/321,044 to the methods by Rao because such a substitution is no more than the simple substitution of one known element for another. One of ordinary skill in the art would be able to motivated to combine the teachings in these references with a reasonable expectation of success since the described teachings pertain to machine learning methods applied to protein sequences.
Conclusion
No claims are allowed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANCINI A FONSECA LOPEZ whose telephone number is (571)270-0899. The examiner can normally be reached Monday - Friday 8AM - 5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Olivia Wise can be reached at (571) 272-2249. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/F.F.L./Examiner, Art Unit 1685
/JANNA NICOLE SCHULTZHAUS/Examiner, Art Unit 1685
Prosecution Insights