DETAILED ACTION
Applicant's response, filed 15 June 2026, has been fully considered. The following rejections and/or objections are either reiterated or newly applied. They constitute the complete set presently being applied to the instant application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 8-10 are cancelled.
Claims 1-7 and 11-20 are pending.
Claims 1-7 and 11-20 are rejected.
Priority
This application claims the benefit under 35 U.S.C. § 119(a) and 37 CFR § 1.55 to United Kingdom patent application no. GB 2107714.4 filed on May 28, 2021 and no. GB 2108956.0 filed on June 22, 2021. Therefore, the instant application has the effective filing date of May 28, 2021.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 27 May 2022, 10 January 2023, 20 November 2023, and 08 July 2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements have been considered by the examiner.
Drawings
The objection to the drawings is withdrawn, in view of submission of proper sequence disclosures.
Specification
The objection to the specification is maintained and re-recited herein.
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
The disclosure is objected to for an informality in the form of legal jargon (“said position”) found in lines 5 and 7 of the abstract.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7 and 11-20 are rejected under 35 U.S.C. 101.
Eligibility Step 1: Subject matter eligibility evaluation in accordance with MPEP § 2106:
Claims 1-7 and 11-17 are directed to a statutory category (method).
Claims 18-19 are directed to a statutory category (system).
Claim 20 is directed to a statutory category (machine).
Therefore, in accordance with MPEP § 2106.03 claims 1-20 have patent eligible subject matter.
[Eligibility Step 1: YES]
Eligibility Step 2A: This step determines whether a claim is directed to a judicial exception in accordance with MPEP § 2106.
Eligibility Step 2A -- Prong One: Limitations are analyzed to determine if the claims recite any concepts that could equate to a judicial exception (i.e. abstract idea, law of nature, or natural phenomenon). Possible judicial exceptions are explored below.
Claims 1, 18, 20: selecting two or more positions in the sequence of letters; (mental process)
performing a process to generate second data, the process comprising, for each of the selected positions in the sequence of letters, applying a language model to the sequence of letters to determine probability values for the said selected positions, wherein the second data comprises two or more probability values for each of the selected positions in the amino acid chain, wherein each probability value associated with the said a selected position is associated with a different one of the set of possible amino acids from the other probability values associated with the selected position, wherein the probability values represent conditional probabilities that a respective one of the set of possible amino acids could be found at the respective selected position given that the rest of the amino acid chain comprises the sequence of amino acids as expressed in the representation of the first data; (mathematical concept)
generating a directed graph comprising nodes and edges, wherein each node represents a combination of amino acids selected from ordered lists of amino acids for the selected positions, the ordered lists being ordered according to the associated probability values, and wherein the directed graph represents a ranking of combinations of amino acids for the selected positions according to a sum of probability values associated with the amino acids for the selected positions; (mental process)
generating output data comprising a representation of one or more alternative amino acid chains from the first data by traversing the edges and nodes of the directed graph and generating an alternative amino acid chain at each traversed node, wherein the one or more alternative amino acid chains are the identified candidate amino acid chains. (mental process)
Claim 2: The computer-implemented method of claim 1, wherein performing the process to generate second data comprises performing the process for each position in the sequence of letters to determine one or more probability values for each position. (mathematical concept)
Claim 3: The computer-implemented method of claim 1, wherein performing the process to generate second data comprises performing the process for each position in the sequence of letters to determine two or more probability values for each position (mathematical concept)
Claim 4: wherein performing the process to generate second data comprises performing the process for each position in the sequence of letters to determine one or more probability values for each position, (mathematical concept)
and wherein the method comprises determining a probability value of a second type based on the probability values associated with the respective letter for each position. (mathematical concept)
Claim 5: The computer-implemented method of claim 4, wherein the probability value of the second type is determined based on a product of the probability values associated with the respective letter for each position. (mathematical concept)
Claim 6: The computer-implemented method of claim 4, wherein the probability value of the second type is determined based on a sum of log functions of each of the probability values associated with the respective letter for each position. (mathematical concept)
Claim 7: generating a second probability value of the second type associated with an amino acid chain which is different to the amino acid chain represented in the first data; (mathematical concept)
generating third data representing a comparison of the first probability value of the second type and the second probability value of the second type. (mental process)
Claim 11: The computer-implemented method of claim 10, wherein generating the fourth data comprises determining one or more alternative amino acid chains by: determining a first ordered list of amino acids associated with a first selected position, the first ordered list being ordered according to probability values associated with each of the amino acids for the first selected position; (mental process)
determining a second ordered list of amino acids associated with a second selected position, the second ordered list being ordered according to probability values associated with each of the amino acids for the selected position; (mental process)
generating one or more alternative amino acid chains by selecting amino acids from the first ordered list and the second ordered list, wherein the selection prioritizes amino acids for each position according to the associated probability values. (mental process, mathematical concept)
Claim 12: The computer-implemented method of claim 1, wherein performing the process for the said position comprises masking a said letter at the said position (mental process)
and wherein applying the language model to the sequence of letters includes applying the language model to the sequence of letters with the said letter masked (mathematical concept, mental process).
Claim 13: The computer-implemented method of claim 1, wherein the applying the language model comprises selecting the language model from a set of one or more language models. (mental process)
Claim 14: The computer-implemented method of claim 13, wherein the language model is selected based on the first data. (mental process)
Claim 16: and training the Transformer model to identify a respective set of known amino acid chains (mathematical concept)
Limitations that generate secondary data by taking existing information and manipulating it using mathematical functions (encoding, softmax, log functions, product), describe organizing information through mathematical calculations, that could be executed by hand or on pen and paper. Training the transformer architecture, according to the current disclosure, entails inputting data (amino acid chains) to a model, and outputting numerical data, that undergoes secondary (softmax) calculations [spec: 0088-0089]. As such limitations of this nature fall within the mathematical concepts grouping of abstract ideas, exemplified by Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014).
Limitations that select, determine, or remove a particular type of data equate to analysis techniques that require mere mental observations of and notations of data. As such limitations that involve activities of this manner fall into the mental process grouping of abstract ideas.
Therefore, the claims appear to recite judicial exceptions (abstract ideas – mental processes, mathematical concepts).
[Eligibility Step 2A – Prong One: YES]
Eligibility Step 2A – Prong Two: A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. If the claim contains no additional claim elements beyond the abstract idea, the claim fails to integrate the abstract idea into a practical application (MPEP 2106.04(d)).
Eligibility Step 2B: Claim elements are probed for inventive concept equating to significantly more than the judicial exception (MPEP 2106.04(II)).
The following limitations are additional elements that are analyzed to determine if they integrate the judicial exceptions into practical applications:
Claim 1, 18, 20: obtaining first data, wherein the first data includes a representation of an amino acid chain, the representation comprising a sequence of two or more letters, wherein each letter of the sequence of letters corresponds to a respective amino acid of a set of possible amino acids and a position of each letter in the sequence of letters represents a respective position of a said amino acid in the amino acid chain; and
Claim 16: The computer-implemented method of claim 15, wherein the Transformer model is trained by: providing the Transformer model with a set of masked amino acid chains, each masked amino acid chain comprising a known amino acid chain in which at least one amino acid is masked;
Claim 17: The computer-implemented method of claim 15, wherein the method comprises obtaining a selection of a temperature value for use in the softmax function.
These limitations complete necessary data gathering activities for the claimed invention and do not place necessary limits on or integrate the abstract ideas into practical application.
[Eligibility Step 2A – Prong Two: YES]
Such data gathering activities that assess and measure data from prior processing to be used in a diagnosis are classified as insignificant extra-solution activity. These activities are considered well-known and conventional within the art, as exemplified by CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011).
[Eligibility Step 2B: NO]
Additional elements that may be categorized differently include:
Claims 1, 18, 20: wherein the language model is trained using one or more datasets representing amino acid chains.
Claim 3: wherein each probability value associated with a said position is associated with a different one of the set of possible amino acids from other probability values associated with the said position
Claim 4: wherein the one or more probability value for each position are probability values of a first type and include probability values associated with a respective letter in the sequence of letters for each position,
Claim 7: The computer-implemented method of claim 4, wherein the probability value of the second type is a first probability value of the second type,
Claim 15: The computer-implemented method of claim 1, wherein the language model comprises a Transformer model including at least an encoder and trained using the one or more datasets representing amino acid chains; and optionally, wherein an output of the Transformer model is input to a softmax function and the softmax function is dependent on a temperature value.
The limitations above specify the type of data that is gathered (amino acid chains, first type probability values) and the analysis techniques used to transform them (transformer model, probability values). Selecting a particular data source or type of data to be manipulated is classified as an insignificant extra-solution activity and does not integrate the judicial exceptions of the claimed invention as a whole into practical application per MPEP 2106.05(g).
[Eligibility Step 2A – Prong Two: YES]
The courts classify performing repetitive calculations (i.e. probability values) as a well-understood, routine, and conventional activity per Flook, 437 U.S. at 594, 198 USPQ2d at 199. Furthermore, Gao et al. (Patterns; Vol. 1(9); 2020) reviews deep learning in protein structural modeling and design and affirms the routine, well-understood, and conventional nature of transformer models, as presented within the disclosure, with the same applications and advantages.
[Eligibility Step 2B: NO]
Additional elements that may be categorized differently include:
Claim 1: A computer-implemented method for evaluating an amino acid chain, the computer implemented method comprising:
Claim 18: A computer system comprising at least one processor and at least one storage, the storage including: a trained language model which has been trained using one or more datasets representing amino acid chains; and computer-executable instructions which, when executed by the at least one processor, cause the computer system to:
Claim 19: The computer system of claim 18, wherein the computer system includes one or more user interfaces.
Claim 20: A non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed by one or more processors, cause the processors to:
The limitations recite components of generic computing systems or the implementation of a method onto generic computer environment, in the form of computationally evaluating data with a user interface and language model. Elements of this nature do not integrate the judicial exceptions into practical application when viewed separately or in the context of the invention as a whole, as the computer acts as a mere tool to execute the judicial exceptions (evaluation), exemplified by Versata Development Group v. SAP America, 793 F.3d 1306, 1335, 115 USPQ2d 1681, 1702 (Fed. Cir. 2015) and Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 573 U.S. 208, 223, 110 USPQ2d 1976, 1983 (2014).
[Eligibility Step 2A – Prong Two: YES]
Furthermore, the components when viewed separately, or in the context of a whole claimed invention are well-understood, routine, and conventional within the art and do not result in a significant improvement to technology per FairWarning IP, LLC v. Iatric Sys., 839 F.3d 1089, 1095, 120 USPQ2d 1293, 1296 (Fed. Cir. 2016) for accelerating data analysis solely from the capabilities of a general-purpose computer; Interval Licensing LLC v. AOL, Inc., 896 F.3d 1335, 1344-45, 127 USPQ2d 1553, 1559-60 (Fed. Cir. 2018) for displaying information on a computer display, without meaningful limits; and Gao et al. (Patterns; Vol. 1(9); 2020) for language models trained on amino acid sequences. Thus, the elements lack inventive concept.
[Eligibility Step 2B: NO]
As such, claims 1-20 are directed to judicial exceptions and rejected under 35 U.S.C 101, in accordance with Alice/Mayo, MPEP 2143 evaluation.
Response to Arguments
Applicant argues the amended claim limitations recite a specific, structured method for identifying candidate amino acid chains for drug development that integrate the judicial exceptions into practical application under step 2A, prong 2 (page 2, para. 3); and that the amended limitations provide an improvement to the technological process of computational drug development and protein design by using a language model to generate conditional probabilities and constructing a directed graph to identify candidate amino acid chains to make the claimed method computationally less expensive and faster than traditional structure-based modelling and optimisation algorithms (page 2, para. 4).
Examiner responds the amended claim limitations are classifies as judicial exceptions in the form of abstract ideas (mental processes and mathematical concepts); and therefore, the judicial exceptions alone cannot provide the improvement per the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981)) in subsection II of MPEP 2106.05(a)
Applicant argues the claims are further eligible at Step 2b as the ordered combination of amended claim limitations directed to generating a directed graph are not routine, well-understood, nor conventional by the previously cited prior art (page 3, para. 1).
Examiner responds the amended limitations directed towards generating a directed graph are classified as judicial exceptions in the form of abstract ideas herein and are therefore not evaluated for conventionality at Stqp 2b. Alternatively, MPEP 2106.05 (II) recites that Step 2B asks does the claim recite additional elements that amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
Applicants’ argument, that Bikard, Nambiar, and Dabre individually and combined fail to teach or suggest the amended limitations (page 5), has been fully considered and is persuasive.
As such, the previous rejections to claims 1-7 and 11-20 are withdrawn, in view of claim amendments; and the rejections to claims 8-10 are withdrawn in view of claim cancellations.
The following rejections are newly recited and necessitated by claim amendments.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-7, 11, 13-15, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Bikard (2021/0193259 A1), in view of Blanchard et al. (arXiv:1909.03469v1; 2019), Zhang et al. (University of Saskatchewan; p. 1-49; 2005), and Selifonov et al. (2002/0183934).
Bikard describes computer implemented methods and devices that can cause a processor to generate protein sequences via an autoregressive neural network; Blanchard describes accurate computation of softmax functions.
Claims 1, 18, and 20 are drawn to computer implemented methods, systems and mediums that obtain a representation of an amino acid chain (first data); select two or more positions in the sequence of letters; and compute two or more probability values for each of the selected positions via a language model (probabilistic model designed to predict the likelihood of a sequence of words), in which each probability value associated with a selected position is associated with a different one of the set of possible amino acids from the other probability values associated with the selected position.
Bikard et al. teaches obtaining a training dataset comprising protein sequences [0120]; encoding samples of the learning dataset into latent vectors, also referred to as latent codes or representations [0061] with a probabilistic encoder [0064]; obtaining probabilities for amino acids to be selected at position i in the generated sequence [0124]; incrementing index i is by one and a test is carried out to determine whether or not the value of index i has reached the length of the protein sequence to be generated, i.e. to determine whether or not probabilities have been obtained for each location of the protein sequence [0125]; selecting sequentially each amino acid of the ordered sequence [0021]; and checking the likelihood of usefulness of the ordered sequence [0021].
Therefore Bikard et al. teaches obtaining a latent vector representation of an amino acid chain; and generating two or more probabilities for each amino acid in the sequence.
Claims 1, 18, and 20 are further directed to the probability values representing conditional probabilities that a respective one of the set of possible amino acids could be found at the respective selected position, given that the rest of the amino acid chain comprises the sequence of amino acids as expressed in the first data representation; and training the language model using one or more datasets representing amino acid chains.
Bikard et al. teaches the latent vector and additional information are then applied to up-sampler to generate up-sampled latent vector, denoted z′ [0080]; generating a conditional probability distribution table wherein the probability of each amino acid is computed as a function of the previous amino acids and of the up-sampled latent vector z′ [0136]; and training the module with a protein sequence dataset [0120].
Bikard et al. further teaches use of a processing device [0267] and read only memory for storing computer programs for implementing the invention [0269].
Claims 1, 18, and 20 are further directed to generating a directed graph that includes edges and nodes, which represent a combination of amino acids selected from ordered lists of amino acids for selected positions; the lists are ordered according to the associated probability values; and the directed graph represents a ranking of combinations of amino acids for the selected positions according to a sum of probability values associated with the amino acids for the selected positions.
Bikard et al. teaches quantitative assessment of generation quality can be performed using the PFAM family Hidden Markov models, that is designed to compute the probability that a sequence is a member of a given family [0256]; and is reported as an E-value, which measures the probability of a sequence achieving the same score as the sequence in question by chance [0256].
Therefore Bikard et al. teaches generating a directed graph, in the form of a Hidden Markov model, is applicable assessment of the method.
Bikard et al. does not teach explicitly teach the nodes representing a combination of amino acids selected from ordered lists of amino acids for selected positions; the lists being ordered according to the associated probability values; nor the graph representing the sum of probability values (claims 1, 18, and 20).
Zhang et al. describes an improved fully connected Hidden Markov Model (HMM) for rational vaccine design.
Zhang et al. teaches an HMM is a statistical model (page 22, column 1) that can be used for protein modeling, gesture recognition, and facial recognition; a good biological example of using an HMM is to form a “profile HMM” which is developed from profiling a family of proteins; where the resulting model can then be used to search against databases to discover other members of the family (page 23, column 1).
Zhang et al. teaches to explain hidden Markov models, it is necessary to discuss Markov chains first (page 23, column 1); a Markov chain has some basic elements, “states”, “symbols” and transitions between states, in which each state transition is associated with a probability (page 23, column 1); all transitions going out of a state add up to 1 (page 23, column 1); and figure 2.3 is a graphical representation of a Markov chain for DNA sequences, where each circle is a state and the letters “A”, “C”, “G”, and “T” are the symbols emitted or generated by each state (page 23, column 1).
Zhang et al. teaches a Markov chain can be represented as a directed graph, where the nodes correspond to states in the model and edges correspond to state transitions (page 23, column 1).
Zhang et al. teaches an HMM can have various topologies; section 2.5.3 discussed two types of topologies, pHMM and fcHMM, and problems associated with each of them; and in response to these problems, a topology whose connectivity lies between that of a pHMM and a fcHMM, called a partially connected or pcHMM, is explored in this research (page 41, column 1), in which a pcHMM is constructed by first forming a fcHMM; then eliminating some of the transitions or states in the fcHMM to generate the pcHMM (page 41, column 1).
Zhang et al. teaches the reason to do topological reduction from a fcHMM is that in some cases, the model’s overall performance will increase as the number of transitions/states decrease (page 42, column 1); for the state removal approach, first, the states are ordered by their highest incoming transition rates (page 41 column 1); and since state removal give notably higher AROC values than transition removal, state removal is more effective than transition removal for allele HLA-A data (page 63, column 1).
Therefore Zhang et al. teaches using a Markov chain to create a directed graph, in the form of a Hidden Markov model, where nodes represent states that are ordered according to transition rates, representing summed probability values associated with for the selected states.
Zhang et al. does not explicitly teach the states representing a combination of amino acids; nor generating output data including a representation of one or more alternative amino acid chains from the first data by traversing the edges and nodes of the directed graph and generating an alternative amino acid chain at each traversed node, wherein the one or more alternative amino acid chains are the identified candidate amino acid chains (claims 1, 18, and 20).
Selifonov et al. describes methods of making character strings, polynucleotides and polypeptides with desired characteristics.
Selifonov et al. teaches generating an HMM matrix, exemplified in FIG. 15, that shows a family of 8 amino acid peptides [0221]; can capture the complete variation among the family as probabilities between all possible states, such as all possible combinations of amino acids [0220]; and in each position, the peptide can be a specific amino acid, shown as one of the 20 present in the boxes [0221], associated with a probability [0221].
Selifonov et al. further teaches HMM can be used in other ways as well, instead of applying the generated profile to identify previously unidentified family members, the HMM profile can be used as a template to generate de novo family members; and a sub-program, HMMEMIT, reads the profile and constructs de novo sequences based on that [0222].
Therefore Bikard et al. teaches a method of generating conditional probabilities of an input amino acid representation, in the form of an up-sampled latent vector, via a probabilistic model; and further teaches the model can be verified through generation of Hidden Markov directed graph. Zhang et al. teaches Hidden Markov models are directed graphs, which represent an ordered list of the sum of probabilities/transitions of states; and are often used for protein family modelling. Therefore, the process of generating a hidden Markov model, according to the techniques of Zhang et al. can be combined with the method of Bikard et al. with each element merely performing the same function as they do separately, with the results of the combination being predictable. Zhang et al. further provides motivation for one of ordinary skill in the art to use the state removal approach, which generates the ordered list of state transitions.
Selifonov et al. teaches, when Hidden Markov models are applied to protein family modeling, the states represent combinations of amino acids; and reading the graph, is a way to generate alternative amino acid sequences. It would be further obvious to one of ordinary skill in the art to combine this method with the technique of Bikard et al. in view of Zhang et al. as they are directed to the same purpose of generating pfam protein hidden Markov models and can be combined with each element merely performing the same function as they do separately, with the results of the combination being predictable.
Claim 2 is directed to generating at least one probability value for each position in the amino acid sequence.
Bikard teaches using the input to generate multiple variables [0122], one of the which is the probability for each amino acid to be selected at a given position of the sequence [0122].
Claim 3 is directed to wherein performing the process to generate second data comprises performing the process for each position in the sequence of letters to determine two or more probability values for each position, wherein each probability value associated with a said position is associated with a different one of the set of possible amino acids from other probability values associated with the said position.
Bikard teaches that the decoder predicts the probability that each amino acid will occupy a given position and the chosen amino acid is the one with the highest probability [0152].
Claim 4 is directed to wherein performing the process to generate second data comprises performing the process for each position in the sequence of letters to determine one or more probability values for each position, wherein the one or more probability value for each position are probability values of a first type and include probability values associated with a respective letter in the sequence of letters for each position, and wherein the method comprises determining a probability value of a second type based on the probability values associated with the respective letter for each position.
Bikard teaches a step of modifying probabilities obtained from the autoregressive neural network, denoted the first autoregressive neural network, as a function of probabilities obtained from a second autoregressive neural network different from the first autoregressive neural network [0025].
Claim 5 is directed to wherein the probability value of the second type is determined based on a product of the probability values associated with the respective letter for each position.
Bikard teaches that the joint data distribution is modeled as a product of conditionals [0076].
Claim 6 is directed to wherein the probability value of the second type is determined based on a sum of log functions of each of the probability values associated with the respective letter for each position.
Bikard teaches a conditional probability distribution table wherein the probability of each amino acid is computed as a function of the previous amino acids in the sequence and of the up-sampled latent vector [0042] where the output x’ is the result of a softmax function, as displayed in fig. 3 [0103]. The output of the softmax function inherently is the result of a sum of log functions, as evidenced by Blanchard (page 1, column 1).
Claim 7 is directed to wherein the probability value of the second type is a first probability value of the second type, and the method further includes: generating a second probability value of the second type associated with an amino acid chain which is different to the amino acid chain represented in the first data; and generating third data representing a comparison of the first probability value of the second type and the second probability value of the second type.
Bikard further teaches comparing the distribution from amino acids in generated sequences with patterns of amino acid occurrences of the training dataset, as displayed in figure 8 [0145].
Claim 11 is directed to wherein generating the output data includes determining one or more alternative amino acid chains by: determining a first ordered list of amino acids associated with a first selected position, the first ordered list being ordered according to probability values associated with each of the amino acids for the first selected position; determining a second ordered list of amino acids associated with a second selected position, the second ordered list being ordered according to probability values associated with each of the amino acids for the selected position; and generating one or more alternative amino acid chains by selecting amino acids from the first ordered list and the second ordered list, wherein the selection prioritizes amino acids for each position according to the associated probability values.
Bikard et al. teaches, each amino acid is determined one after the other in a loop [0117], in which at each iteration, a probability is determined for each amino acid to be the one used to generate the protein sequence [0117], based on the latent code, conditions, and probabilities associated with each amino acid in the protein sequence for the positions preceding the one that is processed; selecting the amino acid at a given position with higher probability [0117]; and generating sequence variants by choosing amino acids that do not have the highest probability but, for instance, choosing the second most probable amino acid [0117].
Additional claims are directed to selecting a language model from a set (claim 13), based on the input amino acid representation (claim 14).
Regarding claims 13 and 14, Bikard teaches the invention as described previously, wherein the autoregressive neural network is of the variational auto-encoder type or of the adversarial auto-encoder type [0028]. Bikard further teaches the main difference between the two model types is the additional information (conditions) that can be included or omitted from the input amino acid representation [0068].
Bikard does not explicitly teach that the probabilistic language model must include a Transformer model, with at least one encoder, trained on an amino acid chain dataset (claim 15).
Bikard does however teach that design choices relating to the structure of the decoder play an important role in determining the ability of the model to generate interesting novel proteins [0241] and compares many model types with different architectures [0231], trained on the same protein dataset [0246]. Bikard further teaches that other generative models can be constructed in which the encoder and decoder architectures can be modified to include attention mechanisms as described in “Attention Is All You Need”, Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, arXiv:1706.03762v4 [0300], which introduced the Transformer architecture and is cited on the applicant’s IDS.
Therefore, though Bikard does not explicitly teach the inclusion of a transformer model, it teaches, motivates, and suggests one ordinary skill in the art to consider an embodiment of the described invention which includes the Transformer architecture. It would similarly be obvious to train the model on protein/amino acid datasets, as Bikard uses the technique to compare other model types.
Claim 19 is directed to the computer system of claim 18, wherein the computer system includes one or more user interfaces.
Bikard et al. teaches including a screen for displaying data and/or serving as a graphical interface with the user [0275].
Claims 12 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Bikard (20210193259 A1), in view of Blanchard et al., Zhang et al. (University of Saskatchewan; p. 1-49; 2005) and Selifonov et al. (2002/0183934), as applied to claims 1-7, 11, 13-15, and 18-20 above, and in further view of Nambiar et al (Proceedings of 11th ACM International Conference on Bioinformatics; Computational Biology and Health Informatics; 2020).
Nambiar describes a neural network that performs numerous protein prediction tasks.
Nambiar teaches a transformer encoder architecture that is trained by inputting a tokenized amino acid sequence (page 3, column 2) and masked language modelling task that, given the input sequence, selects a random sample of tokens in the sequence to be replaced with a special token [MASK] and predicts the masked token (page 4, column 1). Nambiar further teaches feeding the aggregate sequence representation corresponding from the transformer into an output layer, which consists of a single-layer feed-forward neural network and softmax classifier (page 5, column 1).
As such, Nambiar teaches a computer-implemented method analogous to Bikard and the claimed invention. Though Bikard does not actively mask a letter of the input sequence, Bikard teaches that applying a dropout mask to some percentage of the positions in the “true context” can aid in putting more information in the latent code for amino acid predictions [0262]. Therefore, Bikard provides sufficient motivation for one of ordinary skill in the art to mask some parts of the input data, using the technique as described by Nambiar, with a reasonable expectation of success and improvement.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Bikard (20210193259 A1), in view of Blanchard, Zhang et al. (University of Saskatchewan; p. 1-49; 2005), Selifonov et al. (2002/0183934), and Nambiar, as applied to claims 1-7, 11-16, and 18-20 previously, and in further view of Dabre et al (arXiv:2009.09372, 2020).
Claim 17 is contingent upon the optional embodiment of the Transformer-based model’s output being input to a softmax function dependent on a temperature value (claim 15). The limitation, as claimed does not require prior art references, however in the interest of compact prosecution, is directed to having a selection of temperature values for the softmax function, as embodied in Nambiar.
Dabre describes the role of softmax tempering in neural network-based language models.
Dabre teaches training models for each of the softmax temperature values: 1.0 (default softmax), 1.2, 1.4, 1.6, 1.8, 2.0, 3.0, 4.0, 5.0, and 10.0 (page 3, column 2) and evaluating the softmax tempering on top of the Transformer model (Vaswani et al., 2017), because it gives the state-of-the-art results for NMT (page 3, column 2).
Therefore Bikard, Dabre, and Nambiar teach use of the Transformer architecture in their language model prediction tasks. Nambiar further inputs the transformer-encoded data into a softmax function, and Dabre describes how varying temperature values would affect such that data derived using the same technique. This information provides one of ordinary skill in the art with sufficient motivation to identically evaluate their Transformer derived softmax output after considering a selection of temperature values.
Response to Arguments
Applicant argues since Bikard's probabilities are conditioned solely on the previous locations in the sequence, Bikard does not teach or suggest probability values representing conditional probabilities given the rest of the amino acid chain, which encompasses amino acids both preceding and proceeding the selected position (page 4, para. 2).
Examiner responds Bikard et al. teaches encoding samples of the learning dataset [0061], comprising protein sequences [0120] into latent vectors, also referred to as latent codes or representations [0061]; applying the latent vector and additional information to the up-sampler to generate up-sampled latent vector, denoted z′ [0080]; and generating a conditional probability distribution table wherein the probability of each amino acid is computed as a function of the previous amino acids and of the up-sampled latent vector z′ [0136].
Therefore Bikard et al. does generate conditional probability values given as a function of the first data/entire amino acid sequence representation.
Applicant argues Bikard does not disclose the amended limitations of generating a directed graph of amino acid combinations, ranking those combinations, and traversing the graph to generate alternative amino acid chains (page 4, para. 3); and lacks any disclosure of constructing a directed graph comprising nodes and edges that represent combinations of amino acids selected from ordered lists for selected positions (page 4, para. 3).
Examiner responds Bikard et al. teaches constructing a directed graph in the form of a PFAM family Hidden Markov models, that comprises nodes and edges and evaluates the probability of combinations of amino acids; as indicated in the office action herein; however, examiner agrees Bikard et al. does not explicitly recite the other amended limitations directed to the graph. As such, the previous rejection is withdrawn and a new rejection is recited herein in view of Zhang et al. and Selifonov et al.
Double Patenting
The nonstatutory double patenting rejection to claims 1 and 18-20 is withdrawn in view of claim amendments.
Conclusion
No claims are currently allowed.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Correspondence
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Milana Thompson whose telephone number is (571)272-8740. The examiner can normally be reached Monday - Friday, 9:00-6:00 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Karlheinz Skowronek can be reached at (571) 272-1113. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.K.T./Examiner, Art Unit 1687
/Karlheinz R. Skowronek/Supervisory Patent Examiner, Art Unit 1687
Prosecution Insights