DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-36 are currently pending and examined on the merits.
Claims 1-36 are rejected.
Claims 8, 15, 24-25, 28, and 32 are objected to.
Priority
The instant application claims priority to U.S. Provisional Applications 63/294,830, 63/294,828, 63/294,827, 63/294,820, 63/294,816, and 63/294,813 filed on 29 December 2021. At this point in examination, the effective filing date of claims 1-36 is 29 December 2021.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 28 July 2023, 14 June 2024, 15 August 2024, 25 September 2025, 12 February 2026, and 5 March 2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements have been considered by the examiner.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because of the following:
Reference characters “2292” and “2294” have both been used to designate “Ground Truth” in Figure 22.
Reference characters “2296” and “2298” have both been used to designate “Back Propagation” in Figure 22.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings, submitted 16 September 2022, require correction because several nucleotide and/or amino acid sequences do not meet the proper disclosure requirements, as detailed below.
Nucleotide and/or Amino Acid Sequence Disclosures
Summary of Requirements for Patent Applications Filed On Or After July 1, 2022, That Have Sequence Disclosures
37 CFR 1.831(a) requires that patent applications which contain disclosures of nucleotide and/or amino acid sequences that fall within the definitions of 37 CFR 1.831(b) must contain a “Sequence Listing XML”, as a separate part of the disclosure, which presents the nucleotide and/or amino acid sequences and associated information using the symbols and format in accordance with the requirements of 37 CFR 1.831-1.835. This “Sequence Listing XML” part of the disclosure may be submitted:
1. In accordance with 37 CFR 1.831(a) using the symbols and format requirements of 37 CFR 1.832 through 1.834 via the USPTO patent electronic filing system (see Section I.1 of the Legal Framework for Patent Electronic System (https://www.uspto.gov/PatentLegalFramework), hereinafter “Legal Framework”) in XML format, together with an incorporation by reference statement of the material in the XML file in a separate paragraph of the specification (an incorporation by reference paragraph) as required by 37 CFR 1.835(a)(2) or 1.835(b)(2) identifying:
a. the name of the XML file
b. the date of creation; and
c. the size of the XML file in bytes; or
2. In accordance with 37 CFR 1.831(a) using the symbols and format requirements of 37 CFR 1.832 through 1.834 on read-only optical disc(s) as permitted by 37 CFR 1.52(e)(1)(ii), labeled according to 37 CFR 1.52(e)(5), with an incorporation by reference statement of the material in the XML format according to 37 CFR 1.52(e)(8) and 37 CFR 1.835(a)(2) or 1.835(b)(2) in a separate paragraph of the specification identifying:
a. the name of the XML file;
b. the date of creation; and
c. the size of the XML file in bytes.
SPECIFIC DEFICIENCIES AND THE REQUIRED RESPONSE TO THIS NOTICE ARE AS FOLLOWS:
Specific deficiency - This application fails to comply with the requirements of 37 CFR 1.831-1.834 because it does not contain a “Sequence Listing XML” as a separate part of the disclosure. A “Sequence Listing XML” is required because Figures 5 and 16 in the .
Required response - Applicant must provide:
• A “Sequence Listing XML” part of the disclosure, as described above in item 1. or 2.; together with
o A statement that indicates the basis for the amendment, with specific references to particular parts of the application as originally filed, as required by 37 CFR 1.835(a)(3);
o A statement that the “Sequence Listing XML” includes no new matter as required by 37 CFR 1.835(a)(4)
AND
• A substitute specification in compliance with 37 CFR 1.52, 1.121(b)(3), and 1.125 inserting the required incorporation by reference paragraph as required by 37 CFR 1.835(a)(2), consisting of:
o A copy of the previously-submitted specification, with deletions shown with strikethrough or brackets and insertions shown with underlining (marked-up version);
o A copy of the amended specification without markings (clean version); and
o A statement that the substitute specification contains no new matter.
Specific deficiency - Sequences appearing in the drawings are not identified by sequence identifiers in accordance with 37 CFR 1.831(c). Sequence identifiers for sequences (i.e., “SEQ ID NO:X” or the like) must appear either in the drawings or in the Brief Description of the Drawings.
The sequence disclosures are located in the drawings filed on 16 September 2022: Figures 5 and 16.
Required response – Applicant must provide:
Amended drawings in accordance with 37 CFR 1.121(d) inserting the required sequence identifiers;
AND/OR
A substitute specification in compliance with 37 CFR 1.52, 1.121(b)(3), and 1.125 inserting the required sequence identifiers (i.e., “SEQ ID NO:X” or the like) into the Brief Description of the Drawings, consisting of:
• A copy of the previously-submitted specification, with deletions shown with strikethrough or brackets and insertions shown with underlining (marked-up version);
• A copy of the amended specification without markings (clean version); and
• A statement that the substitute specification contains no new matter.
Specification
Para. [0266], line 9 in the instant specification contains a hyperlink. The disclosure is objected to because it contains an embedded hyperlink and/or other form of browser-executable code. Applicant is required to delete the embedded hyperlink and/or other form of browser-executable code; references to websites should be limited to the top-level domain name without any prefix such as http:// or other browser-executable code. See MPEP § 608.01.
Claim Objections
Claims 8, 15, 24-25, 28, and 32 are objected to because of the following informalities:
In claims 8, 15, 24-25, 28, and 32, line 1, "claims" should read "claim".
There is a typographical error. Appropriate correction is required.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 22 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “about at least” in claim 22, line 4 is a relative term which renders the claim indefinite. The term “about at least” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It is unclear the range of amino acids that could be comprised within "about at least" one amino acid in protein sequences. The specification is also silent as to how many amino acids is .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-36 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite: (a) mathematical concepts, (e.g., mathematical relationships, formulas or equations, mathematical calculations); and (b) mental processes, i.e., concepts performed in the human mind, (e.g., observation, evaluation, judgement, opinion).
Subject matter eligibility evaluation in accordance with MPEP 2106:
Eligibility Step 1: Claims 1-36 are directed to a system (machine) for inter-model prediction score recalibration. Therefore, these claims are encompassed by the categories of statutory subject matter, and thus satisfy the subject matter eligibility requirements under Step 1.
[Step 1: YES]
Eligibility Step 2A: First, it is determined in Prong One whether a claim recites a judicial exception, and if so, then it is determined in Prong Two whether the recited judicial exception is integrated into a practical application of that exception.
Eligibility Step 2A, Prong One: In determining whether a claim is directed to a judicial exception, examination is performed that analyzes whether the claim recites a judicial exception, i.e., whether a law of nature, natural phenomenon, or abstract idea is set forth described in the claim.
Claims 1-36 recite the following steps which fall within the mental processes and/or mathematical concepts groups of abstract ideas, as noted below.
Independent claim 1 further recites:
a first model configured to generate, based in part on evolutionary conservation summary statistics of amino acids in a reference target protein sequence, a first set of pathogenicity scores for a set of variants that mutate the reference target protein sequence to a set of alternate protein sequences, wherein the first set of pathogenicity scores has a first set of score rankings (i.e., mental processes, mathematical concepts);
a second model configured to generate, based in part on epistasis expressed by amino acid patterns spanning a multiple sequence alignment that aligns the reference target protein sequence to a plurality of non-target protein sequences, a second set of pathogenicity scores for the set of variants, wherein the second set of pathogenicity scores has a second set of score rankings (i.e., mental processes, mathematical concepts);
a rank loss determination logic configured to determine a rank loss parameter based on a comparison of the first set of score rankings against the second set of score rankings (i.e., mental processes, mathematical concepts);
a loss function reconfiguration logic configured to reconfigure a loss function based on the rank loss parameter (i.e., mental processes, mathematical concepts);
a training logic configured to use the reconfigured loss function to train the first model (i.e., mental processes, mathematical concepts).
Dependent claim 2 further recites:
wherein the second model processes respective alternate protein sequences in the set of alternate protein sequences as respective inputs and generates respective pathogenicity scores in the second set of pathogenicity scores as respective outputs (i.e., mental processes, mathematical concepts).
Dependent claim 3 further recites:
wherein the second model is pre-trained to process the multiple sequence alignment as an input and generate a reconstruction of the multiple sequence alignment as an output (i.e., mental processes, mathematical concepts).
Dependent claim 4 further recites:
wherein the second model represents a reconstruction of a given alternate protein sequence as base-wise probability scores for each amino acid in the given alternate protein sequence (i.e., mental processes, mathematical concepts).
Dependent claim 5 further recites:
wherein a joint probability determined from the base-wise probability scores is used as a pathogenicity score for a given variant that mutates the reference target protein sequence to the given alternate protein sequence (i.e., mental processes, mathematical concepts).
Dependent claim 6 further recites:
wherein respective coefficient and latent space configurations of the second model are pre-trained to process and reconstruct respective multiple sequence alignments that have respective reference target protein sequences as respective query sequences (i.e., mental processes, mathematical concepts).
Dependent claim 7 further recites:
wherein the second model has a particular coefficient and latent space configuration corresponding to the reference target protein sequence (i.e., mental processes, mathematical concepts).
Dependent claim 8 further recites:
wherein the second model has one to twenty thousand coefficient and latent space configurations corresponding to one to twenty thousand reference protein sequences in human proteome (i.e., mental processes, mathematical concepts).
Dependent claim 9 further recites:
wherein the rank loss determination logic is further configured to determine the rank loss parameter based on a combination of the first set of score rankings and the second set of score rankings (i.e., mental processes, mathematical concepts).
Dependent claims 10 and 18 further recite:
wherein the combination is a weighted combination (i.e., mental processes, mathematical concepts).
Dependent claim 11 further recites:
wherein the weights used to generate the weighted combination are preset (i.e., mental processes, mathematical concepts).
Dependent claim 12 further recites:
wherein the weights are differentiable and learned in a re-ranking layer that is trained as part of the training of the first model (i.e., mental processes, mathematical concepts).
Dependent claim 15 further recites:
a third model configured to generate, based in part on the epistasis expressed by the amino acid patterns spanning the multiple sequence alignment, a third set of pathogenicity scores for the set of variants, wherein the third set of pathogenicity scores has a third set of score rankings (i.e., mental processes, mathematical concepts);
the rank loss determination logic further configured to determine the rank loss parameter based on a comparison of the first set of score rankings, the second set of score rankings, and the third set of score rankings (i.e., mental processes, mathematical concepts);
the loss function reconfiguration logic further configured to reconfigure the loss function based on the rank loss parameter (i.e., mental processes, mathematical concepts);
the training logic further configured to use the reconfigured loss function to train the first model (i.e., mental processes, mathematical concepts).
Dependent claim 17 further recites:
wherein the rank loss determination logic is further configured to determine the rank loss parameter based on a combination of the first set of score rankings, the second set of score rankings, and the third set of score rankings (i.e., mental processes, mathematical concepts).
Dependent claim 19 further recites:
wherein weights used to generate the weighted combination are preset (i.e., mental processes, mathematical concepts).
Dependent claim 20 further recites:
wherein the weights are differentiable and learned as part of the training of the first model (i.e., mental processes, mathematical concepts).
Dependent claim 21 further recites:
wherein the weights are differentiable and learned in stacked re-ranking layers that are trained as part of the training of the first model using activation functions that generate non-linear combinations of the first set of score rankings, the second set of score rankings, and the third set of score rankings (i.e., mental processes, mathematical concepts).
Dependent claim 22 further recites:
a fourth model configured to generate, based in part on masked representations of the evolutionary conservation summary statistics, a fourth set of pathogenicity scores for the set of variants, wherein the masked representations mask evolutionary conservation summary statistic data about at least one amino acid in the alternate protein sequences, and wherein the fourth set of pathogenicity scores has a fourth set of score rankings (i.e., mental processes, mathematical concepts);
the rank loss determination logic further configured to determine the rank loss parameter based on a comparison of the first set of score rankings, the second set of score rankings, and the fourth set of score rankings (i.e., mental processes, mathematical concepts);
the loss function reconfiguration logic further configured to reconfigure the loss function based on the rank loss parameter (i.e., mental processes, mathematical concepts);
the training logic further configured to use the reconfigured loss function to train the first model (i.e., mental processes, mathematical concepts).
Dependent claim 23 further recites:
the rank loss determination logic further configured to determine the rank loss parameter based on a comparison of the first set of score rankings, the second set of score rankings, the third set of score rankings, and the fourth set of score rankings (i.e., mental processes, mathematical concepts);
the loss function reconfiguration logic further configured to reconfigure the loss function based on the rank loss parameter (i.e., mental processes, mathematical concepts);
the training logic further configured to use the reconfigured loss function to train the first model (i.e., mental processes, mathematical concepts).
Dependent claim 24 further recites:
wherein the training logic is further configured to use the reconfigured loss function to train the fourth model (i.e., mental processes, mathematical concepts).
Dependent claim 25 further recites:
the loss function reconfiguration logic further configured to reconfigure, based on the rank loss parameter, a first loss function for the first model and a fourth loss function for the fourth model (i.e., mental processes, mathematical concepts);
the training logic further configured to use the reconfigured first function to train the first model, and to use the reconfigured fourth function to train the fourth model (i.e., mental processes, mathematical concepts).
Dependent claim 26 further recites:
wherein the first model is further configured to generate, based in part on three-dimensional (3D) structural representations of amino acids in the reference target protein sequence, the first set of pathogenicity scores (i.e., mental processes, mathematical concepts).
Dependent claim 27 further recites:
wherein the first model is further configured to generate, based in part on the reference target protein sequence, the first set of pathogenicity scores (i.e., mental processes, mathematical concepts).
Dependent claim 28 further recites:
wherein the first model is further configured to generate, based in part on the alternate protein sequences, the first set of pathogenicity scores (i.e., mental processes, mathematical concepts).
Dependent claim 29 further recites:
wherein the fourth model is further configured to generate, based in part on masked representations of the 3D structural representations of the amino acids in the reference target protein sequence, the fourth set of pathogenicity scores, wherein the masked representations of the 3D structural representations mask 3D structural data about at least one amino acid in the reference target protein sequence (i.e., mental processes, mathematical concepts).
Dependent claim 30 further recites:
wherein the fourth model is further configured to generate, based in part on a masked representation of the reference target protein sequence, the fourth set of pathogenicity scores, wherein the masked representation masks at least one amino acid in the reference target protein sequence (i.e., mental processes, mathematical concepts).
Dependent claim 31 further recites:
wherein the fourth model is further configured to generate, based in part on masked representations of the alternate protein sequences, the fourth set of pathogenicity scores, wherein the masked representations of the alternate protein sequences mask at least one amino acid in the reference target protein sequence (i.e., mental processes, mathematical concepts).
Dependent claim 32 further recites:
wherein the evolutionary conservation summary statistics are determined from evolutionary profiles (i.e., mental processes, mathematical concepts).
Dependent claim 33 further recites:
wherein the evolutionary profiles include position-specific score matrices (PSSMs) (i.e., mental processes, mathematical concepts).
Dependent claim 34 further recites:
wherein the evolutionary profiles include position-specific frequency matrices (PSFMs) (i.e., mental processes, mathematical concepts).
Dependent claim 35 further recites:
wherein the reference target protein sequence is a sub-sequence in a region in the reference target protein sequence (i.e., mental processes, mathematical concepts).
Dependent claim 36 further recites:
wherein the alternate protein sequences are sub-sequences in regions in the alternate protein sequences (i.e., mental processes, mathematical concepts);
The abstract ideas recited in the claims are evaluated under the broadest reasonable interpretation (BRI) of the claim limitations when read in light of and consistent with the specification. As noted in the foregoing section, the claims are determined to contain limitations that can practically be performed in the human mind with the aid of a pencil and paper, and therefore recite judicial exceptions from the mental process grouping of abstract ideas. Additionally, the recited limitations that are identified as judicial exceptions from the mathematical concepts grouping of abstract ideas are abstract ideas irrespective of whether or not the limitations are practical to perform in the human mind.
Therefore, claims 1-12, 15, and 17-36 recite an abstract idea.
[Step 2A, Prong One: YES]
Eligibility Step 2A, Prong Two: In determining whether a claim is directed to a judicial exception, further examination is performed that analyzes if the claim recites additional elements that, when examined as a whole, integrates the judicial exception(s) into a practical application (MPEP 2106.04(d)). A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. The claimed additional elements are analyzed to determine if the abstract idea is integrated into a practical application (MPEP 2106.04(d)(I); MPEP 2106.05(a-h)). If the claim contains no additional elements beyond the abstract idea, the claim fails to integrate the abstract idea into a practical application (MPEP 2106.04(d)(III)).
The judicial exceptions identified in Eligibility Step 2A, Prong One are not integrated into a practical application because of the reasons noted below.
Claims 2-12, 15, and 17-36 do not recite any elements in addition to the judicial exception, and thus are part of the judicial exception.
Claim 13 recites wherein the second model is a variational autoencoder (VAE). The variational autoencoder is used to generate a second set of pathogenicity scores, which provides nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). Therefore, the claimed additional element does not integrate the abstract ideas into a practical application.
Claim 14 recites wherein the second model is a generative adversarial network (GAN). The generative adversarial network is used to generate a second set of pathogenicity scores, which provides nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). Therefore, the claimed additional element does not integrate the abstract ideas into a practical application.
Claim 16 recites wherein the third model is a Transformer-based model. The Transformer-based model is used to generate a third set of pathogenicity scores, which provides nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). Therefore, the claimed additional element does not integrate the abstract ideas into a practical application.
Claim 1 recite the additional non-abstract element (EIA) of a general-purpose computer system or parts thereof:
a system (claim 1).
The EIA do not provide any details of how specific structures of the computer elements are used to implement the JE. The claims require nothing more than a general-purpose computer to perform the functions that constitute the judicial exceptions. The computer elements of the claims do not provide improvements to the functioning of the computer itself (as in DDR Holdings, LLC v. Hotels.com LP); they do not provide improvements to any other technology or technical field (as in Diamond v. Diehr); nor do they utilize a particular machine (as in Eibel Process Co. v. Minn. & Ont. Paper Co.). Hence, these are mere instructions to apply the JE using a computer, and therefore the claim does not recite integrate that JE into a practical application.
Thus, the additionally recited elements merely invoke a computer as a tool, and/or amount to insignificant extra-solution data gathering activity, and as such, when all limitations in claims 1-5, 7, 9-12, and 14 have been considered as a whole, the claims are deemed to not recite any additional elements that would integrate a judicial exception into a practical application. Claims 1-2 and 9-10 contain additional elements that would not integrate a judicial exception into a practical application and are further probed for inventive concept in Step 2B.
[Step 2A, Prong Two: NO]
Eligibility Step 2B: Because the claims recite an abstract idea, and do not integrate that abstract idea into a practical application, the claims are probed for a specific inventive concept. The judicial exception alone cannot provide that inventive concept or practical application (MPEP 2106.05). Identifying whether the additional elements beyond the abstract idea amount to such an inventive concept requires considering the additional elements individually and in combination to determine if they amount to significantly more than the judicial exception (MPEP 2106.05A i-vi).
The claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception(s) because of the reasons noted below.
With respect to claim 1: The limitations identified above as non-abstract elements (EIA) related to general-purpose computer systems do not rise to the level of significantly more than the judicial exception. These elements do not improve the functioning of the computer itself, or comprise an improvement to any other technical field (Trading Technologies Int’l v. IBG, TLI Communications). They do not require or set forth a particular machine (Ultramercial v. Hulu, LLC., Alice Corp. Pty. Ltd v. CLS Bank Int’l), they do not affect a transformation of matter, nor do they provide an unconventional step. Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception are insufficient to provide significantly more (as discussed in Alice Corp., CyberSource v. Retail Decisions, Parker v. Flook, Versata Development Group v. SAP America).
The additional element of wherein the second model is a variational autoencoder (claim 13) is conventional. Evidence for conventionality is shown by Wei et al. (IEEE Access, 2020, 1-18). Wei et al. reviews “Accurate prediction of the effects of sequence variation is a major goal in biological research … Riesselman et al. [58] used VAEs to infer biological sequences from large multi-sequence alignments and predict the effects of mutations and organize sequence information, all while being grounded with biologically motivated architecture learned in unsupervised fashion” (pg. 9, col. 2, para. 2, lines 1-11). This shows that variational autoencoders are used to predict mutation effect scores, which makes it a conventional element in the art.
The additional element of wherein the second model is a generative adversarial network (GAN) (claim 14) is conventional. Evidence for conventionality is shown by Gao et al. (Patterns, Cell Press Open Access, 2020, 1-23). Gao et al. reviews “Repecka et al. trained ProteinGAN on the bacterial enzyme malate dehydrogenase (MDH) to generate new enzyme sequences that were active and soluble in vitro, some with over 100 mutations, with a 24% success rate.” (pg. 14, col. 1, para. 2, lines 5-9). This shows that generative adversarial networks can reveal potential mutations and pathogenicity in protein sequence analysis, which makes it a conventional element in the art.
The additional element of wherein the third model is a Transformer-based model (claim 16) is conventional. Evidence for conventionality is shown by Gao et al. (Patterns, Cell Press Open Access, 2020, 1-23). Gao et al. reviews “Rives et al. trained a transformer model with 670 million parameters on 86 billion amino acids across 250 million protein sequences spanning evolutionary diversity. Their transformer model was superior to traditional LSTM-based models on tasks, such as the prediction of secondary structure and long-range contacts, as well as the effect of mutations on activity on deep mutational scanning benchmarks.” (pg. 7, col. 1, para. 2, lines 18-25). This shows that a Transformer-based model can predict mutation effect, which makes it a conventional element in the art.
[Step 2B: NO]
Therefore, claims 1-36 are patent ineligible under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-15, 17-21, 26-28, 32-33, and 35-36 are rejected under 35 U.S.C. 103 as being unpatentable over Sundaram et al. (Nature Genetics, 2018, 50, 1-29), as provided in the IDS filed 7/28/2023, in view of Riesselman et al. (Nature Methods, 2018, 15, 1-27), as provided in the IDS filed 7/28/2023, and Pasumarthi et al. (Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019, 2970-2978), as provided in the IDS filed 7/28/2023.
With respect to claim 1:
With respect to the recited a first model configured to generate, based in part on evolutionary conservation summary statistics of amino acids in a reference target protein sequence, a first set of pathogenicity scores for a set of variants that mutate the reference target protein sequence to a set of alternate protein sequences, wherein the first set of pathogenicity scores has a first set of score rankings, Sundaram et al. discloses “Architecture of the deep residual network for pathogenicity prediction, PrimateAI. Predicted pathogenicity is on a scale from 0 (benign) to 1 (pathogenic). The network takes as input the human amino acid (AA) reference and alternate sequence (51 AAs) centered at the variant, the position weight matrix (PWM) conservation profiles calculated from 99 vertebrate species” (pg. 24, Fig. 3, lines 1-5). This suggests a model PrimateAI that generates pathogenicity scores for variants that mutate a reference protein sequence to an alternate protein sequence based on position weight matrix conservation profiles. The prediction scores have score rankings produced from the Wilcoxon rank-sum test (pg. 25, Fig. 3, lines 6-8).
Sundaram et al. does not disclose a second model configured to generate, based in part on epistasis expressed by amino acid patterns spanning a multiple sequence alignment that aligns the reference target protein sequence to a plurality of non-target protein sequences, a second set of pathogenicity scores for the set of variants, wherein the second set of pathogenicity scores has a second set of score rankings.
However, Riesselman et al. discloses “we developed nonlinear latent-variable models for biological sequence families and leveraged approximate inference techniques to infer the families from large multiple-sequence alignments. We show how a Bayesian deep latent-variable model can be used to reveal latent structure in sequence families and predict the effects of mutations” (pg. 2, para. 4, lines 1-4). Also, further discloses “We introduce a nonlinear latent-variable model
p
(
x
|
Ѳ
)
to implicitly capture higher-order interactions between positions in a sequence.” (pg. 3, para. 2, lines 1-2). Riesselman et al. discloses “To determine where the model over- or underpredicted the
∆
E
for each mutation, we transformed the experimental measurements and mutation-effect predictions to normalized ranks on the interval [0,1].” (pg. 15, para. 4, lines 2-4). This suggests a model that generates pathogenicity predictions based on epistatic interactions between positions in multiple sequence alignments that aligns a reference protein sequence to non-target protein sequences. These predictions are further transformed into normalized rankings.
Sundaram et al. and Riesselman et al. do not disclose a rank loss determination logic configured to determine a rank loss parameter based on a comparison of the first set of score rankings against the second set of score rankings.
However, Pasumarthi et al. discloses “the pairwise logistic loss is defined as:
l
^
y
,
y
^
=
∑
j
=
1
n
∑
k
=
1
n
I
y
j
>
y
k
l
o
g
(
1
+
exp
y
^
k
-
y
^
j
)
)
where
I
(
∙
)
is the indicator function.” (pg. 2972, col. 2, para. 4, lines 6-9). Also, further discloses “pairwise [5, 22] or listwise [7, 42, 43] methods either model the pairwise preferences or define a loss over entire ranked list.” (pg. 2970-2971, col. 2, para. 6, lines 9-11). This suggests a rank loss determination logic that determines a loss parameter based on a comparison between two ranked scores.
Sundaram et al. and Riesselman et al. do not disclose a loss function reconfiguration logic configured to reconfigure a loss function based on the rank loss parameter.
However, Pasumarthi et al. discloses “Losses in TensorFlow are functions that take in inputs, labels and a weight, and return a weighted loss value. The library has a pre-defined set of pointwise, pairwise and listwise ranking losses. The loss key is an enum over supported loss functions. These losses are exposed using the factory function tfr.losses.make_loss_fn that takes a loss key (name) and a weights tensor and returns a loss function compatible with Estimator.” (pg. 2975, col. 1, para. 3, lines 2-8). This suggests a loss function reconfiguration logic that takes in a loss key parameter and returns a reconfigured loss function.
Sundaram et al. and Riesselman et al. do not disclose a training logic configured to use the reconfigured loss function to train the first model.
However, Pasumarthi et al. discloses a ranking head object that uses the reconfigured loss function generated from tfr.losses.make_loss_fn to train a model (pg. 2975, col. 2, para. 2, lines 9-12 in code snippet).
It would have been prima facie obvious to one of ordinary skill in the art to combine the first model disclosed by Sundaram et al. with the second model disclosed by Riesselman et al. and the logic disclosed by Pasumarthi et al. One would be motivated to combine the teachings because the predictions of the deep latent-variable model disclosed by Riesselman et al. were more accurate than those of a previously published pairwise-interaction approach to model epistasis, which in turn was more accurate than commonly used supervised methods (pg. 6, para. 3, lines 3-6). This means that it will be able to support the first model in generating more accurate pathogenicity scores, later trained with each other. Pasumarthi et al. also discloses that their TensorFlow Ranking library can be used to solve real-world, large-scale ranking problems with hundreds of millions of training examples, and scales well to large clusters (pg. 2977, col. 2, para. 3, lines 5-8). This means that the library is capable of training multitudes of examples between the first and second model using their score rankings. There is a likelihood of success, since all teachings involve neural network models for determining metrics, which are well known techniques in the field of bioinformatics.
With respect to claim 2:
Sundaram et al. does not disclose wherein the second model processes respective alternate protein sequences in the set of alternate protein sequences as respective inputs and generates respective pathogenicity scores in the second set of pathogenicity scores as respective outputs.
However, Riesselman et al. discloses “We calculated the mutation effect prediction by taking the difference of the mean of 2,000 ELBO samples of the wild-type and a given mutated sequence.” (pg. 14, para. 2, lines 2-3). This suggests that the model processes alternate protein sequences and generates pathogenicity prediction scores.
With respect to claim 3:
Sundaram et al. does not disclose wherein the second model is pre-trained to process the multiple sequence alignment as an input and generate a reconstruction of the multiple sequence alignment as an output.
However, Riesselman et al. discloses “DeepSequence and other mutation-effect-prediction methods train on multiple-sequence alignments” (pg. 7, para. 2, lines 3-4). Also, further discloses “We model both the conditional distribution of the generative model and the approximate posterior with neural networks, which results in a flexible model-inference combination known as a variational autoencoder (VAE)” (pg. 3-4, para. 2, lines 15-18). This suggests that the model is pre-trained using the multiple sequence alignment as input and generates a reconstruction of the multiple sequence alignment as output, which is depicted in Figure 1b.
With respect to claim 4:
Sundaram et al. does not disclose wherein the second model represents a reconstruction of a given alternate protein sequence as base-wise probability scores for each amino acid in the given alternate protein sequence.
However, Riesselman et al. discloses “Our probabilistic model is specified by two components: a prior distribution
p
(
z
)
that specifies the marginal distribution of the hidden variables
z
, and a conditional distribution
p
(
x
|
z
,
Ѳ
)
that specifies how a sequence
x
is generated given the hidden variables.” (pg. 9, para. 3, lines 1-3). Also, further discloses “We model both the conditional distribution of the generative model and the approximate posterior with neural networks, which results in a flexible model-inference combination known as a variational autoencoder (VAE)” (pg. 3-4, para. 2, lines 15-18). Riesselman et al. discloses “each letter
x
i
is conditionally independent of every other position given the hidden variables
z
” (pg. 10, para. 1, lines 4-5). This suggests that the reconstruction of a protein sequence is represented in base-wise probability scores for each amino acid in the sequence.
With respect to claim 5:
Sundaram et al. does not disclose wherein a joint probability determined from the base-wise probability scores is used as a pathogenicity score for a given variant that mutates the reference target protein sequence to the given alternate protein sequence.
However, Riesselman et al. discloses “We model both the conditional distribution of the generative model and the approximate posterior with neural networks, which results in a flexible model-inference combination known as a variational autoencoder (VAE) (Fig. 1b). After the model is fit to a given family through optimization of the variational parameters
Ф
, it can be readily applied to predict the effects of arbitrary types and numbers of mutations. We quantify effects with an approximation to the log-ratio by replacing each log probability with the ELBO (Fig. 2).” (pg. 3-4, para. 2, lines 15-21). Also, further discloses “We consider the log-ratio
log
p
(
x
M
u
t
a
n
t
|
Ѳ
)
p
(
x
W
i
l
d
-
t
y
p
e
|
Ѳ
)
as a heuristic metric for the relative favorability of a mutated sequence,
x
(
M
u
t
a
n
t
)
, compared with that of a wild-type sequence,
x
(
W
i
l
d
-
t
y
p
e
)
. This log-ratio heuristic has been shown to accurately predict the effects of mutations across multiple kinds of generative models
p
(
x
|
Ѳ
)
.” (pg. 3, para. 1, lines 8-13). Riesselman et al. discloses “We calculated the mutation effect prediction by taking the difference of the mean of 2,000 ELBO samples of the wild-type and a given mutated sequence.” (pg. 14, para. 2, lines 2-3). This suggests that ELBO approximates the joint probability determined from base-wise probability scores for mutant vs. wild-type sequences, which is used as a pathogenicity score.
With respect to claim 6:
Sundaram et al. does not disclose wherein respective coefficient and latent space configurations of the second model are pre-trained to process and reconstruct respective multiple sequence alignments that have respective reference target protein sequences as respective query sequences.
However, Riesselman et al. discloses “Five latent variable models were fit to each alignment, as well as a pairwise and independent model using the same sequence weighting and model fitting techniques.” (pg. 16, para. 3, lines 5-7). Also, further discloses “we obtained multiple-sequence alignments of the corresponding protein family in five search iterations of the profile HMM homology search tool jackhmmer against the UniRef100 database of nonredundant protein sequences” (pg. 7, para. 5, lines 2-4). Riesselman et al. discloses “We model both the conditional distribution of the generative model and the approximate posterior with neural networks, which results in a flexible model-inference combination known as a variational autoencoder (VAE)” (pg. 3-4, para. 2, lines 15-18). This suggests that coefficient weighting and latent space configurations were pre-trained to each sequence alignment and reconstruct the multiple sequence alignments that have respective target and query protein sequences using the variational autoencoder.
With respect to claim 7:
Sundaram et al. does not disclose wherein the second model has a particular coefficient and latent space configuration corresponding to the reference target protein sequence.
However, Riesselman et al. discloses “Five latent variable models were fit to each alignment, as well as a pairwise and independent model using the same sequence weighting and model fitting techniques.” (pg. 16, para. 3, lines 5-7). This suggests that a particular coefficient weighting and latent space configuration corresponds to a target protein sequence.
With respect to claim 8:
Sundaram et al. does not disclose wherein the second model has one to twenty thousand coefficient and latent space configurations corresponding to one to twenty thousand reference protein sequences in human proteome.
However, Riesselman et al. discloses “Five latent variable models were fit to each alignment, as well as a pairwise and independent model using the same sequence weighting and model fitting techniques.” (pg. 16, para. 3, lines 5-7). Also, further discloses “We compared predictions from DeepSequence to a collection of 42 high-throughput mutational scans (712,218 mutations across 108 sets of experiments on 34 proteins and a tRNA…)” (pg. 4, para. 4, lines 1-3). This suggests that there is at least one coefficient weighting and latent space configuration corresponding to at least one reference protein sequences in human proteome in the sets of experiments.
With respect to claim 9:
Sundaram et al. and Riesselman et al. do not disclose wherein the rank loss determination logic is further configured to determine the rank loss parameter based on a combination of the first set of score rankings and the second set of score rankings.
However, Pasumarthi et al. discloses “the pairwise logistic loss is defined as:
l
^
y
,
y
^
=
∑
j
=
1
n
∑
k
=
1
n
I
y
j
>
y
k
l
o
g
(
1
+
exp
y
^
k
-
y
^
j
)
)
where
I
(
∙
)
is the indicator function.” (pg. 2972, col. 2, para. 4, lines 6-9). Also, further discloses “pairwise [5, 22] or listwise [7, 42, 43] methods either model the pairwise preferences or define a loss over entire ranked list.” (pg. 2970-2971, col. 2, para. 6, lines 9-11). This suggests a rank loss determination logic that determines a loss parameter based on a combination of two scores over all pairs of ranked scores.
With respect to claims 10 and 18:
Sundaram et al. and Riesselman et al. do not disclose wherein the combination is a weighted combination.
However, Pasumarthi et al. discloses “One proposed method [40, 41] is to compute Inverse Propensity Weights (IPW) for each position in the ranked list. By incorporating these scores in the training process (usually by way of re-weighting items during loss computation), one may produce a better ranking function.” (pg. 4, para. 2, lines 2-6). This suggests weights computed for each position in the ranked lists before passing into the loss function to create a weighted combination.
With respect to claim 11:
Sundaram et al. and Riesselman et al. do not disclose wherein the weights used to generate the weighted combination are preset.
However, Pasumarthi et al. discloses “One proposed method [40, 41] is to compute Inverse Propensity Weights (IPW) for each position in the ranked list. By incorporating these scores in the training process (usually by way of re-weighting items during loss computation), one may produce a better ranking function.” (pg. 4, para. 2, lines 2-6). This suggests weights computed for each position in the ranked lists during training before passing into the loss function to create a weighted combination.
With respect to claim 12:
Sundaram et al. and Riesselman et al. do not disclose wherein the weights are differentiable and learned in a re-ranking layer that is trained as part of the training of the first model.
However, Pasumarthi et al. discloses “TensorFlow enables such propagation of gradients through automatic differentiation [1]: each operation in the computation graph is equipped with a gradient expression with respect to its input tensors. In this way, the gradient of a complex composition of tensor operations can be automatically inferred during a backward pass through the computation graph. This allows for composition of a large number of operations to construct deeper networks.” (pg. 4, col. 2, para. 2, lines 6-12). This suggests a backpropagation algorithm for training neural networks, where weight tensors are differentiable and learned through the backward pass layer and updated using gradients.
With respect to claim 13:
Sundaram et al. does not disclose wherein the second model is a variational autoencoder (VAE).
However, Riesselman et al. discloses “We model both the conditional distribution of the generative model and the approximate posterior with neural networks, which results in a flexible model-inference combination known as a variational autoencoder (VAE) (Fig. 1b). After the model is fit to a given family through optimization of the variational parameters
Ф
, it can be readily applied to predict the effects of arbitrary types and numbers of mutations.” (pg. 3-4, para. 2, lines 15-20). This suggests that the second model is a variational autoencoder.
With respect to claim 14:
Sundaram et al. does not disclose wherein the second model is a generative adversarial network (GAN).
However, Riesselman et al. discloses “While this work was in progress, other nonlinear latent-variable models were proposed for sequence families, evidencing the benefits of more parametrically powerful models for sequence variation.” (pg. 6, para. 4, lines 4-7). This suggests that the second model could also be a generative adversarial network, which is a nonlinear latent variable model.
With respect to claim 15:
Sundaram et al. does not disclose a third model configured to generate, based in part on the epistasis expressed by the amino acid patterns spanning the multiple sequence alignment, a third set of pathogenicity scores for the set of variants, wherein the third set of pathogenicity scores has a third set of score rankings.
However, Riesselman et al. discloses “we developed nonlinear latent-variable models for biological sequence families and leveraged approximate inference techniques to infer the families from large multiple-sequence alignments. We show how a Bayesian deep latent-variable model can be used to reveal latent structure in sequence families and predict the effects of mutations” (pg. 2, para. 4, lines 1-4). Also, further discloses “We introduce a nonlinear latent-variable model
p
(
x
|
Ѳ
)
to implicitly capture higher-order interactions between positions in a sequence.” (pg. 3, para. 2, lines 1-2). Riesselman et al. discloses “To determine where the model over- or underpredicted the
∆
E
for each mutation, we transformed the experimental measurements and mutation-effect predictions to normalized ranks on the interval [0,1].” (pg. 15, para. 4, lines 2-4). This suggests a model that generates pathogenicity predictions based on epistatic interactions between positions in multiple sequence alignments that aligns a reference protein sequence to non-target protein sequences. These predictions are further transformed into normalized rankings.
Sundaram et al. and Riesselman et al. do not disclose the rank loss determination logic further configured to determine the rank loss parameter based on a comparison of the first set of score rankings, the second set of score rankings, and the third set of score rankings.
However, Pasumarthi et al. discloses “Finally, as an example of a listwise loss [11], our library provides the implementation of Softmax Cross-Entropy, ListNet [7], and ListMLE [43] among others. For example, the Softmax Cross-Entropy loss is defined as follows:
l
^
y
,
y
^
=
-
∑
j
=
1
n
y
j
l
o
g
(
exp
y
^
j
∑
j
=
1
n
exp
y
^
j
)
” (pg. 2972, col. 2, para. 4, lines 9-13). Also, further discloses “pairwise [5, 22] or listwise [7, 42, 43] methods either model the pairwise preferences or define a loss over entire ranked list.” (pg. 2970-2971, col. 2, para. 6, lines 9-11). Pasumarthi et al. discloses “the model may need entire lists of items during training (to compute a listwise loss, for example)” (pg. 2973, col. 2, para. 3, lines 13-15). This suggests a rank loss determination logic that determines a loss parameter based on a comparison between three or more sets of ranked scores.
Sundaram et al. and Riesselman et al. do not disclose the loss function reconfiguration logic further configured to reconfigure the loss function based on the rank loss parameter.
However, Pasumarthi et al. discloses “Losses in TensorFlow are functions that take in inputs, labels and a weight, and return a weighted loss value. The library has a pre-defined set of pointwise, pairwise and listwise ranking losses. The loss key is an enum over supported loss functions. These losses are exposed using the factory function tfr.losses.make_loss_fn that takes a loss key (name) and a weights tensor and returns a loss function compatible with Estimator.” (pg. 2975, col. 1, para. 3, lines 2-8). This suggests a loss function reconfiguration logic that takes in a loss key parameter and returns a reconfigured loss function.
Sundaram et al. and Riesselman et al. do not disclose the training logic further configured to use the reconfigured loss function to train the first model.
However, Pasumarthi et al. discloses a ranking head object that uses the reconfigured loss function generated from tfr.losses.make_loss_fn to train a model (pg. 2975, col. 2, para. 2, lines 9-12 in code snippet).
With respect to claim 17:
Sundaram et al. and Riesselman et al. do not disclose wherein the rank loss determination logic is further configured to determine the rank loss parameter based on a combination of the first set of score rankings, the second set of score rankings, and the third set of score rankings.
However, Pasumarthi et al. discloses “Finally, as an example of a listwise loss [11], our library provides the implementation of Softmax Cross-Entropy, ListNet [7], and ListMLE [43] among others. For example, the Softmax Cross-Entropy loss is defined as follows:
l
^
y
,
y
^
=
-
∑
j
=
1
n
y
j
l
o
g
(
exp
y
^
j
∑
j
=
1
n
exp
y
^
j
)
” (pg. 2972, col. 2, para. 4, lines 9-13). Also, further discloses “pairwise [5, 22] or listwise [7, 42, 43] methods either model the pairwise preferences or define a loss over entire ranked list.” (pg. 2970-2971, col. 2, para. 6, lines 9-11). Pasumarthi et al. discloses “the model may need entire lists of items during training (to compute a listwise loss, for example)” (pg. 2973, col. 2, para. 3, lines 13-15). This suggests a rank loss determination logic that determines a loss parameter based on a combination of three or more sets of ranked scores.
With respect to claim 19:
Sundaram et al. and Riesselman et al. do not disclose wherein weights used to generate the weighted combination are preset.
However, Pasumarthi et al. discloses “One proposed method [40, 41] is to compute Inverse Propensity Weights (IPW) for each position in the ranked list. By incorporating these scores in the training process (usually by way of re-weighting items during loss computation), one may produce a better ranking function.” (pg. 4, para. 2, lines 2-6). This suggests weights computed for each position in the ranked lists during training before passing into the loss function to create a weighted combination.
With respect to claim 20:
Sundaram et al. and Riesselman et al. do not disclose wherein the weights are differentiable and learned as part of the training of the first model.
However, Pasumarthi et al. discloses “TensorFlow enables such propagation of gradients through automatic differentiation [1]: each operation in the computation graph is equipped with a gradient expression with respect to its input tensors. In this way, the gradient of a complex composition of tensor operations can be automatically inferred during a backward pass through the computation graph. This allows for composition of a large number of operations to construct deeper networks.” (pg. 4, col. 2, para. 2, lines 6-12). This suggests a backpropagation algorithm for training neural networks, where weight tensors are differentiable and learned through the backward pass layer and updated using gradients.
With respect to claim 21:
Sundaram et al. and Riesselman et al. do not disclose wherein the weights are differentiable and learned in stacked re-ranking layers that are trained as part of the training of the first model using activation functions that generate non-linear combinations of the first set of score rankings, the second set of score rankings, and the third set of score rankings.
However, Pasumarthi et al. discloses “TensorFlow enables such propagation of gradients through automatic differentiation [1]: each operation in the computation graph is equipped with a gradient expression with respect to its input tensors. In this way, the gradient of a complex composition of tensor operations can be automatically inferred during a backward pass through the computation graph. This allows for composition of a large number of operations to construct deeper networks.” (pg. 4, col. 2, para. 2, lines 6-12). Also, further discloses “We consider a simple 3-layer feedforward neural network with ReLU [32] non-linear activation units” (pg. 2976, col. 2, para. 1, lines 1-3). Pasumarthi et al. discloses “The scoring function h is typically parameterized by a set of parameters Ѳ” (pg. 2972, col. 1, para. 3, lines 1-2). This suggests that weight tensors are differentiable and learned through the 3-layer neural network trained using ReLU activation functions that generate non-linear combinations of scores.
With respect to claim 26:
With respect to the recited wherein the first model is further configured to generate, based in part on three-dimensional (3D) structural representations of amino acids in the reference target protein sequence, the first set of pathogenicity scores, Sundaram et al. discloses “To incorporate information about protein structure, we trained separate networks to predict the secondary structure and solvent accessibility from the sequence alone, and then included these as subnetworks in the full model (Fig. 3b and Supplementary Fig. 5).” (pg. 5, para. 4, lines 6-9). This suggests that the model can generate pathogenicity scores based on 3D structural information of amino acids in target protein sequences.
With respect to claim 27:
With respect to the recited wherein the first model is further configured to generate, based in part on the reference target protein sequence, the first set of pathogenicity scores, Sundaram et al. discloses “Architecture of the deep residual network for pathogenicity prediction, PrimateAI. Predicted pathogenicity is on a scale from 0 (benign) to 1 (pathogenic). The network takes as input the human amino acid (AA) reference and alternate sequence (51 AAs) centered at the variant, the position weight matrix (PWM) conservation profiles calculated from 99 vertebrate species” (pg. 24, Fig. 3, lines 1-5). This suggests that the model can generate pathogenicity scores based on a reference protein sequence.
With respect to claim 28:
With respect to the recited wherein the first model is further configured to generate, based in part on the alternate protein sequences, the first set of pathogenicity scores, Sundaram et al. discloses “Architecture of the deep residual network for pathogenicity prediction, PrimateAI. Predicted pathogenicity is on a scale from 0 (benign) to 1 (pathogenic). The network takes as input the human amino acid (AA) reference and alternate sequence (51 AAs) centered at the variant, the position weight matrix (PWM) conservation profiles calculated from 99 vertebrate species” (pg. 24, Fig. 3, lines 1-5). This suggests that the model can generate pathogenicity scores based on alternate protein sequences.
With respect to claim 32:
With respect to the recited wherein the evolutionary conservation summary statistics are determined from evolutionary profiles, Sundaram et al. discloses “Architecture of the deep residual network for pathogenicity prediction, PrimateAI. Predicted pathogenicity is on a scale from 0 (benign) to 1 (pathogenic). The network takes as input the human amino acid (AA) reference and alternate sequence (51 AAs) centered at the variant, the position weight matrix (PWM) conservation profiles calculated from 99 vertebrate species” (pg. 24, Fig. 3, lines 1-5). This indicates that the evolutionary conservation summary statistics are determined from position weight matrix conservation profiles.
With respect to claim 33:
With respect to the recited wherein the evolutionary profiles include position-specific score matrices (PSSMs), Sundaram et al. discloses “Architecture of the deep residual network for pathogenicity prediction, PrimateAI. Predicted pathogenicity is on a scale from 0 (benign) to 1 (pathogenic). The network takes as input the human amino acid (AA) reference and alternate sequence (51 AAs) centered at the variant, the position weight matrix (PWM) conservation profiles calculated from 99 vertebrate species” (pg. 24, Fig. 3, lines 1-5). This indicates that the conservation profiles include position weight matrices, which are also position-specific score matrices.
With respect to claim 35:
With respect to the recited wherein the reference target protein sequence is a sub-sequence in a region in the reference target protein sequence, Sundaram et al. discloses “Architecture of the deep residual network for pathogenicity prediction, PrimateAI. Predicted pathogenicity is on a scale from 0 (benign) to 1 (pathogenic). The network takes as input the human amino acid (AA) reference and alternate sequence (51 AAs) centered at the variant, the position weight matrix (PWM) conservation profiles calculated from 99 vertebrate species” (pg. 24, Fig. 3, lines 1-5). This suggests that the reference target protein sequence is a subsequence centered at the variant in the reference target protein sequences, also depicted in Figure 3.
With respect to claim 36:
With respect to the recited wherein the alternate protein sequences are sub-sequences in regions in the alternate protein sequences, Sundaram et al. discloses “Architecture of the deep residual network for pathogenicity prediction, PrimateAI. Predicted pathogenicity is on a scale from 0 (benign) to 1 (pathogenic). The network takes as input the human amino acid (AA) reference and alternate sequence (51 AAs) centered at the variant, the position weight matrix (PWM) conservation profiles calculated from 99 vertebrate species” (pg. 24, Fig. 3, lines 1-5). This suggests that the alternate protein sequence is a subsequence centered at the variant in the alternate protein sequences, also depicted in Figure 3.
Claims 16, 22-25, and 29-31 are rejected under 35 U.S.C. 103 as being unpatentable over Sundaram et al. (Nature Genetics, 2018, 50, 1-29), as provided in the IDS filed 7/28/2023, Riesselman et al. (Nature Methods, 2018, 15, 1-27), as provided in the IDS filed 7/28/2023, and Pasumarthi et al. (Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019, 2970-2978), as provided in the IDS filed 7/28/2023, as applied to claims 1-15, 17-21, 26-28, 32-33, and 35-36 above, in view of Rives et al. (Proceedings of the National Academy of Sciences, 2021, 118(15), 1-12).
Sundaram et al., Riesselman et al., and Pasumarthi et al. are applied to claims 1-15, 17-21, 26-28, 32-33, and 35-36 above.
With respect to claim 16:
Sundaram et al., Riesselman et al., and Pasumarthi et al. do not disclose wherein the third model is a Transformer-based model.
However, Rives et al. discloses “The Transformer processes inputs through a series of blocks that alternate self-attention with feed-forward connections. Self-attention allows the network to build up complex representations that incorporate context from across the sequence. Since self-attention explicitly constructs pairwise interactions between all positions in the sequence, the Transformer architecture directly represents residue-residue interactions.” (pg. 2, col. 1, para. 8, lines 1-7). Also, further discloses “We pretrain separate 12-layer Transformer models on the Pfam multiple sequence alignments” (pg. 8, col. 1, para. 2, lines 2-4). Rives et al. discloses “We adapt the Transformer protein language model to predict the quantitative effect of mutations.” (pg. 9, col. 1, para. 3, lines 7-8). This suggests a Transformer-based model that generates pathogenicity scores based on epistatic interactions spanning multiple sequence alignments.
With respect to claim 22:
Sundaram et al., Riesselman et al., and Pasumarthi et al. do not disclose a fourth model configured to generate, based in part on masked representations of the evolutionary conservation summary statistics, a fourth set of pathogenicity scores for the set of variants, wherein the masked representations mask evolutionary conservation summary statistic data about at least one amino acid in the alternate protein sequences, and wherein the fourth set of pathogenicity scores has a fourth set of score rankings.
However, Rives et al. discloses “We train models using the masked language modeling objective (6). Each input sequence is corrupted by replacing a fraction of the amino acids with a special mask token. The network is trained to predict the missing tokens from the corrupted sequence … For each sequence
x
, we sample a set of indices
M
to mask, replacing the true token at each index
i
with the mask token. For each masked token, we independently minimize the negative log likelihood of the true amino acid
x
i
given the masked sequence
x
/
M
as context. Intuitively, to make a prediction for a masked position, the model must identify dependencies between the masked site and the unmasked parts of the sequence.“ (pg. 2, col. 2, para. 1, lines 1-13). Also, further discloses “We adapt the Transformer protein language model to predict the quantitative effect of mutations.” (pg. 9, col. 1, para. 3, lines 7-8). This suggests a model that generates pathogenicity scores for a set of variants based on masked representations of summary statistics, which masks at least one amino acid in the protein sequences. The scores generated from this model are evaluated and ranked for performance in Table 1.
It would have been prima facie obvious to one of ordinary skill in the art to combine the teachings from Sundaram et al., Riesselman et al., and Pasumarthi et al. with the Transformer-based model disclosed by Rives et al. One would be motivated to combine the teachings to include a Transformer-based model because Transformer models have better ECE values (10.45 and 11.79, respectively) than the small and large LSTM models, which shows that the Transformer enables higher fidelity modeling of protein sequences for a comparable number of parameters (pg. 3, col. 1, para. 1, lines 1-5). ECE is an exponentiated cross entropy metric that describes the mean uncertainty of the model among its set of options for every prediction, ranging from one for an ideal model to 25 (the number of unique amino acid tokens in the data) for a completely random prediction (pg. 2, col. 2, para. 5, lines 4-10). This means that the Transformer-based model makes more accurate predictions, therefore will support the other models in generating more accurate pathogenicity scores. There is a likelihood of success, since all teachings involve neural network models for determining metrics, which are well known techniques in the field of bioinformatics.
Sundaram et al., Riesselman et al., and Rives et al. do not disclose the rank loss determination logic further configured to determine the rank loss parameter based on a comparison of the first set of score rankings, the second set of score rankings, and the fourth set of score rankings.
However, Pasumarthi et al. discloses “Finally, as an example of a listwise loss [11], our library provides the implementation of Softmax Cross-Entropy, ListNet [7], and ListMLE [43] among others. For example, the Softmax Cross-Entropy loss is defined as follows:
l
^
y
,
y
^
=
-
∑
j
=
1
n
y
j
l
o
g
(
exp
y
^
j
∑
j
=
1
n
exp
y
^
j
)
” (pg. 2972, col. 2, para. 4, lines 9-13). Also, further discloses “pairwise [5, 22] or listwise [7, 42, 43] methods either model the pairwise preferences or define a loss over entire ranked list.” (pg. 2970-2971, col. 2, para. 6, lines 9-11). Pasumarthi et al. discloses “the model may need entire lists of items during training (to compute a listwise loss, for example)” (pg. 2973, col. 2, para. 3, lines 13-15). This suggests a rank loss determination logic that determines a loss parameter based on a comparison between three or more sets of ranked scores.
Sundaram et al., Riesselman et al., and Rives et al. do not disclose the loss function reconfiguration logic further configured to reconfigure the loss function based on the rank loss parameter.
However, Pasumarthi et al. discloses “Losses in TensorFlow are functions that take in inputs, labels and a weight, and return a weighted loss value. The library has a pre-defined set of pointwise, pairwise and listwise ranking losses. The loss key is an enum over supported loss functions. These losses are exposed using the factory function tfr.losses.make_loss_fn that takes a loss key (name) and a weights tensor and returns a loss function compatible with Estimator.” (pg. 2975, col. 1, para. 3, lines 2-8). This suggests a loss function reconfiguration logic that takes in a loss key parameter and returns a reconfigured loss function.
Sundaram et al., Riesselman et al., and Rives et al. do not disclose the training logic further configured to use the reconfigured loss function to train the first model.
However, Pasumarthi et al. discloses a ranking head object that uses the reconfigured loss function generated from tfr.losses.make_loss_fn to train a model (pg. 2975, col. 2, para. 2, lines 9-12 in code snippet).
With respect to claim 23:
Sundaram et al., Riesselman et al., and Rives et al. do not disclose the rank loss determination logic further configured to determine the rank loss parameter based on a comparison of the first set of score rankings, the second set of score rankings, the third set of score rankings, and the fourth set of score rankings.
However, Pasumarthi et al. discloses “Finally, as an example of a listwise loss [11], our library provides the implementation of Softmax Cross-Entropy, ListNet [7], and ListMLE [43] among others. For example, the Softmax Cross-Entropy loss is defined as follows:
l
^
y
,
y
^
=
-
∑
j
=
1
n
y
j
l
o
g
(
exp
y
^
j
∑
j
=
1
n
exp
y
^
j
)
” (pg. 2972, col. 2, para. 4, lines 9-13). Also, further discloses “pairwise [5, 22] or listwise [7, 42, 43] methods either model the pairwise preferences or define a loss over entire ranked list.” (pg. 2970-2971, col. 2, para. 6, lines 9-11). Pasumarthi et al. discloses “the model may need entire lists of items during training (to compute a listwise loss, for example)” (pg. 2973, col. 2, para. 3, lines 13-15). This suggests a rank loss determination logic that determines a loss parameter based on a comparison between three or more sets of ranked scores.
Sundaram et al., Riesselman et al., and Rives et al. do not disclose the loss function reconfiguration logic further configured to reconfigure the loss function based on the rank loss parameter.
However, Pasumarthi et al. discloses “Losses in TensorFlow are functions that take in inputs, labels and a weight, and return a weighted loss value. The library has a pre-defined set of pointwise, pairwise and listwise ranking losses. The loss key is an enum over supported loss functions. These losses are exposed using the factory function tfr.losses.make_loss_fn that takes a loss key (name) and a weights tensor and returns a loss function compatible with Estimator.” (pg. 2975, col. 1, para. 3, lines 2-8). This suggests a loss function reconfiguration logic that takes in a loss key parameter and returns a reconfigured loss function.
Sundaram et al., Riesselman et al., and Rives et al. do not disclose the training logic further configured to use the reconfigured loss function to train the first model.
However, Pasumarthi et al. discloses a ranking head object that uses the reconfigured loss function generated from tfr.losses.make_loss_fn to train a model (pg. 2975, col. 2, para. 2, lines 9-12 in code snippet).
With respect to claim 24:
Sundaram et al., Riesselman et al., and Rives et al. do not disclose wherein the training logic is further configured to use the reconfigured loss function to train the fourth model.
However, Pasumarthi et al. discloses a ranking head object that uses the reconfigured loss function generated from tfr.losses.make_loss_fn to train a model (pg. 2975, col. 2, para. 2, lines 9-12 in code snippet).
With respect to claim 25:
Sundaram et al., Riesselman et al., and Rives et al. do not disclose the loss function reconfiguration logic further configured to reconfigure, based on the rank loss parameter, a first loss function for the first model and a fourth loss function for the fourth model.
However, Pasumarthi et al. discloses “Losses in TensorFlow are functions that take in inputs, labels and a weight, and return a weighted loss value. The library has a pre-defined set of pointwise, pairwise and listwise ranking losses. The loss key is an enum over supported loss functions. These losses are exposed using the factory function tfr.losses.make_loss_fn that takes a loss key (name) and a weights tensor and returns a loss function compatible with Estimator.” (pg. 2975, col. 1, para. 3, lines 2-8). Also, further discloses “this decomposition also provides modularity and the ability to switch between various combinations of scoring functions and ranking heads.” (pg. 2974, col. 1, para. 2, lines 1-3). Pasumarthi et al. discloses “The library is flexible and highly configurable: it provides an easy-to-use API to support different scoring mechanisms, loss functions, example weights, and evaluation metrics.” (pg. 2971, col. 1, para. 4, second bullet). This suggests the ability of a loss function reconfiguration logic that takes in loss key parameters and returns a combination of reconfigured loss functions.
Sundaram et al., Riesselman et al., and Rives et al. do not disclose the training logic further configured to use the reconfigured first function to train the first model, and to use the reconfigured fourth function to train the fourth model.
However, Pasumarthi et al. discloses a ranking head object that uses the reconfigured loss function generated from tfr.losses.make_loss_fn to train a model (pg. 2975, col. 2, para. 2, lines 9-12 in code snippet). Also, further discloses “this decomposition also provides modularity and the ability to switch between various combinations of scoring functions and ranking heads.” (pg. 2974, col. 1, para. 2, lines 1-3). Pasumarthi et al. discloses “The library is flexible and highly configurable: it provides an easy-to-use API to support different scoring mechanisms, loss functions, example weights, and evaluation metrics.” (pg. 2971, col. 1, para. 4, second bullet). This suggests the ability of a training logic to use a plurality of reconfigured loss functions to train a combination of models.
With respect to claim 29:
Sundaram et al., Riesselman et al., and Pasumarthi et al. do not disclose wherein the fourth model is further configured to generate, based in part on masked representations of the 3D structural representations of the amino acids in the reference target protein sequence, the fourth set of pathogenicity scores, wherein the masked representations of the 3D structural representations mask 3D structural data about at least one amino acid in the reference target protein sequence.
However, Rives et al. discloses “We train models using the masked language modeling objective (6). Each input sequence is corrupted by replacing a fraction of the amino acids with a special mask token. The network is trained to predict the missing tokens from the corrupted sequence … For each sequence
x
, we sample a set of indices
M
to mask, replacing the true token at each index
i
with the mask token. For each masked token, we independently minimize the negative log likelihood of the true amino acid
x
i
given the masked sequence
x
/
M
as context. Intuitively, to make a prediction for a masked position, the model must identify dependencies between the masked site and the unmasked parts of the sequence.“ (pg. 2, col. 2, para. 1, lines 1-13). Also, further discloses “We adapt the Transformer protein language model to predict the quantitative effect of mutations.” (pg. 9, col. 1, para. 3, lines 7-8). Rives et al. discloses “While the model cannot observe protein structure directly, it observes patterns in the sequences of its training data that are determined by structure. In principle, the network could compress sequence variations by capturing commonality in structural elements across the data, thereby encoding structural information into the representations.” (pg. 5, col. 2, para. 1, lines 6-11). This suggests a model that generates pathogenicity scores for a set of variants based on masked representations of summary statistics, which masks at least one amino acid in the protein sequences. The 3D structural information is encoded within the masked sequence representations.
With respect to claim 30:
Sundaram et al., Riesselman et al., and Pasumarthi et al. do not disclose wherein the fourth model is further configured to generate, based in part on a masked representation of the reference target protein sequence, the fourth set of pathogenicity scores, wherein the masked representation masks at least one amino acid in the reference target protein sequence.
However, Rives et al. discloses “We train models using the masked language modeling objective (6). Each input sequence is corrupted by replacing a fraction of the amino acids with a special mask token. The network is trained to predict the missing tokens from the corrupted sequence … For each sequence
x
, we sample a set of indices
M
to mask, replacing the true token at each index
i
with the mask token. For each masked token, we independently minimize the negative log likelihood of the true amino acid
x
i
given the masked sequence
x
/
M
as context. Intuitively, to make a prediction for a masked position, the model must identify dependencies between the masked site and the unmasked parts of the sequence.“ (pg. 2, col. 2, para. 1, lines 1-13). Also, further discloses “We adapt the Transformer protein language model to predict the quantitative effect of mutations.” (pg. 9, col. 1, para. 3, lines 7-8). This suggests a model that generates pathogenicity scores for a set of variants based on masked representations of summary statistics, which masks at least one amino acid in the protein sequences.
With respect to claim 31:
Sundaram et al., Riesselman et al., and Pasumarthi et al. do not disclose wherein the fourth model is further configured to generate, based in part on masked representations of the alternate protein sequences, the fourth set of pathogenicity scores, wherein the masked representations of the alternate protein sequences mask at least one amino acid in the reference target protein sequence.
However, Rives et al. discloses “We train models using the masked language modeling objective (6). Each input sequence is corrupted by replacing a fraction of the amino acids with a special mask token. The network is trained to predict the missing tokens from the corrupted sequence … For each sequence
x
, we sample a set of indices
M
to mask, replacing the true token at each index
i
with the mask token. For each masked token, we independently minimize the negative log likelihood of the true amino acid
x
i
given the masked sequence
x
/
M
as context. Intuitively, to make a prediction for a masked position, the model must identify dependencies between the masked site and the unmasked parts of the sequence.“ (pg. 2, col. 2, para. 1, lines 1-13). Also, further discloses “We adapt the Transformer protein language model to predict the quantitative effect of mutations.” (pg. 9, col. 1, para. 3, lines 7-8). This suggests a model that generates pathogenicity scores for a set of variants based on masked representations of summary statistics, which masks at least one amino acid in the protein sequences.
Claim 34 is rejected under 35 U.S.C. 103 as being unpatentable over Sundaram et al. (Nature Genetics, 2018, 50, 1-29), as provided in the IDS filed 7/28/2023, Riesselman et al. (Nature Methods, 2018, 15, 1-27), as provided in the IDS filed 7/28/2023, and Pasumarthi et al. (Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019, 2970-2978), as provided in the IDS filed 7/28/2023, as applied to claims 1-15, 17-21, 26-28, 32-33, and 35-36 above, in view of Davo (Position weight matrix, 1 October 2013, Dave Tang's Blog, https://davetang.org/muse/2013/10/01/position-weight-matrix/, 1-8).
Sundaram et al., Riesselman et al., and Pasumarthi et al. are applied to claims 1-15, 17-21, 26-28, 32-33, and 35-36 above.
With respect to claim 34:
Riesselman et al. and Pasumarthi et al. do not disclose wherein the evolutionary profiles include position-specific frequency matrices (PSFMs).
However, Sundaram et al. discloses “Architecture of the deep residual network for pathogenicity prediction, PrimateAI. Predicted pathogenicity is on a scale from 0 (benign) to 1 (pathogenic). The network takes as input the human amino acid (AA) reference and alternate sequence (51 AAs) centered at the variant, the position weight matrix (PWM) conservation profiles calculated from 99 vertebrate species” (pg. 24, Fig. 3, lines 1-5). This describes evolutionary conservation profiles as including position weight matrices.
Davo discloses “To convert a PFM to the corresponding PWM, the frequencies are converted to normalized frequency values on a log-scale.” (pg. 2, para. 2, lines 1-2). This describes a method of converting from a position frequency matrix (PFM) to a position weight matrix (PWM).
It would have been prima facie obvious to one of ordinary skill in the art to modify the teachings disclosed by Sundaram et al., Riesselman et al., and Pasumarthi et al. to incorporate the conversion of position frequency matrices to position weight matrices disclosed by Davo. One would be motivated to use position-specific frequency matrices to calculate position weight matrices because any given sequence can be quantitatively scored against a motif model using(pg. 7, para. 1, lines 3-4). This means that amino acids can be better scored for how often they occur at each position in a reference target protein sequence using position weight matrices calculated from position frequency matrices. Therefore, the evolutionary profiles involve position-specific frequency matrices. There is a likelihood of success, since all teachings involve complex mathematics such as neural network models and probabilities for determining metrics, which are well known techniques in the field of bioinformatics.
Conclusion
No claims are allowed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jammy Luo whose telephone number is (571)272-2358. The examiner can normally be reached Monday - Friday, 9:00 AM - 5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Larry D Riggs can be reached at (571)270-3062. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.N.L./Examiner, Art Unit 1686
/LARRY D RIGGS II/Supervisory Patent Examiner, Art Unit 1686
Prosecution Insights