DETAILED ACTION
This office action is responsive to the above identified application filed 11/27/2023. The application contains claims 1-20, all examined and rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The Information Disclosure Statement with references submitted 3/14/2024, has been considered and entered into the file.
Claim Objections
Applicant is advised that should claim 13 be found allowable, claim 18 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Applicant is advised that should claim 14 be found allowable, claim 19 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Applicant is advised that should claim 15 be found allowable, claim 20 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. Claim 1 is rejected under 35 USC 101 because the claimed inventions are directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
While independent claims 1, 12 and 17 are each directed to a statutory category, it recites a series of steps which appears to be directed to an abstract idea (mental process).
Claims 1-20 are rejected under 35 U.S.C. § 101 because the instant application is directed to non-patentable subject matter. Specifically, the claims are directed toward at least one judicial exception without reciting additional elements that amount to significantly more than the judicial exception. The rationale for this determination is in accordance with the guidelines of USPTO, applies to all statutory categories, and is explained in detail below.
When considering subject matter eligibility under 35 U.S.C. 101, (1) it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter. If the claim does fall within one of the statutory categories, (2a) it must then be determined whether the claim is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea), and if so (2b), it must additionally be determined whether the claim is a patent-eligible application of the exception. If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim amounts to significantly more than the abstract idea itself. Examples of abstract ideas include certain methods of organizing human activities; a mental processes; and mathematical concepts, (2019 PEG)
STEP 1.
Per Step 1, the claims are determined to include process, machine, and manufacture as in independent Claim 1, 12, and 17, and in the therefrom dependent claims. Therefore, the claims are directed to a statutory eligibility category.
At step 2A, prong 1, The invention is directed to what is akin to Mental Process (see Alice), As such, the claims include an abstract idea. When considering the limitations individually and as a whole the limitations directed to the abstract idea are:
“generating an initial sequence based on the specified structure by the structure encoder; and optimizing the protein sequence through an iterative process, wherein the iterative process comprises progressively refining the protein sequence” (Mental process, observation, evaluation and judgment)
The claim recites additional elements as
“using machine learning models” (merely indicates a field of use or technological environment in which the judicial exception is performed and fails to add an inventive concept to the claims. See MPEP 2106.05(h));
“ configuring a machine learning model by implanting a structural adapter into a sequence decoder” (merely indicates a field of use or technological environment in which the judicial exception is performed and fails to add an inventive concept to the claims. See MPEP 2106.05(h));
“wherein the machine learning model is configured to generate a protein sequence from a specified structure, wherein the machine learning model is endowed with protein structural awareness by the structural adapter, wherein the machine learning model is equipped with protein sequential evolutionary knowledge by the sequence decoder, and wherein the machine learning model comprises the structural adapter, the sequence decoder, and a structure encoder” (limitations are directed to generally linking the use of a judicial exception to a particular technological environment or type or source of data or field of use MPEP 2106.05(h));
“wherein the iterative process comprises progressively refining the protein sequence by iterative decoding, and wherein the structural adapter non-linearly imposes representations of the specified structure on a sequence predicted in the iterative process” (data description, which is directed to generally linking the use of a judicial exception to a particular technological environment or type or source of data or field of use MPEP 2106.05(h)).
This judicial exception is not integrated into a practical application. The elements are recited at a high level of generality, i.e. a generic computing system performing generic functions including generic processing of data. Accordingly the additional elements do not integrate the abstract into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore the claims are directed to an abstract idea. (2019 Revised Patent Subject Matter Eligibility Guidance ("2019 PEG"). Thus, under Step 2A of the Mayo framework, the Examiner holds that the claims are directed to concepts identified as abstract.
STEP 2B.
Because the claims include one or more abstract ideas, the examiner now proceeds to Step 2B of the analysis, in which the examiner considers if the claims include individually or as an ordered combination limitations that are "significantly more" than the abstract idea itself. This includes analysis as to whether there is an improvement to either the "computer itself," "another technology," the "technical field," or significantly more than what is "well-understood, routine, or conventional" (WURC) in the related arts.
The instant application includes in Claim 1 additional steps to those deemed to be abstract idea(s).
When taken the steps individually, these steps are:
“using machine learning models” (merely indicates a field of use or technological environment in which the judicial exception is performed and fails to add an inventive concept to the claims. See MPEP 2106.05(h));
“ configuring a machine learning model by implanting a structural adapter into a sequence decoder” (merely indicates a field of use or technological environment in which the judicial exception is performed and fails to add an inventive concept to the claims. See MPEP 2106.05(h));
“wherein the machine learning model is configured to generate a protein sequence from a specified structure, wherein the machine learning model is endowed with protein structural awareness by the structural adapter, wherein the machine learning model is equipped with protein sequential evolutionary knowledge by the sequence decoder, and wherein the machine learning model comprises the structural adapter, the sequence decoder, and a structure encoder” (limitations are directed to generally linking the use of a judicial exception to a particular technological environment or type or source of data or field of use MPEP 2106.05(h));
“wherein the iterative process comprises progressively refining the protein sequence by iterative decoding, and wherein the structural adapter non-linearly imposes representations of the specified structure on a sequence predicted in the iterative process” (data description, which is directed to generally linking the use of a judicial exception to a particular technological environment or type or source of data or field of use MPEP 2106.05(h)).
In the instant case, Claim 1 is directed to above mentioned abstract idea. Technical functions such as receiving, and extracting are common and basic functions in computer technology. The individual limitations are recited at a high level and do not provide any specific technology or techniques to perform the functions claimed.
In addition, when the claims are taken as a whole, as an ordered combination, the combination of steps does not add "significantly more" by virtue of considering the steps as a whole, as an ordered combination. The instant application, therefore, still appears only to implement the abstract idea to the particular technological environments using what is well-understood, routine, and conventional in the related arts. The steps are still a combination made to the abstract idea. The additional steps only add to those abstract ideas using well understood and conventional functions, and the claims do not show improved ways of, for example, an unconventional non-routine functions for analyzing model operations or updating the model that could then be pointed to as being "significantly more" than the abstract ideas themselves.
Moreover, Examiner was not able to identify any "unconventional" steps, which, when considered in the ordered combination with the other steps, could have transformed the nature of the abstract idea previously identified. The instant application, therefore, still appears to only implement the abstract ideas to the particular technological environments using what is well-understood, routine, and conventional (WURC) in the related arts.
Further, note that the limitations, in the instant claims, are done by the generically
recited computing devices. The limitations are merely instructions to implement the abstract idea on a computing device that is recited in an abstract level and require no more than a generic computing devices to perform generic functions.
Claim 12 recites a system comprising “at least one processor”, and “at least one memory communicatively coupled to the at least one processor” configured to perform the same method as set forth in claim 1, the added element of “at least one processor” and “at least one memory communicatively coupled to the at least one processor” do not transform the judicial exception into a practical application because they are tantamount to a mere instruction to apply the judicial exception to a generic computer. The additional elements are also not sufficient to amount to significantly more than the judicial exception because the action of implementing the method on a general purpose computer with at least one processor and at least one memory is tantamount to a mere instruction to apply the judicial exception to a computer.
Claim 12 is therefore rejected according to the same findings and rationale as provided above.
Claim 17 recites a “A non-transitory computer-readable storage medium, storing computer-readable instructions that upon execution by a processor cause the processor to implement operations” configured to perform the same method as set forth in claim 1, the added element of “A non-transitory computer-readable storage medium, storing computer-readable instructions that upon execution by a processor cause the processor to implement operations” do not transform the judicial exception into a practical application because they are tantamount to a mere instruction to apply the judicial exception to a generic computer. The additional elements are also not sufficient to amount to significantly more than the judicial exception because the action of implementing the method on a general purpose computer with at least one processor and at least one memory is tantamount to a mere instruction to apply the judicial exception to a computer.
Claim 17 is therefore rejected according to the same findings and rationale as provided above.
Independent claims 12 and 17 are the same analogy and rejected using similar analysis as claim 1.
CONCLUSION
It is therefore determined that the instant application not only represents an abstract idea identified as such based on criteria defined by the Courts and on USPTO examination guidelines, but also lacks the capability to bring about "Improvements to another technology or technical field" (Alice), bring about "Improvements to the functioning of the computer itself" (Alice), "Apply the judicial exception with, or by use of, a particular machine" (Bilski), "Effect a transformation or reduction of a particular article to a different state or thing" (Diehr), "Add a specific limitation other than what is well-understood, routine and conventional in the field" (Mayo), "Add unconventional steps that confine the claim to a particular useful application" (Mayo), or contain "Other meaningful limitations beyond generally linking the use of the judicial exception to a particular technological environment" (Alice), transformed a traditionally subjective process performed by humans into a mathematically automated process executed on computers (McRO), or limitations directed to improvements in computer related technology, including claims directed to software (Enfish).
The dependent claims, when considered individually and as a whole, likewise do not provide "significantly more" than the abstract idea for similar reasons as the independent claim.
claims 2 disclose “generating a functionally valid sequence for structurally non-deterministic regions (Mental process) by the machine learning model (merely indicates a field of use or technological environment in which the judicial exception is performed and fails to add an inventive concept to the claims. See MPEP 2106.05(h)), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 3 disclose “wherein the machine learning model is enabled to handle the structurally non-deterministic regions based on the protein sequential evolutionary knowledge, and wherein the machine learning model is structurally sensitive to determine nuanced sequential specificity of protein groups with structural similarity” (data description, which is directed to generally linking the use of a judicial exception to a particular technological environment or type or source of data or field of use MPEP 2106.05(h)), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 4 disclose “synthesizing diverse and structurally valid sequences (mental process) by the machine learning model” merely indicates a field of use or technological environment in which the judicial exception is performed and fails to add an inventive concept to the claims. See MPEP 2106.05(h)) , It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 5 disclose “generating antibody sequences or de novo protein sequences by the machine learning model” (mental process), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 6 disclose “wherein the machine learning model is modularizable, wherein the sequence decoder and the structure encoder have been pretrained, and wherein only the structural adapter is trained during a training process of the machine learning model” (data description, which is directed to generally linking the use of a judicial exception to a particular technological environment or type or source of data or field of use MPEP 2106.05(h)), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 7 disclose “wherein the structural adapter comprises a multi-head attention and a bottleneck feedforward network (FFN), and wherein the structural adapter is configured to acquire protein geometric information from the structure encoder” (data description, which is directed to generally linking the use of a judicial exception to a particular technological environment or type or source of data or field of use MPEP 2106.05(h)), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea.
The dependent claims which impose additional limitations also fail to claim patent eligible subject matter because the limitations cannot be considered statutory. The dependent claim(s) have been examined individually and in combination with the preceding claims, however they do not cure the deficiencies of claim 1 ; where all claims are directed to the same abstract idea, "addressing each claim of the asserted patents [is] unnecessary." Content Extraction &. Transmission LLC v, Wells Fargo Bank, Natl Ass'n, 776 F.3d 1343, 1348 (Fed. Cir. 2014). If applicant believes the dependent claims are directed towards patent eligible subject matter, they are invited to point out the specific limitations in the claim that are directed towards patent eligible subject matter. Claims for the other statutory classes are similarly analyzed.
For at least these reasons, the claimed inventions of each of dependent claims 2-11, 13-16,, and 18-120 ,are directed or indirect to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more and are rejected under 35 USC 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1,4-12, 14-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over “Learning inverse folding from millions of predicted structures“ published 2022 [hereinafter D1] in view of [US 2025/0131209 A1, hereinafter D2] in view of “MASKED INVERSE FOLDING WITH SEQUENCE TRANSFER FOR PROTEIN REPRESENTATION LEARNING“ published 3/2023 [hereinafter D3] in view of [US 2024/0355413 A1, hereinafter D4].
With regard to Claim 1,
D1 teach a method for generating protein sequences using machine learning models, comprising:
configuring a machine learning model (Abstract, “predicting a protein sequence from its backbone atom coordinates. … a sequence-to-sequence transformer with invariant geometric input processing layers”),
wherein the machine learning model is configured to generate a protein sequence from a specified structure (D1, P. 2, “recovering the native sequence of a protein
from the coordinates of its backbone atoms“, “the model is tasked with recovering the native sequence of a protein from the coordinates of its backbone atoms”, P. 2, 2, “p(Y |X), where for a protein of length n, given a sequence X of spatial coordinates (x1, . . . , xi, . . . , x3n) for each of the backbone atoms N, Cα, C in the structure, the objective is to predict Y the native sequence (y1, . . . , yi, . . . , yn) of amino acids. This density is modeled autoregressively through a sequence-to-sequence encoder-decoder”),
wherein the machine learning model is endowed with protein structural awareness (D1, P. 3, “We use GVP-GNN encoder layers to extract geometric features, followed by a generic autoregressive encoder-decoder Transformer (Vaswani et al., 2017). In GVP-GNN, the input features are translation-invariant and each layer is rotation-equivariant. We perform a change of basis on the vector features from GVP-GNN into local reference frames defined for each amino acid to derive rotation invariant features (Appendix A.3). In ablation studies increasing the number of GVP-GNN encoder layers improves the overall model performance (Figure C.1), indicating that the geometric reasoning capability in GVP-GNN is complementary to the Transformer layers”), sequence decoder (D1, P. 16, A.3, “The encoder only receives the structural features. The decoder receives the encoder output”, “The last decoder layer produces a 20-way scalar output per position and softmax activation to predict the probabilities for the amino acid identity at the next position in the sequence” P. 3, 2.2, “a hybrid model consisting of a GVP-GNN structural encoder followed by a generic transformer (GVP-Transformer)”);
wherein the machine learning model comprises the sequence decoder and a structure encoder (D1, P. 16, A.3, “The encoder only receives the structural features. The decoder receives the encoder output”, “The last decoder layer produces a 20-way scalar output per position and softmax activation to predict the probabilities for the amino acid identity at the next position in the sequence” P. 3, 2.2, “a hybrid model consisting of a GVP-GNN structural encoder followed by a generic transformer (GVP-Transformer)”);
generating an initial sequence based on the specified structure by the structure encoder (D1, P. 16, A.3, “The encoder only receives the structural features. The decoder receives the encoder output”, “The last decoder layer produces a 20-way scalar output per position and softmax activation to predict the probabilities for the amino acid identity at the next position in the sequence”); and
imposes representations of the specified structure on a sequence predicted (D1, P. 2, “recovering the native sequence of a protein from the coordinates of its backbone atoms“, “the model is tasked with recovering the native sequence of a protein from the coordinates of its backbone atoms”, P. 2, 2, “p(Y |X), where for a protein of length n, given a sequence X of spatial coordinates (x1, . . . , xi, . . . , x3n) for each of the backbone atoms N, Cα, C in the structure, the objective is to predict Y the native sequence (y1, . . . , yi, . . . , yn) of amino acids. This density is modeled autoregressively through a sequence-to-sequence encoder-decoder”, EQ(1)).
D1 does not explicitly teach implanting a structural adapter i
D2 teach configuring a machine learning model by implanting a structural adapter into a sequence decoder (D2, Fig. 3, 306, ¶35, “trainable domain adapter layers 242 a, 242 b are incorporated into the encoder 202 and/or the decoder 204”, ¶38, ¶40, “ decoder DA 242b is disposed (e.g., inserted or injected) in the decoder 204 downstream of the decoder feedforward layer 232, e.g., downstream of the decoder MR adapter layer 240 b, and upstream of the decoder output 212”, ¶¶41-42, “domain adapters 242 a, 242 b (referred to as encoder DA and decoder DA, respectively) are additional adapter layers in the encoder 202 and the decoder 204, respectively, that are trained for a domain adaptation task”, ¶48, “encoder and decoder MR adapter layers 240 a, 240 b and the DA layers 242 a, 242 b shown in FIG. 2 may not initially be injected in the encoder 202 and the decoder 204. However, if the NMT architecture 200 has previously been adapted for noise and/or domain adaptation, the MR layers 240 a, 240 b (with previous noise adapters such as noise adapters 260, 262, 264) and/or the DA layers 242 a, 242 b (for previously known domains) may have been integrated in the encoder 202 and the decoder 204”),
by the structural adapter (D2, Fig. 3, 306, ¶35, “trainable domain adapter layers 242 a, 242 b are incorporated into the encoder 202 and/or the decoder 204”, ¶38, “ MR adapter layers 240 a, 240 b (referred to as encoder and decoder MR adapter layers, respectively) each include one or more (as shown in the example NMT architecture 200, three) individual noise adapter layers (noise adapters)”, ¶44),
wherein the machine learning model comprises the structural adapter, the sequence decoder and a structure encoder (D2, Fig. 3, 306, ¶35, “trainable domain adapter layers 242 a, 242 b are incorporated into the encoder 202 and/or the decoder 204”, ¶40, “ decoder DA 242b is disposed (e.g., inserted or injected) in the decoder 204 downstream of the decoder feedforward layer 232, e.g., downstream of the decoder MR adapter layer 240 b, and upstream of the decoder output 212”, ¶¶41-42, “domain adapters 242 a, 242 b (referred to as encoder DA and decoder DA, respectively) are additional adapter layers in the encoder 202 and the decoder 204, respectively, that are trained for a domain adaptation task”),
optimizing through an iterative process, wherein the iterative process comprises progressively refining by iterative decoding (“¶80, “domain adapter (DA) layers (corresponding to domain adapter layers 242 a, 242 b) were fine-tuned on the clean in-domain dataset to create a domain-adapted NMT model … The DA was fine-tuned for 3k steps with validation every 200 steps”, ¶81, “three types of noise adapters (NAs) were fine-tuned with their respective noisy datasets”, ¶82, “During training, only the multimodal fusion layer was fine-tuned with a merge of the noisy multimodal datasets while keeping the rest frozen”, ¶94, “FF and MF also converged faster in terms of training steps than AF, providing good results after only a few hundred steps of tuning”),
and wherein the structural adapter non-linearly (D2, “a down projection layer 282 from an initial embedding size to a bottleneck dimension, e.g., with an activation 284 such as ReLU, followed by an up projection layer 286 to the initial embedding size, followed by a residual connection 290”) in the iterative process (“¶80, “domain adapter (DA) layers (corresponding to domain adapter layers 242 a, 242 b) were fine-tuned on the clean in-domain dataset to create a domain-adapted NMT model … The DA was fine-tuned for 3k steps with validation every 200 steps”, ¶81, “three types of noise adapters (NAs) were fine-tuned with their respective noisy datasets”, ¶82, “During training, only the multimodal fusion layer was fine-tuned with a merge of the noisy multimodal datasets while keeping the rest frozen”, ¶94, “FF and MF also converged faster in terms of training steps than AF, providing good results after only a few hundred steps of tuning”).
D1 and D2 are analogous art to the claimed invention because they are from a similar field of endeavor of adapting a pretrained machine model to a domain. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1 resulting in resolutions as disclosed by D2 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1 as described above to provide the advantage of using adapters that may be composable, transferable, and/or trainable. Example adapters include trained domain adapters and modality-specific adapters, which can be separately learnt during training for addressing diverse and/or noisy input coming from various sources (D2, ¶25). This is simply combining prior art elements according to known methods to yield predictable results, use of known technique to improve similar devices (methods, or products) in the same way, and applying a known technique to a known device (method, or product) ready for improvement to yield predictable results (MPEP 2143).
D1-D2 does not explicitly teach wherein the machine learning model is equipped with protein sequential evolutionary knowledge
D3 teach a method for generating protein sequences using machine learning models (Abstract, “reconstruct a protein’s amino-acid sequence given its structure”), comprising:
the machine learning model is equipped with protein sequential evolutionary knowledge (D3, P. 4, “we transfer sequence information from CARP-640M by directly replacing the sequence embedding in Equation 5 with the outputs from CARP-640M pretrained on UniRef50”, P. 3, “MIF having 20 times fewer parameters and being trained on only the 19 thousand examples in CATH compared to the 42 million sequences in UniRef50”), and
imposes representations of the specified structure on a sequence predicted in iterative process (D3, P. 2, “a model learns to reconstruct the original protein sequence from a corrupted version”, P. 15, “reconstruct the original amino acids conditioned on the corrupted sequence and the backbone structure”, Eq(2)).
D1-D2 and D3 are analogous art to the claimed invention because they are from a similar field of endeavor of structure based protein sequence design. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D3 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to benefit from knowledge transfer as sequence transfer improves perplexity and recovery as sequence transfer is necessary for model performance improvement (D3, P. 4, 2.1).
D1-D2-D3 does not explicitly teach optimizing the protein sequence through an iterative process, wherein the iterative process comprises progressively refining the protein sequence by iterative decoding. This is simply combining prior art elements according to known methods to yield predictable results, use of known technique to improve similar devices (methods, or products) in the same way, and applying a known technique to a known device (method, or product) ready for improvement to yield predictable results (MPEP 2143).
D4 tech optimizing the protein sequence through an iterative process, wherein the iterative process comprises progressively refining the protein sequence by iterative decoding (Claim 76, “determining the one or more velocity fields and updating the values of the feature vectors in an iterative fashion”, ¶274, “current values of the position and/or orientation components can then be updated according to the current velocity field, and the process repeated in an iterative fashion, until a final iteration (e.g., time step) is reached”, ¶284, “generate velocity fields for iteratively updating values of these, sequence, feature vectors, updating probabilities of various types of amino acids at sites across a polypeptide chain, until they, e.g., coalesce into final values”, ¶311, “determining a final sequence representation x1 comprises repeatedly generating velocity field predictions using a machine learning model and updating a sequence representation, e.g., from xt to xt+Δt”).
D1-D2-D3 and D4 are analogous art to the claimed invention because they are from a similar field of endeavor of generative design of custom biologics. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2-D3 resulting in resolutions as disclosed by D4 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2-D3 as described above to increase accuracy. This is simply combining prior art elements according to known methods to yield predictable results, use of known technique to improve similar devices (methods, or products) in the same way, and applying a known technique to a known device (method, or product) ready for improvement to yield predictable results (MPEP 2143).
With regard to Claim 4,
D1-D2-D3-D4 teach the method of claim 1, further comprising: synthesizing diverse and structurally valid sequences by the machine learning model (D1, P. 2, 2, “We can design sequences by sampling, or by finding sequences that maximize the conditional probability given the desired structure”, P. 1, Abstract, “The model generalizes to a variety of more complex tasks including design of protein complexes, partially masked structures, binding interfaces, and multiple states”, P. 5, 3.1, “predicted sequences are sampled with low temperature T = 1e−6 from the model. While the model is calibrated (Figure C.5), a lower temperature results in sequences with higher likelihoods (and hence typically higher sequence recovery) and lower diversity”, Table c.8 ,” Table C.8. 60 randomly sampled RBD dual-state sequence designs”, D5, ¶754, “Diversity. This metric evaluates the range of structural variations produced by the design process, emphasizing the capability to generate a wide array of distinct structures”, ¶734, “Various metrics have been devised to evaluate the quality of structures and models that generate them, in accordance with these goals”, ¶735, “Generated structures were evaluated using a designability criterion, which assesses whether a generated scaffold can be assigned a viable protein sequence”). The same motivation to combine for claim 1 equally applies for current claim.
With regard to Claim 5,
D1-D2-D3-D4 teach the method of claim 1, further comprising: generating antibody sequences or de novo protein sequences by the machine learning model (D1, “Designing novel amino acid sequences that encode proteins with desired properties, known as de novo protein design,”, P.7, 3.2, “performed deep mutational scans across a set of de novo designed mini proteins with 10 different folds”, D5, ¶408, “design de novo CDR regions of an antibody“). The same motivation to combine for claim 1 equally applies for current claim.
With regard to Claim 6,
D1-D2-D3-D4 teach the method of claim 1, wherein the machine learning model is modularizable (D2, ¶25, “ adapters may be composable, transferable, and/or trainable”, ¶48, “encoder and decoder MR adapter layers 240 a, 240 b and the DA layers 242 a, 242 b shown in FIG. 2 may not initially be injected in the encoder 202 and the decoder 204. However, if the NMT architecture 200 has previously been adapted for noise and/or domain adaptation, the MR layers 240 a, 240 b (with previous noise adapters such as noise adapters 260, 262, 264) and/or the DA layers 242 a, 242 b (for previously known domains) may have been integrated in the encoder 202 and the decoder 204”), wherein the sequence decoder and the structure encoder have been pretrained (D3, “The pretrained CARP-640M weights were not finetuned during training on CATH4.2”, D2, Fig. 2, 302, claim 1, “the pretrained NMT model comprises an encoder including an encoder attention mechanism and encoder feed forward layer and a decoder”), and wherein only the structural adapter is trained during a training process of the machine learning model (D2, Fig. 3, 308, ¶54, “the parameters of the pretrained NMT model are frozen. The parameters of the domain adapter layers 242 a, 242 b may be initialized in any suitable manner, and the parameters of the remaining pretrained NMT model to be adapted may be kept frozen”, claim 7, “pretrained NMT model with the injected domain adapter on a machine translation task while the pretrained NMT model is frozen”, ¶55, “containing the domain adapter layers 242 a, 242 b (but preferably with the MR layers 240 a, 240 b removed) are frozen”, ¶56, “ model 200 are frozen alongside the other parameters of the NMT model, D3, “The pretrained CARP-640M weights were not finetuned during training on CATH4.2”). The same motivation to combine for claim 1 equally applies for current claim.
With regard to Claim 7,
D1-D2-D3-D4 teach the method of claim 1, wherein the structural adapter comprises a multi-head attention and a bottleneck feedforward network (FFN) (D2, Fig. 2, ¶44, “The example adapter layer structure includes a layer normalization layer 280, a down projection layer 282 from an initial embedding size to a bottleneck dimension, e.g., with an activation 284 such as ReLU, followed by an up projection layer 286 to the initial embedding size, followed by a residual connection 290”, ¶67, “multimodal fusion layer 270 incorporating multi fusion, the attention mechanism may be embodied in a multi-head attention mechanism, e.g., as provided in the transformer disclosed in the '978 Patent and in Vaswani et al., Attention is all you need”), and wherein the structural adapter is configured to acquire protein geometric information from the structure encoder (D1, P. 3, 2.2, “We use GVP-GNN encoder layers to extract geometric features, followed by a generic autoregressive encoder-decoder Transformer … the geometric reasoning capability in GVP-GNN is complementary to the Transformer layers”, D2, ¶39, “output of the encoder feedforward layer 222 can be dynamically routed to one of the individual noise adapters 260, 262, 264 associated with that noise source in the encoder MR adapter layer 240“, ¶44, “ASR noise adapter 260 in the MR layer 240 a (or the OCR noise adapter 262 or the UGC noise adapter 264) receives an input from the encoder feedforward layer 222”). The same motivation to combine for claim 1 equally applies for current claim.
With regard to Claim 8,
D1-D2-D3-D4 teach the method of claim 1, wherein the iterative process further comprises: sampling the predicted sequence via greedy deterministic decoding (D1, P. 5, 3.1., “To maximize sequence recovery, the predicted sequences are sampled with low temperature T = 1e−6 from the model. While the model is calibrated (Figure C.5), a lower temperature results in sequences with higher likelihoods (and hence typically higher sequence recovery) and lower diversity. Empirically at temperature as low as 1e−6 the sampling is almost deterministic”, P.2, 2, “We can design sequences by sampling, or by finding sequences that maximize the conditional probability given the desired structure”, D5, ¶63, ¶78, ¶284, “identification of a particular, single, amino acid type via, for example, an argmax function”, ¶309). The same motivation to combine for claim 1 equally applies for current claim.
With regard to Claim 9,
D1-D2-D3-D4 teach the method of claim 1, wherein the machine learning model is trained to reconstruct a protein native sequence from its corrupted version (D1, “the model is tasked with recovering the native sequence of a protein from the coordinates of its backbone atoms”, D3, P. 2, 2.1, “a model learns to reconstruct the original protein sequence from a corrupted version”, “pretraining task of reconstructing a corrupted protein sequence conditioned on its backbone structure Masked Inverse Folding”, P. 15, “We use the BERT corruption scheme and train the model to reconstruct the original amino acids conditioned on the corrupted sequence and the backbone structure”), which enables the machine learning model to iteratively refine the predicted sequence (D1, P. 2, 2, “This density is modeled autoregressively through a sequence-to-sequence encoder-decoder”, Eq. (1), D2, “¶80, “domain adapter (DA) layers (corresponding to domain adapter layers 242 a, 242 b) were fine-tuned on the clean in-domain dataset to create a domain-adapted NMT model … The DA was fine-tuned for 3k steps with validation every 200 steps”, ¶81, “three types of noise adapters (NAs) were fine-tuned with their respective noisy datasets”, ¶82, “During training, only the multimodal fusion layer was fine-tuned with a merge of the noisy multimodal datasets while keeping the rest frozen”, ¶94, “FF and MF also converged faster in terms of training steps than AF, providing good results after only a few hundred steps of tuning”, D3, P. 2, “a model learns to reconstruct the original protein sequence from a corrupted version”, P. 15, “reconstruct the original amino acids conditioned on the corrupted sequence and the backbone structure”, Eq(2), D5, ¶708, “during generation, positions and orientations for local frame representations of masked nodes were pushed forward according to flow matching methods described herein, from t=0 to t=1, while given/known nodes were held fixed at their input values”, ¶284, “generate velocity fields for iteratively updating values of these, sequence, feature vectors, updating probabilities of various types of amino acids at sites across a polypeptide chain, until they, e.g., coalesce into final values”, ¶311, “determining a final sequence representation x1 comprises repeatedly generating velocity field predictions using a machine learning model and updating a sequence representation, e.g., from xt to xt+Δt”). The same motivation to combine for claim 1 equally applies for current claim.
With regard to Claim 10,
D1-D2-D3-D4 teach the method of claim 1, wherein the sequence decoder comprises a pretrained protein language model, and wherein the pretrained protein language model has learned the protein sequential evolutionary knowledge from protein sequence data (D1,P. 2, 2, “p(Y |X), where for a protein of length n, given a sequence X of spatial coordinates (x1, . . . , xi, . . . , x3n) for each of the backbone atoms N, Cα, C in the structure, the objective is to predict Y the native sequence (y1, . . . , yi, . . . , yn) of amino acids. This density is modeled autoregressively through a sequence-to-sequence encoder-decoder”, P. 3, “We use GVP-GNN encoder layers to extract geometric features, followed by a generic autoregressive encoder-decoder Transformer”, D2, claim 28, “a pretrained decoder including an attention mechanism and a feed forward layer, the decoder receiving the encoder output and generating a decoder output corresponding to translated text”, ¶47, “ providing a pretrained text-to-text (e.g., NMT) model, such as an NMT model that is trained for a machine translation task“, D3, Abstract, “using the outputs from a pretrained sequence-only protein masked language model as input to the inverse folding model further improves pretraining perplexity”, P. 4, “we transfer sequence information from CARP-640M by directly replacing the sequence embedding in Equation 5 with the outputs from CARP-640M pretrained on UniRef50”, P. 4, “we transfer sequence information from CARP-640M by directly replacing the sequence embedding in Equation 5 with the outputs from CARP-640M pretrained on UniRef50”, P. 3, “MIF having 20 times fewer parameters and being trained on only the 19 thousand examples in CATH compared to the 42 million sequences in UniRef50”, P. 2, 2.1, “CARP-640M, a dilated convolutional protein masked language model with approximately 640 million parameters trained on UniRef50”). The same motivation to combine for claim 1 equally applies for current claim.
With regard to Claim 11,
D1-D2-D3-D4 teach the method of claim 1, wherein the structure encoder is pretrained (D2, claim 24, “the pretrained encoder and the pretrained decoder are pretrained on out-of-domain data”), and wherein the pretrained structure encoder is kept frozen during a training process of the machine learning model (D2, ¶23, “a pretrained encoder including an attention mechanism and a feed forward layer, the encoder receiving the noisy input and generating an encoder output”, claim 5, “the pretrained NMT model is pretrained on out-of-domain data”, claim 24, “the domain adapter is trained while the pretrained encoder and the pretrained decoder are frozen; and wherein each of the individual noise adapters are trained while the pretrained encoder, the pretrained decoder, and the domain adapter are frozen“, claim 7, “training the pretrained NMT model with the injected domain adapter on a machine translation task while the pretrained NMT model is frozen”, ¶54, “the parameters of the remaining pretrained NMT model to be adapted may be kept frozen”, ¶58, “to update the parameters of the noise adapter while the remaining parameters of the NMT architecture remain frozen”, ¶44, “(e.g., updating parameters of an adapter to be trained while freezing other parameters of the NMT architecture 200”, D3, “The pretrained CARP-640M weights were not finetuned during training on CATH4.2”). The same motivation to combine for claim 1 equally applies for current claim.
With regard to Claim 12,
Claim 12 is similar in scope to claim 1; therefore it is rejected under similar rational. Further D1-D2-D3-D4 teach a system for generating protein sequences using machine learning models, comprising: at least one processor; and at least one memory communicatively coupled to the at least one processor and comprising computer-readable instructions that upon execution by the at least one processor cause the at least one processor to perform operations (D1, P.1, “Research. 3New York University. Code and weights available at https://github.com/facebookresearch/esm.”, D2, claim 32, D3, P. 9, “Model code is available at https://github.com/microsoft/protein-sequence-models. Pretrained model weights and AlphaFold predictions used in this work are available at https://zenodo.org/record/6573779#.Y3ufU-zMITs”, D4, claim 102, ¶5).
With regard to Claim 14,
Claim 14 is similar in scope to claim 4; therefore it is rejected under similar rational.
With regard to Claim 15,
Claim 15 is similar in scope to claim 5; therefore it is rejected under similar rational.
With regard to Claim 16,
Claim 16 is similar in scope to claim 6; therefore it is rejected under similar rational.
With regard to Claim 17,
Claim 17 is similar in scope to claim 1; therefore it is rejected under similar rational. Further D1-D2-D3-D4 teach A non-transitory computer-readable storage medium, storing computer-readable instructions that upon execution by a processor cause the processor to implement operations (D1, P.1, “Research. 3New York University. Code and weights available at https://github.com/facebookresearch/esm.”, D2, claim 32, D3, P. 9, “Model code is available at https://github.com/microsoft/protein-sequence-models. Pretrained model weights and AlphaFold predictions used in this work are available at https://zenodo.org/record/6573779#.Y3ufU-zMITs”, D4, claim 102, ¶5).
With regard to Claim 19,
Claim 19 is similar in scope to claim 4; therefore it is rejected under similar rational.
With regard to Claim 20,
Claim 20 is similar in scope to claim 5; therefore it is rejected under similar rational.
Claims 2-3, 13, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over “Learning inverse folding from millions of predicted structures“ published 2022 [hereinafter D1] in view of [US 2025/0131209 A1, hereinafter D2] in view of “MASKED INVERSE FOLDING WITH SEQUENCE TRANSFER FOR PROTEIN REPRESENTATION LEARNING“ published 3/2023 [hereinafter D3] in view of [US 2024/0355413 A1, hereinafter D4] in view of “Reprogramming Pretrained Language Models for Antibody Sequence Infilling” published 6/2023 [hereinafter D5].
With regard to Claim 2,
D1-D2-D3-D4 teach the method of claim 1. The same motivation to combine for claim 1 equally applies for current claim.
D1-D2-D3-D4 does not explicitly teach generating a functionally valid sequence for structurally non-deterministic regions by the machine learning model.
D5 teach generating a functionally valid sequence for structurally non-deterministic regions by the machine learning model (Table 9, Table 10, Fig. 1, P. 1, Introduction, “CDR-H3 loop shows substantial variability in sequence and structure, and hence cannot be described by a canonical structure model. When compared to other protein loop structures, the CDR-H3 stands out with its significantly higher structural diversity.”, “It is composed of six hypervariable loops”, P. 4, “CDR-H3 is the longest and most diverse and therefore represents the most challenging prediction task.”, Abstract, “up to more than a two-fold increase of diversity over the baselines, without losing structural integrity and naturalness”, P. 9, 5, “generated sequences demonstrate enhanced antigen binding specificity and virus neutralization ability”, P. 16, F. “Only 2 of them (5.7%) show red flag … the generated antibody sequences by ReprogBert do not pose any significant developability concern”).
D1-D2-D3-D4 and D5 are analogous art to the claimed invention because they are from a similar field of endeavor of protein sequence design. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2-D3-D4 resulting in resolutions as disclosed by D5 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2-D3-D4 as described above to repurposes pretrained models on a source language to adapt to the tasks that are in a different language and have scarce data – where it may be difficult to train a high performing model from scratch or effectively finetune an existing pre-trained model on the specific task and to provide a model that is able on low-resourced antibody sequence dataset to provide highly diverse CDR sequences (D5,Abstract). This is simply combining prior art elements according to known methods to yield predictable results, use of known technique to improve similar devices (methods, or products) in the same way, and applying a known technique to a known device (method, or product) ready for improvement to yield predictable results (MPEP 2143).
With regard to Claim 3,
D1-D2-D3-D4-D5 teach the method of claim 2, wherein the machine learning model is enabled to handle the structurally non-deterministic regions based on the protein sequential evolutionary knowledge (D3, P. 4, “we transfer sequence information from CARP-640M by directly replacing the sequence embedding in Equation 5 with the outputs from CARP-640M pretrained on UniRef50”, P. 3, “MIF having 20 times fewer parameters and being trained on only the 19 thousand examples in CATH compared to the 42 million sequences in UniRef50”, D5, Table 9, Table 10, Fig. 1, P. 1, Introduction, “CDR-H3 loop shows substantial variability in sequence and structure, and hence cannot be described by a canonical structure model. When compared to other protein loop structures, the CDR-H3 stands out with its significantly higher structural diversity.”, “It is composed of six hypervariable loops”, P. 4, “CDR-H3 is the longest and most diverse and therefore represents the most challenging prediction task.”, Abstract, “up to more than a two-fold increase of diversity over the baselines, without losing structural integrity and naturalness”, P. 9, 5, “generated sequences demonstrate enhanced antigen binding specificity and virus neutralization ability”, P. 16, F. “Only 2 of them (5.7%) show red flag … the generated antibody sequences by ReprogBert do not pose any significant developability concern”), and wherein the machine learning model is structurally sensitive to determine nuanced sequential specificity of protein groups with structural similarity (D1, P. 5, Fig. 5, “Increased sequence recovery for buried residues suggests the model learns dense hydrophobic packing constraints in the core”, P. 7, “On locally flexible residues, multi-state design results in lower sequence perplexity than single-state design (Figure 7).”P. 5, 3.1, “When we break down performance on core residues and surface residues, as expected, core residues are more constrained and have a high native sequence recovery rate of 72%, while surface residues are not as constrained and have a lower sequence recovery of 39%”, model’s residue prediction change with local structure context (sensitive to structural difference within protein), P. 6, Fig. 7, P. 17, Fig. B.1., Fig. B. 2, “54% of the CATH topology split test set has at least one match in the train set with TM-score above 0.5, and 27% of the topology split test set has at least one match in the train set with TM-score above 0.6”. Table c.4.). The same motivation to combine for claim 2 equally applies for current claim.
With regard to Claim 13,
Claim 13 is similar in scope to claim 2; therefore it is rejected under similar rational.
With regard to Claim 18,
Claim 18 is similar in scope to claim 2; therefore it is rejected under similar rational.
Conclusion
The prior art made of record and not relied upon is considered pertinent to the applicant’s disclosure.
US Patent No. 12655176 issued to Kim et al. that disclose “generating a protein sequence using a trained neural network, and more particularly, for filling a partially filled sequence by modeling the sequence as a graph network and performing a graph convolution within the neural network” See at least Col. 1, lines 14-18
Examiner has pointed out particular references contained in the prior arts of record in the body of this action for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and Figures may apply as well. It is respectfully requested from the applicant, in preparing the response, to consider fully the entire references as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior arts or disclosed by the examiner. It is noted that any citation to specific pages, columns, figures, or lines in the prior art references any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331-33, 216 USPQ 1038-39 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMED ABOU EL SEOUD whose telephone number is (303)297-4285. The examiner can normally be reached Monday-Thursday 9:00am-6:00pm MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMED ABOU EL SEOUD/Primary Examiner, Art Unit 2148
Prosecution Insights