DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Instant application eligible to benefit from the foreign application as claimed by applicant on 11/28/2020 and effective filling date was considered as 11/28/2020.
Information Disclosure Statement
IDS has been submitted on 11/10/2023, and 9/3/2025 and 2/6/2026 considered by the examiner.
Claim Status
Claims 1-16 and 28-31 are pending and examined on the merits.
Claims 17-27 are cancelled.
Claims 1-16 and 28-31 are rejected.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-16, and 28-31 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 2A, Prong 1
In accordance with MPEP § 2106, the instant claims 1-14, are drawn to a method, claim 15 to a system and claim 16, 28-31 to a CRM and therefore are found to recite statutory subject matter (Step 1: YES). The instant claims are then analyzed to determine if the claims recite any concepts that equate to an abstract idea, law of nature or natural phenomenon (Step 2A, Prong 1). The instant claims recite the following limitations that equate to an abstract idea:
Claim 1, 15, and 16 recites:
Determining a predicted structure of a protein. (Mental process and mathematical concept) Predicting a folded structure or performing sequence analysis is classified as a mental process because it represents concepts that can practically be performed in the human mind. Because structural prediction relies on mathematical relationships, calculations, algorithms, or spatial modeling coordinates, it also triggers the mathematical concepts grouping under abstract ideas
Maintaining graph data representing a graph of the protein. The graph comprises a set of nodes and a set of edges, wherein the set of nodes comprises a plurality of amino acid nodes that each represent a respective amino acid in the protein, and wherein the set of edges comprises a respective edge connecting each pair of amino acid nodes in graph. (Mathematical concept) A graph data structure comprising nodes, edges, and connections relies on fundamental graph theory, which classifies under mathematical concepts. Representing protein data using nodes and edges inherently involves mathematical relationships and algorithmic calculations.
Receiving the pair embeddings; updating the pair embeddings in accordance with values of the update block parameters of the update block. (Mathematical concept and mental process) Updating parameters and processing embeddings heavily relies on mathematical formulas and algorithms. Receiving and updating things a human could theoretically do using a pen, paper, or human reasoning.
For each edge in the graph that connects a pair of amino acid nodes: generating a respective representation of each of a plurality of cycles in the graph that include the edge by, for each cycle, processing embeddings for edges in the cycle in accordance with the values of the update block parameters of the update block to generate the representation of the cycle. (Mental process) “Processing embeddings... in accordance with the values of the update block parameters" falls directly into the mathematical concepts grouping, which covers mathematical relationships, formulas, and calculations.
Updating the pair embedding for the edge using the representations of the cycles in the graph that include the edge (Mathematical concept and mental process)
Claim 2 recites “generating a respective representation of each of a plurality of cycles in the graph that include the edge comprises generating a respective representation of every cycle in the graph that includes the edge and that has a predefined length”. (Mathematical concept and mental process) The step involves calculations, manipulations, or enumerations of abstract mathematical structures (graph cycles, edge relationships, and predefined lengths). The action of “generating a respective representation” of logical connections can theoretically be performed entirely in the human mind.
Claim 4 recites “processing the pair embedding for the edge and the representations of the cycles in the graph that include the edge, in accordance with the values of the update block parameters of the update block, to generate a residual embedding; and adding the residual embedding to the pair embedding for the edge.” (Mathematical concept) The limitation describes applying specific parameter values, generating a “residual embedding” (a mathematical representation or vector), and adding it to another embedding falls under algorithms, calculations, and mathematical equations/formulas.
Claim 6 recites “each update block receives a multiple sequence alignment (MSA) representation for the protein that represents a respective MSA corresponding to each amino acid chain in the protein; and wherein the pair embedding for each edge is updated based at least in part on the MSA representation.” (Mathematical concept and mental process) The steps of receiving a representation and updating data structures (pair embeddings) mirror analytical tasks that can conceptually be evaluated or visualized by a human mind. Calculating or updating edge values and matrix representations relies on underlying mathematical relationships, formulas, and numerical manipulations.
Claim 8 recites “for each edge in the graph that connects a pair of amino acid nodes, generating the respective representation of each of the plurality of cycles in the graph that include the edge comprises generating a respective representation of each of one or more cycles in the graph that include an edge that connects a MSA sequence node to an amino acid node.” (Mathematical concept and mental process) Terms involving “generating representations,” “edges,” “cycles,” and “nodes” are treated as mathematical relationships, formulas, or calculations. The claim elements involve calculating and generating relationships that can be practically performed in the human mind.
Claim 11 recites “each update block further performs operations comprising: updating the MSA representation based on the pair embeddings”. (Mental process)
Claim 12 recites “updating the MSA representation based on the pair embeddings comprises: updating the MSA representation using attention over embeddings in the MSA representation, wherein the attention is conditioned on the pair embeddings”. (Mathematical concept) It covers calculations, formulas, and machine learning operations like attention mechanisms.
Claim 13 recites
Updating the MSA representation using attention over the embeddings in the MSA representation comprises: generating, based on the MSA representation, a plurality of attention weights. (Mental process and Mathematical concept)
Generating, based on the pair embeddings, a respective attention bias corresponding to each of the attention weights. (Mental process and mathematical concept)
Generating a plurality of biased attention weights based on the attention weights and the attention biases. (Mathematical concept)
Updating the embeddings in the MSA representation using attention over the embeddings in the MSA representation based on the biased attention weights. (Mathematical concept)
Claim 14 recites “updating the embeddings in the MSA representation using attention based on the biased attention weights comprises, for each embedding in the MSA representation: updating the embedding, based on the biased attention weights, using attention over only embeddings in the MSA representation that are located in a same row as the embedding in an arrangement of the embeddings in the MSA representation into a two-dimensional array”. (Mathematical concept and mental process)
Claim 28 recites “generating a respective representation of each of a plurality of cycles in the graph that include the edge comprises generating a respective representation of every cycle in the graph that includes the edge and that has a predefined length”. (Mathematical concept and mental process) Graph theory, path/cycle lengths, and structural relationships are mathematical calculations and relationships “defined as mathematical relationships, mathematical formulas or equations, and mathematical calculations. The steps describe data manipulation and analysis that “can be performed in the human mind, or by a human using a pen and paper.”
Claim 30 recites “updating the pair embedding for the edge using the representations of the cycles in the graph that include the edge comprises: processing the pair embedding for the edge and the representations of the cycles in the graph that include the edge, in accordance with the values of the update block parameters of the update block, to generate a residual embedding; and adding the residual embedding to the pair embedding for the edge”. (Mental process and mathematical concept) The steps involving processing pair embeddings, update block parameters, and generating residual embeddings are mathematical algorithms. The concept of analyzing cycles and updating path representations relies on cognitive reasoning, which examiners frequently flag under the mental process grouping.
Claim 31 recites “to generate a residual embedding comprises: summing the representations of the cycles in the graph that include the edge; and processing the pair embedding for the edge and the sum of the representations of the cycles in the graph that include the edge using one or more neural network layers to generate the residual embedding”. (Mathematical concept) The steps of “summing” representations and utilizing "one or more neural network layers" fall squarely into the abstract idea grouping of mathematical formulas, calculations, and algorithms.
Claim 3, 7, 10 and 29 has no active steps and further narrows the limitations to their corresponding base claims.
As such claims 1-16, and 28-31 recite an abstract idea (Step 2A, Prong 1: YES).
Step 2A, Prong 2
Claims found to recite a judicial exception under Step 2A, Prong 1 are then further analyzed to determine if the claims as a whole integrate the recited judicial exception into a practical application or not (Step 2A, Prong 2). Specifically, the claims recite the following additional elements:
Claim 1 recites
Obtaining a respective pair embedding for each edge in the graph that connects a pair of amino acid nodes.
Processing an input comprising the pair embeddings using an embedding neural network.
The embedding neural network comprises a sequence of update blocks and uses the update blocks to repeatedly update the pair embeddings, wherein each update block has a plurality of update block parameters.
Using the embedding neural network, determining the predicted structure of the protein based on the pair embeddings.
Claim 5 recites “to generate a residual embedding comprises: summing the representations of the cycles in the graph that include the edge; and processing the pair embedding for the edge and the sum of the representations of the cycles in the graph that include the edge using one or more neural network layers to generate the residual embedding.”
Claim 9 recites “each update block further performs operations comprising: applying a transformation operation to the MSA representation; and updating the pair embeddings by adding a result of the transformation operation to the pair embeddings.”
Claim 15 recites “one or more computers; and one or more storage devices communicatively coupled to the one or more computer”
Claim 16 recites “one or more non-transitory computer storage media storing instructions that when executed by one or more computers.”
For claim 1, obtaining or calculating vector representations is a fundamental building block of modern machine learning. Simply applying it to a specific dataset (amino acid nodes) does not impose a meaningful restriction on its use. Under USPTO guidance, merely “obtaining” data or representing pre-existing information is not practically applied. It constitutes of analyzing or predicting a sequence/structure without actually performing a practical, industrial, or medical use.
For the limitation “processing an input … neural network”, merely processing data with a neural network does not integrate an abstract idea into a practical application, a claim must do more than just apply a mathematical formula on a general-purpose computer. The added element fails to link the pair embeddings to any real-world physical or technical output.
For additional elements “the embedding neural network …. update block parameter” and “determining the … the pair embeddings. fails to integrate into practical application because it recites generic, conventional computer components performing routine mathematical operations (conventional neural network) without providing a specific technical improvement or inventive concept. Using update blocks to repeatedly update embeddings is standard machine learning practice. Neural networks and parameters are well-known tools that do not change how the computer works.
The additional element in claim 5 “to generate a residual embedding comprises: summing the …. to generate the residual embedding” This additional element lacks an inventive concept and practical application under 35 U.S.C. 101 because it merely recites generic mathematical summing and conventional neural network processing applied to abstract data manipulation, without transforming the underlying information into a specific practical output or improving computer functionality.
The additional element in claim 9 “each update block further … operation to the pair embeddings” lacks integration to a practical application because it recites generic mathematical operations (applying a transformation and adding results to pair embeddings) that amount to routine computer math rather than a practical, patent-eligible application.
The additional elements in claim 15 and 16 are simply a conventional computing system that was used for executing the series of functions or steps.
The core of the claimed invention remains the algorithmic construction of a protein structure identification analysis of the amino acids based on graph-based method. Because bioinformaticians and geneticists routinely rely on publicly available tools such as alpha-fold for protein structure determination, utilizing these tools to determine protein structure is a routine, well-understood, and conventional process in the art.
These limitations describe mere data collection (gathering) and the application of abstract mathematical and mental processes, and lack the necessary integrative steps to transform them into a practical application per MPEP 2106.
The additional elements merely recite a conventional computer processor, a storage and memory used to execute the claimed instructions. Under Alice Corp. v. CLS Bank Int'l, simply implementing or analyzing data using a generic, general-purpose computer does not transform an unpatentable abstract idea into a patent-eligible application.
Under the MPEP 2106.05(g) guidelines regarding insignificant extra-solution activity, the mere act of crunching and gathering data on a conventional computing system is well-understood, routine, and conventional in the art of bioinformatics pipelines. The recited limitations serve solely as data-gathering or analyzing activities. Because these additional elements do not reflect any specific improvement to computer functioning or physical technology, the claim fails to integrate the judicial exception into a practical application, and instead amounts to insignificant, routine post-solution activity.
There are no limitations that indicate that the protein structure identification process requires anything other than a conventional computer to execute the instructions (a series of steps). As such, these limitations equate to mere instructions to implement the abstract idea on a generic computer that the courts have stated does not render an abstract idea eligible in Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984.
The above recited additional elements do not provide a practical application of the recited judicial exception. As such, claims 1-16, and 28-31 are directed to an abstract idea (Step 2A, Prong 2: NO).
Step 2B
Claims found to be directed to a judicial exception are then further evaluated to determine if the claims recite an inventive concept that provides significantly more than the judicial exception itself (Step 2B). The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claims recite additional elements that equate to mere instructions to apply the recited exception in a generic computing environment or well-understood, routine and conventional activity.
As discussed above, there are no additional limitations to indicate that the claimed protein structure identification process requires anything other than generic computer components in order to carry out the recited abstract idea in the claims. Claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984.
The additional elements do not comprise an inventive concept when considered individually or as an ordered combination that transforms the claimed judicial exception into a patent-eligible application of the judicial exception. Therefore, the claims do not amount to significantly more than the judicial exception itself (Step 2B: No). As such, claims 1-16, and 28-31 are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 1-5, 7, 8, 15, 16, and 28-31 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (PNAS | January 21, 2020 | vol. 117 | no. 3, 1496-1503) in view of Ingraham et al. (Advances in Neural information Processing systems (NeurIPS), vol.32 (2019)) and Glimmer et al. (Proceedings of the 34th International Conference on Machine Learning (ICML), PMIR 70 (2017))
Yang et al. discloses a deep neural network system for predicting protein structure from multiple sequential alignments (MSA). The system operates on pairs of amino acid residues, generating pairwise representation encoding inter-residue distance and orientation. Specifically, Yang et al. teaches:
The protein is represented as pairwise relationships between all amino acid residues, where for each residue pair (i, j) the system computes geometric descriptors (distance, dihedral angles, and spherical coordinates) that fully characterize the relative orientation. This corresponds to a set of pairwise embeddings for edges in a complete residue graph (pg. 1497, c1 bottom; pg. 1497, c2, bottom; Figure 1) suggesting the limitation of “obtaining a respective pair embedding for each edge in the graph that connects a pair of amino acid nodes; processing an input comprising the pair embeddings”
Employs a deep residual convolutional neural network comprising 36 stacked residual blocks. Each residual block receives the current pairwise representations, processes them, and updates them by adding residual connections (pg. 1497-1498, Figure 1) suggesting the limitation of “an input comprising the pair embeddings using an embedding neural network, wherein the embedding neural network comprises a sequence of update blocks and uses the update blocks to repeatedly update the pair embeddings.”
Models every residue pair, constructing a fully connected residue graph where each node represents an amino acid residue and each edge represents the pairwise geometric relationship between two residues (pg. 1497, Figure 1) suggesting the limitation of “Maintaining graph data representing a graph of the protein, wherein the graph comprises a set of nodes… amino acid nodes… and the set of edges… connecting each pair of amino acid nodes.”
Uses the predicted pairwise geometry (pair embedding) to determine protein 3D structure. (pg. 1498, c1, middle and bottom) suggesting the limitation of “Determining the predicted structure of the protein based on the pair embedding”
Yang et al.’s convolutional residual network implicitly captures geometric relationships involving neighboring residue pairs because over a 2D pairwise contact map necessarily aggregates information from residue triplets.
However, Yang et al. does not explicitly recite cycle-based aggregation.
Ingraham et al. teaches:
Applying multi-layer graph attention/message-passing over the neighborhood graph, where each update aggregates information form multi-top paths involving edge features of adjacent edges which correspond to triangles and cycles in the graph. (pg. 3, middle; Figure 1)
Ingraham et al. Representing a protein as a graph with amino acid residues as nodes and spatial neighbors as edges. (Abstract; pg. 5, top, pg. 11, bottom)
Glimmer et al. teaches:
The molecular graph convolutions variant within the masses passing neural networks (MPNN) framework maintains edge state representations that are updated based on features of adjacent edges, e.g., the edge update which aggregates over the two endpoint nodes thereby implicitly capturing triangular relationships. (Abstract; pg. 5, c1, bottom, c2)
Therefore, combined teaching of Ingraham et al. and Glimmer et al.’s suggests the claim limitation of “generating a respective representation of each of a plurality of cycles in the graph that include the edge by, for each cycle, processing embeddings for edges in the cycle in accordance with the values of the update block parameters… and updating the pair embeddings for the edge using the representations of the cycles In the graph that include the edge.”
A PHOSITA would have been motivated to combine Yang et al.'s pairwise geometric-prediction network with Ingraham et al.'s graph-based protein representation and Gilmer et al.'s generalized edge-update/message-passing framework, with a reasonable expectation of predictable success, because each reference's own stated purpose joins directly with the others' teachings in the shared field of applying graph-structured deep learning to protein modeling.
Yang et al. describes a deep residual-convolutional network which takes an MSA as the input and outputs relative distance and orientation information for every residue pair, using a stack of residual blocks (36 in the baseline configuration) that iteratively refine those pairwise representations (pg. 1496–1497; Fig. 1B). This directly supplies the claimed “obtaining a respective pair embedding for each edge” and “embedding neural network... comprises a sequence of update blocks and uses the update blocks to repeatedly update the pair embeddings.” Because Yang et al. already teaches that pairwise embeddings must be iteratively refined to recover 3D structure, a PHOSITA would look to the graph-learning literature for known, generalizable mechanisms for performing that refinement — motivating a search for the kind of structured update mechanisms taught by Ingraham and Gilmer.
Ingraham et al. explains that its graph-conditioned model efficiently captures the complex dependencies in proteins by representing the protein as a graph of amino-acid nodes connected by edges reflecting spatial (not merely sequential) proximity (Abstract; pg. 5, top). This is the same problem Yang et al. is solving, representing inter-residue relationships to predict structure, but recast as node/edge graph data, which is the "maintaining graph data representing a graph of the protein... set of nodes... set of edges" limitation of claim 1. Because Ingraham et al. and Yang et al. address the identical technical problem (representing residue relationships for structure-related prediction) with functionally equivalent data (pairwise/edge relationships), a PHOSITA would have recognized the pairwise embeddings of Yang et al. as a straightforward, substitute for the edge features of Ingraham et al.'s graph, with predictable results because each reference already treats “edge between two residues” as the basic unit of computation.
Gilmer et al. states that its Message Passing Neural Network (MPNN) framework has incredible potential to be useful in chemistry, drug discovery, and materials science and generalizes prior edge/message-passing models so that edge states are updated using features drawn from neighboring, topologically-connected edges (pg. 1263). This is a well-known, field-standard technique for updating an edge representation using structural context beyond the single edge itself, the operation required by the claim's “generating a respective representation of each of a plurality of cycles in the graph that include the edge... and updating the pair embedding for the edge using the representations of the cycles.” Gilmer's own statement of purpose — improving predictions relevant to chemistry and drug discovery places it in the same practical field as Yang et al. and Ingraham et al. (both aimed at structure-based drug design and protein engineering), reinforcing that a PHOSITA working on protein structure prediction would naturally consult Gilmer's generalized edge-update mechanism as an off-the-shelf tool.
Each reference solves a piece of the same problem using the same basic unit of representation (a residue-pair/edge embedding) and the same basic operation (iterative, block-wise updating of that embedding). Substituting Gilmer's known cycle/neighbor-aware edge-update mechanism into Yang's residual pair-embedding update blocks, using Ingraham's graph model as the organizing data structure, is the combination of known elements according to known methods to yield no more than the predictable result of a pair embedding that better captures multi-residue (cycle-level) geometric dependencies (MPEP 2143.01) The combination was obvious since none of the three references teaches away from the others and all three operate on the identical mathematical object (a graph of amino-acid residues connected by pairwise edges)
Regarding claims 15 and 16 (System and media claims):
Claim 15 (system) and claim 16 (computer readable media) are directed to the same subject matter as claim 1, implemented respectively as a computer system and as non-transitory computer storage media. These claims are rejected under 103 for the same region as claim 1, as it is well established that patentability of method claims and their apparatus media counterparts rises and falls together when the only distinction is the claim category. (MPEP 2182, In re, Nuijten,500 F.3d 1346 (Fed. Cir. 2007). The prior art combination renders each element of claim 15 and 16 obvious for the same reason set forth for claim 1 above.
Regarding claim 2, 3, 28 and 29:
Claims 2 requires that the plurality of cycles generated are “every cycle in the graph that includes the edge and that has a predefined length
Claim 3 requires “predefined length” being “three”
Claim 28-29 recite the same limitation for the non-transitory media claim.
It was well-known in the graph neural network and protein structure prediction literature before the critical date that triangles (3-cycles) are the most fundamental higher-order substructures in graphs and that enumerating triangles-based updates over edge representations is both computationally tractable and geometrically meaningful for molecular structures. Glimmer et al. specifically discuss is triangle-based aggregation of edge features (Abstract; pg. 5, c1, bottom, c2). Protein geometry inherently involves residue triplets: the distance between residues A- B and B -C, together with A- C, form a triangle that constraints 3D structure suggesting the claim limitations of 2-3 and 28-29 above.
Moreover, within the core deep learning architecture behind AlphaFold2 architecture (Disclosed publicly by DeepMind at CASP14 in November 2020), the triangular multiplicative update explicitly aggregates over all triangles (3-cycles) involving an edge. Even setting aside the CASP14 preprint, Yang et al.’s convolution over the 2D residue distance map inherently processes all triplets: a 2D convolution over a pair wise distant matrix with a kernel operating on rows and columns simultaneously aggregates exactly the triangle (i, j,k) through the matrix entries at (i, k) and (k, j). (pg. 2, c1, bottom; pg. 7, middle)
A graph aggregation method requires selecting a cycle length to analyze. A person having ordinary skill in the art (PHOSITA) would logically select length-3 cycles (triangles) for the following reasons:
Triangles are the smallest and most fundamental non-trivial cyclic structures in any network or graph.
Length-3 cycles are the cheapest to enumerate. For a protein with N residues, the time complexity is strictly O(N^3) which means the runtime increases proportionally to the cube of the input, making it the fastest baseline computation.
Claim 28 and 29 are non-transitory media counterparts to claim 2-3, respectively and are rejected under 103 for the same reason as claimed 2-3, as set forth above.
Regarding claim 4, 5, 30 and 31:
Claims 4 requires “processing the pair embedding for the edge and the representations of the cycle… to generate a residual embedding; and adding the residual embedding to the pair embedding for the edge”
Claim 5 requires with the residual generated by “summing the representations of the cycles… and processing the pair embedding for the edge and the sum… using one or more neural network layers.”
Yang et al. explicitly employs a deep residual convolutional network in which each block generates a residual that is added back to the input representation. (pg. 1497, Figure 1) suggest the limitation of “generate a residual embedding; and adding the residual embedding to the pair embedding for the edge.”
The use of residual connections in deep network was ubiquitous before the effective filing date. (He et al., Deep Residual Learning for Image Recognition).
Glimmer’s molecular graph convolutions also employ additive updates to edge states. (pg. 3, c1, bottom, pg. 6, c1, middle) suggesting the limitation of “summing the representations of the cycles… and processing the pair embedding for the edge and the sum… using one or more neural network layers.”
Summing cycle representations before processing with a neural network layer is a straightforward and known aggregation technique (summation pooling over a set of substructures representations).
Claim 30 and 31 are non-transitory media counterparts to claim 4-5, respectively and are rejected under 103 for the same reason as claimed 4-5, as set forth above.
Regarding claim 7:
Claim 7 recites: “The set of nodes of the graph comprises a plurality of MSA sequence nodes that each represents a respective MSA sequence… and the set of edges of the graph comprises a respective edge connecting each MSA Sequence node… to each amino acid node in the graph, and…the MSA representation for the protein comprises a respective embedding for each edge… that connects a MSA sequence node to an amino acid node.”
Yang et al. represents the relationship between individual MSA sequence and residue positions as a 2D matrix (MSA rows X residue columns), which is structurally equivalent to a bipartite graph in which MSA sequence nodes are connected by edges to amino acid position nodes (pg. 1496-1497) suggesting the limitation of “a plurality of MSA sequence nodes that each represents ….. connecting each MSA Sequence node …the MSA representation for the …. node to an amino acid node.”
Each entry in the MSA matrix corresponds exactly to the claimed “embedding for each edge connecting a MSA sequence node to an amino acid node.”
Alternatively, the explicit formulation as a bipartite graph of MSA sequence nodes and amino acid nodes where edges carry the power residue alignment embeddings --is the direct structural analog of the MSA representation described in Yang et al. and was a standard design choice for the representing MSA data as graph-structured input for GNNs.
Ingraham et al. teaches that protein graphs can incorporate both sequence information and structural information as different node/edge types (pg. 2, middle).
Extending this to MSA sequences (one node per aligned sequence) connected to amino acid positions is a straightforward application of the bipartite graph construction well-known in the GNN literature (Glimmer at al.)
Regarding claim 8:
Claim 8 recites that, for each edge connecting a pair of amino acid nodes, the cycle representations include “one or more cycles in the graph that include an edge that connects a MSA sequence node to and amino acid node.”
Yang et al.’s 2D convolution over the MSA-derived pairwise contact map aggregates information along both the residue axis and the MSA-sequence axis, effectively propagating information through exactly these MSA mediated triangle. (Figure 2) suggesting the limitation of “one or more cycles in the graph that include an edge that connects a MSA sequence node to and amino acid node.” The concept of propagating pair embedding information through MSA-sequence mediated triangles was inherent in Yang et al.’ architecture.
Implementing these as cycle enumeration over MSA-sequence-node-corrected-edges in the graph of claim 7 is an obvious formalization of in Yang et al.’s implicit computations.
Claim 6 and 9-14 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. in view of Ingraham et al. and Glimmer et al. as applied to claims 1-5, 7, 8, 15,16, and 28-31 above and further in view of Senior et al. (706 | Nature | Vol 577 | 30 January 2020)
Yang et al. in view of Ingraham et al. and Glimmer et al. are applied to claims 1-5, 7, 8, 15,16, and 28-31.
Regarding claim 6:
Claim 6 recite that “each update block receives a multiple sequence alignment (MSA) representation for the protein that represents a respective MSA corresponding to each amino acid chain in the protein; and… the pair embedding for each edge is updated based at least in part on the MSA representation.”
Yang et al. in view of Ingraham et al. and Glimmer et al. does not explicitly teach “each update block receives a multiple sequence alignment (MSA) representation for the protein that represents a respective MSA corresponding to each amino acid chain in the protein”
Yang et al.’s primary input is a multiple sequence alignment representing co-evolutionary relationships among protein sequences, and the pairwise representations are derived directly from the MSA. (pg. 1496 – 1497)
Each of the 36 residual blocks in Yang et al. teaching operates on representations that were initialized from MSA-derived features, so each block’s update of pair embeddings inherently incorporates the MSA.
Senior et al. similarly uses a neural network that takes an MSA as input and generates pairwise distance potential used for structure prediction. (pg. 706 -708)
Thus, combined teaching of Yang et al. and senior et al. suggest the limitation of “each update block receives a multiple sequence alignment (MSA) ……. on the MSA representation.”
Claim 6 depends from claims 1, which is rendered obvious by the Yang/Ingraham/Gilmer combination discussed above. The only limitation remaining for claim 6 is that each update block receives an MSA representation for the protein, corresponding to each amino-acid chain, and that the pair embedding for each edge be updated based at least in part on that MSA representation. A PHOSITA would have been motivated to further combine that base combination with Senior et al., with a reasonable expectation of predictable success, for the following reasons.
Yang et al. already establishes that the pairwise/edge representation being updated by the claimed update blocks is MSA-derived in the first place. Yang et al.'s network "takes an MSA as the input" and produces the pairwise distance/orientation representations that are then iteratively refined by the 36 stacked residual (update) blocks (pg. 1496–1497). Because Yang et al.'s pair representations are already seeded from MSA-derived co-evolutionary features before being passed into the update-block stack, a PHOSITA would recognize that carrying the MSA data itself alongside the pair embeddings so that each block can draw on it directly, rather than only on a fixed initial encoding of it is a straightforward extension of what Yang et al. already does, not an inventive leap.
Senior et al. independently confirms, in a second, closely related reference, that conditioning a residue-pair/distance-prediction network on the MSA is the standard, expected input for this problem. Senior et al. explains that its network was trained to make accurate predictions of the distances between pairs of residues, with those predictions built from features constructed from the MSA via sequence-database search (pg. 706–708; Fig. 1a, “Feature extraction stages (constructing the MSA... and computing MSA-based features)”). Senior et al. thus independently corroborates — from a second, high-profile CASP13 system built on the same problem as Yang et al. — that using an MSA representation as the conditioning input to a deep residual network that predicts pairwise/distance potentials was a known, field-standard design choice as of the relevant time, not a result-oriented after-the-fact selection.
Combining these two teachings with the base Yang/Ingraham/Gilmer graph-and-cycle architecture predictably yields claim 6's limitation. Both Yang et al. and Senior et al. independently teach that (i) an MSA is the natural input from which pairwise/residue features are derived, and (ii) a stack of iterative residual/update blocks is the mechanism used to refine those pairwise features toward a structure prediction. Because the base combination already updates pair embeddings block-by-block using cycle/edge information, a PHOSITA seeking to improve the accuracy of that block-wise update consistent with what both Yang et al. and Senior et al. teach is achievable by keeping MSA information available throughout the network rather than discarding it after an initial embedding step would have been motivated to route the MSA representation into each update block and use it, together with the cycle-based pair-embedding update, to refine the pair embeddings. This is simply applying a known technique (MSA-conditioned pairwise refinement, taught by both cited references) to a known method (the cycle-aware update-block architecture of claim 1) to achieve the predictable result of a more accurate, MSA-informed pair embedding — precisely the combination is obvious (MPEP 2143.01), since neither reference teaches away and both operate on the same basic inputs (MSA and residue-pair embeddings) toward the same goal (accurate structure prediction).
Regarding claim 9 and 10:
Claims 9 recites: “applying a transformation operation to the MSA representation; and updating the pair embeddings by adding a result of the transformation operation to the pair embeddings”
Claim 10 recites “the transformation operation comprising an outer product mean operation”
The outer product of MSA vectors across the residue dimension is a standard technique for generating pairwise residue co-evolution features from an MSA. In Yang et al., the co-evolutionary signal extracted from the MSA is used to generate pairwise predictions which is mathematically equivalent to computing and outer product over the MSA representations. (Abstract; pg. 1, c1, middle)
Furthermore, Senior et al. uses pairwise distance potentials derived from outer product like transformation of MSA encoded features. (pg. 2, c1, bottom; pg. 3, c1, bottom)
Thus, combined teaching of Yang et al. and Senior et al. above suggests the “claimed limitation of 9 and 10.”
The outer product mean is a deterministic mathematical operation (average of the outer products of MSA Row vectors) that was the obvious and principled way to convert pre-sequence, per-residue embeddings into pairwise residue representations. No inventive step attaches to selecting the mean other outer product over alternative aggregations (for example, sum, max). The outer product mean is one of a very small number of standard operations for projecting sequence representations into pairs.
Regarding claim 11-14:
Claim 11 recites: “updating the MSA representation based on the pair embeddings”.
Claim 12 recites: “updating the MSA representation using attention over embeddings in the MSA representation, wherein the attention is conditioned on the pair embeddings.
Clint 13-14 adds: Generating attention weights from the MSA representation; generating “attention bias corresponding to each of the attention weights” from pair embeddings, generating biased attention weights; and updating MSA embeddings using attention over only embeddings in the MSA representation that are located in a same row” (row-wise attention).
Senior et al. discloses a bidirectional information flow between sequence (MSA) representations and pairwise distance representations where pairwise distance predictions are updated from sequence features and vice versa through iterative recycling of the network. (pg. 7O7 -708) suggesting the limitations of “updating the MSA representation based on the pair embeddings” and “updating the MSA representation using attention over embeddings in the MSA representation, wherein the attention is conditioned on the pair embeddings.”
Attention mechanisms for conditions on auxiliary context features were well established in the literature before effective filing date (Vaswani et al., “Attention is all you need,” 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.)
Conditioning the attention weights in the MSA processing stream on the pair embeddings is a straightforward extension of context conditions attention replacing the standard self-attention bias with a pair-embedding-derived bias which a PHOSITA would consider an obvious technique for propagating pair-level information into MSA level updates.
Row-wise attention: attending only over residues in the same MSA row is directly analogous to axial attention (Ho et al., AXIAL ATTENTION IN MULTIDIMENSIONAL TRANSFORMERS, 2019) a well-known technique for efficiently applying attention over 2D arrays, and is the natural decomposition for a 2D MSA matrix where rows correspond to aligned sequences. Yang et al. processes the MSA as a 2D matrix, applying row wise attention to update per- row representations is the obvious adaptation of axial attention to this structure.
Conclusion
No Claims are allowed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARSHAD KHAN whose telephone number is (571)272-9812. The examiner can normally be reached Mon-Fri-7:30-5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Larry Riggs can be reached at 5712703062. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.H.K./
Examiner, Art Unit 1686
/LARRY D RIGGS II/Supervisory Patent Examiner, Art Unit 1686
Prosecution Insights