DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to the claimed invention filed on May 17, 2024, in which claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement filed on January 06, 2025 complies with the provisions of 37 CFR 1.97, 1.98 and MPEP § 609. It has been placed in the application file. The information referred to therein has been considered as to the merits.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 1, 19 and 20 recite “each neural network block in the sequence is configured to perform operations”. It is unclear as to which operations the applicant is referred to.
Claims 1, 19 and 20 recite “generating a network output for the time step from the data tokens, the latent tokens, or both.” It is not clear as to what the applicant meant. How a network output for the time step is generated from the data tokens, the latent tokens, or both. It is not clear as to which “both” the applicant is referred to. And what happens to the generated network output for the time step from the data tokens, the latent tokens.
Claims 2-18 are rejected for incorporating the deficiency of their respective base claims by dependency.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1, Statutory Category:
Claims 1-18 are directed to a method
Claim 19 is directed to a computer system.
Claim 20 is directed to one or more non-transitory computer storage media.
Therefore, claims 1-20 fall into at least one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter.
Step 2A, Prong One (Judicial exception recited)
The limitation “generating, from at least the network input for the time step, a plurality of groups of data tokens” in claims 1, 19 and 20, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement, but for the recitation of generic computer components. One can mentally or manually with the aid of pen and paper generate a plurality of groups of data tokens.
The limitation “initializing a plurality of sets of latent tokens for the time step, each set corresponding to a respective one of the plurality of groups” in claims 1, 19 and 20, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement, but for the recitation of generic computer components. One can mentally or manually with the aid of pen and paper initialize a plurality of sets of latent tokens for the time step.
The limitation “generating a network output for the time step from the data tokens, the latent tokens, or both” in claims 1, 19 and 20, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement, but for the recitation of generic computer components. One can mentally or manually with the aid of pen and paper generate a network output for the time step from the data tokens, the latent tokens, or both.
Step 2A, Prong Two (Integrated into a practical application):
This judicial exception is not integrated into a practical application. In particular, the claims recite the following additional elements:
That the method is "implemented by a computing system” is a high-level recitation of a generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
The limitation “obtaining a network input for the time step, wherein the network input comprises a plurality of data tokens” amounts to data-gathering steps which is considered to be insignificant extra-solution activity, (See MPEP 2106.05(g)).
The limitation “processing the data tokens in each group and the plurality of sets of latent tokens through each neural network block in a sequence of neural network blocks to update the plurality of groups of data tokens and the plurality of sets of latent tokens; for each group of data tokens, processing the data tokens in the group using a first neural network to update the data tokens in the group; processing the data tokens in the group and the latent tokens in the corresponding set using a second neural network to update the latent tokens in the corresponding set; processing the plurality of sets of latent tokens using a third neural network to update the latent tokens; and ” recites insignificant extra-solution activity such as mere outputting of the result. The mere outputting of data does not meaningfully limit the abstract idea. Viewing the additional limitations together and the claim as a whole, nothing provides integration into a practical application. (See MPEP 2106.05 (g)).
The limitation “one or more computers, one or more storage devices and one or more non-transitory computer storage media” are recited at a high level of generality such that they amount to on more than mere instructions to apply the exception using a generic component. (see MPEP 2106.05(f)). These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer (see MPEP 2106.05(h)). Note, the mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application.
Step 2B (claim provides an inventive concept):
The conclusions for the mere implementation using a computer are carried over and does not provide significantly more.
With respect to the "obtaining step" identified as insignificant extra-solution activity above when re-evaluated this element is well-understood, routine, and conventional as evidenced by the court cases in MPEP 2106.05(d)(II), "i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network);" and thus remains insignificant extra-solution activity that does not provide significantly more.
With respect to the "processing steps" identified as insignificant extra-solution activity above when re-evaluated this element is well-understood, routine, and conventional in displaying information as evidenced by the court cases in MPEP 2106.05(d)(II), " iv. Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93" and "i. … transmitting data over a network, …Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)".
With respect to the “one or more computers, one or more storage devices and one or more non-transitory computer storage media” amount to elements that have been recognized as well-understood, routine, and conventional activity in particular fields, as demonstrate by: Relevant court decision: the followings are examples of court decisions demonstrating well-understood, routine and conventional activities, see e.g., MPEP 2106.05(d)(II) and MPEP 2106.05(f)(2): Computer readable storage media comprising instructions to implement a method, e.g., see Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea.
Looking at the claim as a whole does not change this conclusion and the claim appears to be ineligible.
Accordingly, claim 1 is directed to an abstract idea. The remaining independent claims 19 and 20 fall short the 35 USC 101 requirement under the same rationale.
The dependent claims 2-18 when analyzed and each taken as a whole are held to be patent ineligible under 35 USC 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea.
Claim 2 recites “prior to processing, for each group of data tokens, the data tokens in the group using the first neural network to update the data tokens in the group, for each group of data tokens, processing the data tokens in the group and the latent tokens in the corresponding set using a fourth neural network to update the data tokens in the group”. This additional element is recited at a high level of generality and would function in its ordinary capacity for processing the data tokens in the group and the latent tokens in the corresponding set using a fourth neural network to update the data tokens in the group, this additional element does not integrate the integrate the judicial exception into a practical application and does not amount to significantly more.
Claim 3 recites “wherein generating a network output for the time step from the data tokens, the latent tokens, or both comprises using a readout neural network to generate a network output for the time step”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 4 recites “wherein for each group of data tokens in the plurality of groups of data tokens, the group of data tokens includes a larger number of tokens than the corresponding set of latent tokens”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 5 recites “wherein a number of tokens in the group of data tokens is dependent on a size of the network input and a number of tokens in the set of latent tokens is fixed and independent of the size of the network input”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 6 recites “wherein initializing a plurality of sets of latent tokens for the time step comprises, for each set of latent tokens: initializing at least a subset of the latent tokens to be equal to a set of learned latent embedding vectors”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 7 recites “at the first iteration of the plurality of iterations, the network input comprises a noisy version of a target output; the network input for each time step is a current version of the target output as of the time step; the network output for the time step defines an estimate of the target output given the current version of the target output as of the time step; and
the method further comprises, at each time step: updating the current version of the target output using the network output for the time step”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 8 recites “at a final time step of the plurality of time steps and after updating the current version of the target output using the network output for the final time step, providing, as a final estimate of the target output, the updated current version of the target output”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 9 recites “wherein the second neural network is configured to apply attention over the latent tokens in the corresponding set and the data tokens of the group with queries derived from the latent tokens and keys and values derived from the data tokens”. This additional element is recited at a high level of generality and would function in its ordinary capacity for applying attention over the latent tokens in the corresponding set and the data tokens of the group with queries derived from the latent tokens and keys and values derived from the data tokens, this additional element does not integrate the integrate the judicial exception into a practical application and does not amount to significantly more.
Claim 10 recites “wherein the fourth neural network is configured to apply attention over the latent tokens in the corresponding set and the data tokens of the group with keys and values derived from the latent tokens and queries derived from the data tokens”. This additional element is recited at a high level of generality and would function in its ordinary capacity for applying attention over the latent tokens in the corresponding set and the data tokens of the group with keys and values derived from the latent tokens and queries derived from the data tokens, this additional element does not integrate the integrate the judicial exception into a practical application and does not amount to significantly more.
Claim 11 recites “wherein the third neural network is configured to apply attention over the plurality of sets of latent tokens with keys and queries and values derived from the plurality of sets of latent tokens”. This additional element is recited at a high level of generality and would function in its ordinary capacity for applying attention over the plurality of sets of latent tokens with keys and queries and values derived from the plurality of sets of latent tokens, this additional element does not integrate the integrate the judicial exception into a practical application and does not amount to significantly more.
Claim 12 recites “wherein the first neural network is configured to apply attention over the data tokens of the group with keys and queries and values derived from the data tokens”. This additional element is recited at a high level of generality and would function in its ordinary capacity for applying attention over the data tokens of the group with keys and queries and values derived from the data tokens, this additional element does not integrate the integrate the judicial exception into a practical application and does not amount to significantly more.
Claim 13 recites “wherein the method further comprises generating the network input from an original network input for the time step, and the original network input comprises a collection of data elements”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 14 recites “generating a respective data token from each of a plurality of subgroups of the collection of data elements”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 15 recites “applying one or more learned projection layers to the data elements in the subgroup to generate the respective data token for the subgroup”. This additional element is recited at a high level of generality and would function in its ordinary capacity for applying one or more learned projection layers to the data elements in the subgroup to generate the respective data token for the subgroup, this additional element does not integrate the integrate the judicial exception into a practical application and does not amount to significantly more.
Claim 16 recites “grouping the respective data tokens into a plurality of groups of data tokens”. This additional element is recited at a high level of generality and would function in its ordinary capacity for grouping the respective data tokens into a plurality of groups of data tokens, this additional element does not integrate the integrate the judicial exception into a practical application and does not amount to significantly more.
Claim 17 recites “wherein each data token comprises an embedding of each subgroup of data elements”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Claim 18 recites “wherein each data token further comprises a positional encoding”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20240256835 (involved in generating multiple parameter blocks by partitioning multiple parameters along the row dimension and the column dimension according to a number of multiple hardware accelerators. A ratio of multiple parameters is determined relative to multiple parameters. An output is calculated for the fully connected layer using either row sharding or column sharding from the ratio. A subset of the multiple parameters blocks is loaded into a particular hardware accelerator. A corresponding input vector is received at the particular accelerator for each parameter block in the subset of parameter blocks. A loss of the output is determined corresponding to a ground truth output of the input. An update is determined to the multiple of parameters of the fully connection layer based on the loss.)
US 20230090941 (involved in generating a first group of tokens from a first frame of a video stream and a second group of tokens from a second frame of the video stream. The first group of tokens associated with to-be-reused features is identified from the first frame and the second group of tokens associated with to-be-computed features from the second frame based on a comparison of tokens from the first group of tokens to corresponding tokens in the second group of tokens. A feature output for portions of the second frame is generated corresponding to the second group of tokens. Features associated with the first group of tokens are combined with the generated feature output for portions of the second frame corresponding to the second group of tokens into a representation of the second frame of the video stream.)
US 20220253672 (involved in obtaining a network input to a neural network (NN) and a set of query embeddings that collectively define a prediction task to be performed by the NN by processing the network input. The network input is processed using the NN to generate a network output that comprises a respective dimension corresponding to each query embedding in the set of query embeddings. The network input is processed using an encoder block of the NN to generate a representation of the network input as a set of latent embeddings. The set of latent and query embeddings are processed using a cross-attention block that generates each dimension of the network output by cross-attention of a corresponding query embedding over the set of latent embeddings.)
US 20220222435 (involved in accessing first sets of tokens associated with a desired task and multiple modalities associated with a context of the desired task. A second set of tokens is determined for each of the modalities using a classifier network associated with the modality. Multiple embedding vectors are generated by mapping the first sets of tokens and the second set of tokens associated with each of the modalities to an embedding space. A sequence of words addressing a desired task is produced by processing multiple embedding vectors with an encoder-decoder network, where the desired task includes a caption describing an event, an answer to a given question related to the event, a question relative to the event, or a context-aware dialog.)
US 20220164626 (involved in obtaining a merge conflict tuple, represents the merge conflict tuple as input sequences, and predicts a resolution strategy to merge changes in programs into a code base using a neural transformer system. A neural encoder transformer model receives a select one of sequences, aggregates output from each model to generate an output probability distribution of classes. A predicted resolution strategy provides the highest probability.)
US 20210312923 (involved in receiving an audio input. The processor that generates a text transcription of the audio input at a sequence-to-sequence speech recognition model to assign a respective set of external-model text tokens to a set of frames in the input. The token has an external model alignment that is provided within the input. A set of hidden states is generated based on the input by the processor. A latency between an output alignment and the alignment is provided below a predetermined latency threshold for each token. The transcription is provided with multiple tokens to output an application program, user interface, and a file storage location.)
US 20210133535 (involved in obtaining a full encoder sequence, inputting the full encoder sequence into a transformer model comprising a parameter sharing decoder pair, where the parameter sharing decoder pair has a first decoder comprising N layers. Each layer has a first masked multi-head attention block and a first feed forward network. Parameters of the first decoder are shared across all N layers of the first decoder. A second decoder has N layers, and each layer has a second masked multi-head attention block and a second feed forward network, where parameters of the second decoder are shared across all N layers of the second decoder.)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEAN M CORRIELUS whose telephone number is (571)272-4032. The examiner can normally be reached Monday-Friday 6:30a-10p(Midflex).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J Lo can be reached at (571)272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEAN M CORRIELUS/Primary Examiner, Art Unit 2159 Sunday, July 26, 2026
Prosecution Insights