Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending and are being examined.
Examiner's Note
The Examiner respectfully requests of the Applicant in preparing responses, to fully consider the entirety of the reference(s) as potentially teaching all or part of the claimed invention. It is noted, REFERENCES ARE RELEVANT AS PRIOR ART FOR ALL THEY CONTAIN. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain.” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). A reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art, including non-preferred embodiments (see MPEP 2123). The Examiner has cited particular locations in the reference(s) as applied to the claim(s) above for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim(s), typically other passages and figures will apply as well.
Information Disclosure Statement
The information disclosure statement (IDS) was submitted on 11/29th/2023, 2/22nd/2024, 07/16th/2025, and 12/12th/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 101
101 Rejection
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 is rejected under 35 USC § 101 because the claimed invention is directed to non-statutory subject matter
Step 1 Analysis:
Claims 1-10 are directed to a method which is directed to a process, one of the statutory categories. Claims 11-20 are directed to a non-transitory computer-readable media, which is directed to a product, one of the statutory categories.
Regarding Claim 1:
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 2A Prong 1 Analysis:
Claim 1 recites in part process steps which, under the broadest reasonable interpretation, are a series of mental processes including an observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process or a mathematical concept but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas. The claim recites in part:
generating a sequence of lexical tokens that represents a lexical text Under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator predicting one token at a time to generate meaningful text). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
generating a graph that represents the lexical text Under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator breaking down a sentence into words, listing the words as nodes, and drawing edges between nodes that have a relation). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
selecting, in the graph, a plurality of traversal paths that represents a subsequence of the sequence of lexical tokens Under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator selecting a path in the graph that represents a sequence of words). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
inferring from the subsequence of the sequence of lexical tokens, by the token sequence encoder, an encoded sequence that represents the subsequence of the sequence of lexical tokens Under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator inferring from the sequence of tokens a next token). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas. Furthermore, the recitation of by the token sequence encoder is mere instructions to implement the exception using generic computer components which does not integrate the exception into a practical application and does not amount to significantly more than the exception itself.
Step 2A Prong 2 Analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
generating a token sequence encoder that is trainable and untrained is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
adjusting the token sequence encoder based on a difference between the encoded sequence that represents the subsequence of the sequence of lexical tokens and the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Step 2B Analysis:
Claim 1 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the additional elements of:
generating a token sequence encoder that is trainable and untrained is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
adjusting the token sequence encoder based on a difference between the encoded sequence that represents the subsequence of the sequence of lexical tokens and the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
For the reasons above, claim 1 is rejected as being directed to non-patentable subject matter under §101.
The additional limitations of the dependent claims contain no additional elements that provide a practical application or amount to significantly more than the abstract idea and are addressed briefly below.
Dependent claim 2 recites:
Step 2A Prong 1:
said selecting the subsequence of the sequence of lexical tokens comprises sliding a fixed-length window over the sequence of lexical tokens that represents the lexical text under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator sliding a fixed-length window over the sequence of lexical tokens to select a subsequence of the sequence of lexical tokens). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The claim does not recite additional elements that would integrate the judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
For the reasons above, claim 2 is rejected as being directed to non-patentable subject matter under §101.
Dependent claim 3:
Step 2A Prong 1: The claim does not contain any limitations to analyze under this step.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
a first machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
a second machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Step 2B: In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
a first machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Furthermore, the courts have found limitations directed to linking data to a field of use, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II)).
a second machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Furthermore, the courts have found limitations directed to linking data to a field of use, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II)).
For the reasons above, claim 3 is rejected as being directed to non-patentable subject matter under §101.
Dependent claim 4:
Step 2A Prong 1:
Measuring ….. a first training loss that is based on the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator summing the loss for each node from root to leaf using a pen and paper). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Measuring ….. a second training loss that is not based on the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator measuring the loss for each token in a sequence of tokens using a pen and paper). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
for the first machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
For the second machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Step 2B: In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
For the first machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Furthermore, the courts have found limitations directed to linking data to a field of use, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II)).
For the second machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Furthermore, the courts have found limitations directed to linking data to a field of use, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II)).
For the reasons above, claim 4 is rejected as being directed to non-patentable subject matter under §101.
Dependent claim 5 recites:
Step 2A Prong 1:
Predicting ….. a lexical token that occurs next in the sequence of lexical tokens adjacent to the subsequence of the sequence of lexical tokens under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator predicting a next word that comes at the end of a sentence). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Inferring ….. the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator inferring the potential paths down a tree that could represent a sentence). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
by the second machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
by the first machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Step 2B: In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
by the second machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Furthermore, the courts have found limitations directed to linking data to a field of use, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II)).
by the first machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Furthermore, the courts have found limitations directed to linking data to a field of use, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II)).
For the reasons above, claim 5 is rejected as being directed to non-patentable subject matter under §101.
Dependent claim 6 recites:
Step 2A Prong 1:
Predicting ….. a lexical token that occurs next in a new sequence of lexical tokens that represents a new lexical text that is syntactically invalid under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator predicting a next invalid word that could come at the end of a sentence). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. In particular, the additional element of:
by the second machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
deploying into a production environment, without the first machine learning model, the token sequence encoder and the second machine learning model is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Step 2B: In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
by the second machine learning model is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Furthermore, the courts have found limitations directed to linking data to a field of use, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II)).
deploying into a production environment, without the first machine learning model, the token sequence encoder and the second machine learning model is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
For the reasons above, claim 6 is rejected as being directed to non-patentable subject matter under §101.
Dependent claim 7 recites:
Step 2A Prong 1:
selecting the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens comprises selecting exactly one traversal path per lexical token in the subsequence of the sequence of lexical tokens under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator selecting a path down a graph). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The claim does not recite additional elements that would integrate the judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
For the reasons above, claim 7 is rejected as being directed to non-patentable subject matter under §101.
Dependent claim 8:
Step 2A Prong 1: The claim does not contain any limitations to analyze under this step.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
multitask learning by the token sequence encoder; said inferring the encoded sequence that represents the subsequence of the sequence of lexical tokens and said adjusting the token sequence encoder occur during said multitask learning by the token sequence encoder is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Step 2B: In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
multitask learning by the token sequence encoder; said inferring the encoded sequence that represents the subsequence of the sequence of lexical tokens and said adjusting the token sequence encoder occur during said multitask learning by the token sequence encoder is recited at a high level of generality and amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use MPEP 2106.05(h). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Furthermore, the courts have found limitations directed to linking data to a field of use, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II)).
For the reasons above, claim 8 is rejected as being directed to non-patentable subject matter under §101.
Dependent claim 9 recites:
Step 2A Prong 1:
counting occurrences of a particular traversal path in the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator counting the number or paths that represent a sentence). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The claim does not recite additional elements that would integrate the judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
For the reasons above, claim 9 is rejected as being directed to non-patentable subject matter under §101.
Dependent claim 10:
Step 2A Prong 1:
the graph is at least one selected from a group consisting of a directed acyclic graph, a dataflow graph, an abstract syntax tree (AST), and an imbalanced tree under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator selecting a graph from a group of graphs). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The claim does not recite additional elements that would integrate the judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
For the reasons above, claim 10 is rejected as being directed to non-patentable subject matter under §101.
Claims 11-20 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a non-transitory compute readable media with similar steps to claims 1-10, and thus are not patent eligible for the same reasons (see above).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7-8,10-11, 17-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over PILAULT (Using Graph Algorithms to Pretrain Graph Transformers – 2023), in view of RETINRAJ (US20240160912A1), further in view of BALAKRISHNAN (US20230360557A1), further in view of FERNANDEZ (US20230366687A1).
Regarding claim 1, PILAULT teaches generating a token sequence encoder that is trainable and untrained ([Page 10, Sec. C] Our Transformer model is a single monolithic architecture that uses a masking Scheme similar to UniLM (Dong et al., 2019), allowing the model to play a variety of roles (encoder only, decoder-only, or encoder-decoder) and tackle a variety of objectives (classification, regression or sequence generation. The examiner notes that PILAULT teaches the use of a model that is based on BERT which uses encoded tokenized text as input, this model is pretrained then fine-tuned which the examiner interpret to be trainable from an initial untrained state).
selecting, in the graph, a plurality of traversal paths that represents a subsequence of the sequence of lexical tokens ([Page 3, Sec. 3.2] We propose Information Gain Paths in Algorithm 1. IP has the same initial condition (ei, r, ej) to start generating the paths and has the same learning objective as SP. IP also helps us reduce the number of paths by selecting the ones that constitute a beneficial context for answering the query based on a measure of information gain. The examiner notes that PILAULT teaches selecting the paths that constitute a beneficial context for answering a query).
However, PILAULT is not relied upon to explicitly teach:
generating a sequence of lexical tokens that represents a lexical text.
generating a graph that represents the lexical text.
inferring from the subsequence of the sequence of lexical tokens, by the token sequence encoder, an encoded sequence that represents the subsequence of the sequence of lexical tokens.
adjusting the token sequence encoder based on a difference between the encoded sequence that represents the subsequence of the sequence of lexical tokens and the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens.
On the other hand, RETINRAJ teaches generating a sequence of lexical tokens that represents a lexical text ([0004] The natural language input can be tokenized to generate a set of tokens that represents the natural language input. The examiner notes that PILAULT and RETINRAJ are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s data pre-processing to incorporate generating a sequence of lexical tokens that represents a lexical text as taught by RETINRAJ [0004] so that the tokens of the first subset of the tokens can be used to evaluate whether a first condition, such as a partial statement, is satisfied [0004]).
Furthermore, RETINRAJ teaches generating a graph that represents the lexical text ([0006] A graph representation of the natural language input can be generated by the computing device. The examiner notes that PILAULT and RETINRAJ are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s graph algorithm to incorporate generating a graph that represents the lexical text as taught by RETINRAJ [0006] in order to determine the similarity score between the graph representation and each tenant-specific graph [0054]).
Furthermore, BALAKRISHNAN teaches inferring from the subsequence of the sequence of lexical tokens, by the token sequence encoder, an encoded sequence that represents the subsequence of the sequence of lexical tokens ([0043] for each input token from a source sequence, the tag-encoded token-level transformation T(xi) is predicted. The examiner notes that PILAULT and BALAKRISHNAN are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s prediction algorithm to incorporate inferring from the subsequence of the sequence of lexical tokens, by the token sequence encoder, an encoded sequence that represents the subsequence of the sequence of lexical tokens as taught by BALAKRISHNAN [0043] to perform text correction [0043]).
Furthermore, FERNANDEZ teaches adjusting the token sequence encoder based on a difference between the encoded sequence that represents the subsequence of the sequence of lexical tokens and the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens ([0049] The predictive data analysis computing entity 106 may be configured to train a prediction model (e.g., nodewise coverage region determination machine learning model, traversal path optimization machine learning model, and/or the event prediction machine learning model). The examiner notes that FERNANDEZ teaches training (adjusting) a traversal path optimization machine learning model based on performance. The examiner further notes that PILAULT and FERNANDEZ are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s training algorithm to incorporate adjusting the token sequence encoder based on a difference between the encoded sequence that represents the subsequence of the sequence of lexical tokens and the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens as taught by FERNANDEZ [0049] to improve the efficiency and accuracy of optimal traversal path determinations for a user [0071]).
Regarding claim 7, PILAULT teaches selecting the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens comprises selecting exactly one traversal path per lexical token in the subsequence of the sequence of lexical tokens ([Page 2, Sec. 3] In this section, we provide an overview of the five different graph algorithms that we use to create pretraining tasks for our experiments: (1) Relational Shortest Path sequence generation (SP); (2) Information gain Path sequence generation (IP); (3) K-Hop Neighbor prediction (KHN); (4) Invariant Adjacency matrix classification (IVA); (5) Local Clustering Coefficient estimation (LCC). The examiner notes that PILAULT teaches selecting the shortest path to a node in a graph).
Regarding claim 8, PILAULT teaches the method of claim 1. However, PILAULT is not relied upon to explicitly teach multitask learning by the token sequence encoder; said inferring the encoded sequence that represents the subsequence of the sequence of lexical tokens and said adjusting the token sequence encoder occur during said multitask learning by the token sequence encoder. On the other hand, BALAKRISHNAN teaches multitask learning by the token sequence encoder; said inferring the encoded sequence that represents the subsequence of the sequence of lexical tokens and said adjusting the token sequence encoder occur during said multitask learning by the token sequence encoder ([0008] In an embodiment, the artificial intelligence scoring engine includes a reinforcement learning agent and maps the extracted features into the scoring rubric with dynamic weight adjustment using a deep neural network with reinforcement learning. The examiner notes that BALAKRISHNAN teaches dynamically adjusts the weights as new data is processed. The examiner further notes that PILAULT and BALAKRISHNAN are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s training algorithm to incorporate multitask learning by the token sequence encoder; said inferring the encoded sequence that represents the subsequence of the sequence of lexical tokens and said adjusting the token sequence encoder occur during said multitask learning by the token sequence encoder as taught by BALAKRISHNAN [0008] to improve scalability and efficiency [0055]).
Regarding claim 10, PILAULT teaches the method of claim 1. However, PILAULT is not relied upon to explicitly teach the graph is at least one selected from a group consisting of a directed acyclic graph, a dataflow graph, an abstract syntax tree (AST), and an imbalanced tree. On the other hand, BALAKRISHNAN teaches the graph is at least one selected from a group consisting of a directed acyclic graph, a dataflow graph, an abstract syntax tree (AST), and an imbalanced tree ([0006] In some embodiments, the attribute can be connected to, for example in a directed acyclic graph or other tree structure, the node, and in other embodiments, the node may be represented by a bucket structure that can contain the attribute and other attributes for the particular entity. The examiner further notes that PILAULT and RETINRAJ are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s graph model to incorporate the graph is at least one selected from a group consisting of a directed acyclic graph, a dataflow graph, an abstract syntax tree (AST), and an imbalanced tree as taught by RETINRAJ [0006] to improve accuracy and reduce computational resource usage [0042]).
Claims 11, 17-18, and 20 are rejected under 35 U.S.C 103 based upon the same rationale as claims 1, 7-8, and 10 respectively as they are the non-transitory computer readable media claims corresponding to the method claims.
Claims 2, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over PILAULT (Using Graph Algorithms to Pretrain Graph Transformers – 2023), in view of RETINRAJ (US20240160912A1), further in view of BALAKRISHNAN (US20230360557A1), further in view of FERNANDEZ (US20230366687A1), further in view of TSVETKOV (US20240362412A1).
Regarding claim 2, PILAULT teaches the method of claim 1. However, PILAULT is not relied upon to explicitly teach said selecting the subsequence of the sequence of lexical tokens comprises sliding a fixed-length window over the sequence of lexical tokens that represents the lexical text. On the other hand, TSVETKOV teaches said selecting the subsequence of the sequence of lexical tokens comprises sliding a fixed-length window over the sequence of lexical tokens that represents the lexical text ([0022] In other implementations, the KP engine 110 analyzes the subject text using a sliding window approach (e.g., selecting candidate phrases with a moving window of one or more tokens). The examiner notes that PILAULT and TSVETKOV are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s machine learning model to incorporate said selecting the subsequence of the sequence of lexical tokens comprises sliding a fixed-length window over the sequence of lexical tokens that represents the lexical text as taught by TSVETKOV [0022] to avoid the use of "stop words," or words within the natural language that typically do not add much information to the text [0022]).
Claim 12 is rejected under 35 U.S.C 103 based upon the same rationale as claim 2 as it is the non-transitory computer readable media claim corresponding to the method claim.
Claims 3, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over PILAULT (Using Graph Algorithms to Pretrain Graph Transformers – 2023), in view of RETINRAJ (US20240160912A1), further in view of BALAKRISHNAN (US20230360557A1), further in view of FERNANDEZ (US20230366687A1), further in view of KARISANI (Semi-Supervised Text Classification via Self-Pretraining).
Regarding claim 3, PILAULT teaches the method of claim 1. However, PILAULT is not relied upon to explicitly teach a first machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens, and a second machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens. On the other hand, KARISANI teaches a first machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens, and a second machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens ([Page 42, Sec. 3] Our algorithm is iterative and utilizes two neural networks as the underlying classifiers. Algorithm 1 illustrates Self-Pretraining in its basic form. The examiner notes that KARISAN1 teaches the use two networks that are based on BERT [Page 41, Para. 1] which uses encoded tokenized text as input. The examiner further notes that PILAULT and KARISANI are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s machine learning model to incorporate a first machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens, and a second machine learning model that accepts as input the encoded sequence that represents the subsequence of the sequence of lexical tokens as taught by KARISANI [Page 42, Sec. 3] to take advantage of Self-Pretraining which outperforms the state of the art in multiple settings [Page 40, last Para.]).
Claim 13 is rejected under 35 U.S.C 103 based upon the same rationale as claim 3 as it is the non-transitory computer readable media claim corresponding to the method claim.
Claims 4, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over PILAULT (Using Graph Algorithms to Pretrain Graph Transformers – 2023), in view of RETINRAJ (US20240160912A1), further in view of BALAKRISHNAN (US20230360557A1), further in view of FERNANDEZ (US20230366687A1), further in view of KARISANI (Semi-Supervised Text Classification via Self-Pretraining), further in view of XIA (Model-Level Dual Learning - 2018).
Regarding claim 4, PILAULT teaches the method of claim 3. However, PILAULT is not relied upon to explicitly teach measuring, for the first machine learning model, a first training loss that is based on the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens, and measuring, for the second machine learning model, a second training loss that is not based on the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens. On the other hand, XIA teaches measuring, for the first machine learning model, a first training loss that is based on the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens, and measuring, for the second machine learning model, a second training loss that is not based on the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens ([Page 4, Sec. 2.3] We bring the duality into model structures of the two tasks and do not change the loss functions. The examiner notes that XIA teaches the use two models with two distinct structures, tasks, and loss functions. The examiner further notes that PILAULT and XIA are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s machine learning model to incorporate measuring, for the first machine learning model, a first training loss that is based on the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens, and measuring, for the second machine learning model, a second training loss that is not based on the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens as taught by XIA [Page 4, Sec. 2.3] to achieve better generation [Page 4, Sec. 2.3]).
Claim 14 is rejected under 35 U.S.C 103 based upon the same rationale as claim 4 as it is the non-transitory computer readable media claim corresponding to the method claim.
Claims 5, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over PILAULT (Using Graph Algorithms to Pretrain Graph Transformers – 2023), in view of RETINRAJ (US20240160912A1), further in view of BALAKRISHNAN (US20230360557A1), further in view of FERNANDEZ (US20230366687A1), further in view of KARISANI (Semi-Supervised Text Classification via Self-Pretraining), further in view of LAPRISE (US20240419907A1).
Regarding claim 5, PILAULT teaches the method of claim 3. However, PILAULT is not relied upon to explicitly teach:
predicting, by the second machine learning model, a lexical token that occurs next in the sequence of lexical tokens adjacent to the subsequence of the sequence of lexical tokens.
inferring, by the first machine learning model, the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens.
On the other hand, LAPRISE teaches predicting, by the second machine learning model, a lexical token that occurs next in the sequence of lexical tokens adjacent to the subsequence of the sequence of lexical tokens ([0046] For example, training an LLM to predict the next token in the text string (corresponding to a next object in an object graph, for example) can help the LLM to learn to establish correct relationships between objects in the environment. The examiner notes that PILAULT and LAPRISE are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s machine learning model to incorporate predicting, by the second machine learning model, a lexical token that occurs next in the sequence of lexical tokens adjacent to the subsequence of the sequence of lexical tokens as taught by LAPRISE [0046] to help the model to learn to establish correct relationships between objects in the environment [0046]).
Furthermore, FERNANDEZ teaches inferring, by the first machine learning model, the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens ([0104] The predictive data analysis computing entity 106 may then select a candidate traversal path with the next largest path score as the optimal traversal path and perform one or more prediction based actions based at least in part on the optimal traversal path. The examiner notes that PILAULT and FERNANDEZ are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s machine learning model to incorporate inferring, by the first machine learning model, the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens as taught by FERNANDEZ [0104] to provide an optimal traversal path [0104]).
Claim 15 is rejected under 35 U.S.C 103 based upon the same rationale as claim 5 as it is the non-transitory computer readable media claim corresponding to the method claim.
Claims 6, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over PILAULT (Using Graph Algorithms to Pretrain Graph Transformers – 2023), in view of RETINRAJ (US20240160912A1), further in view of BALAKRISHNAN (US20230360557A1), further in view of FERNANDEZ (US20230366687A1), further in view of KARISANI (Semi-Supervised Text Classification via Self-Pretraining), further in view of LAPRISE (US20240419907A1), further in view of DONG (US20240193445A1).
Regarding claim 6, PILAULT teaches the method of claim 5. However, PILAULT is not relied upon to explicitly teach deploying into a production environment, without the first machine learning model, the token sequence encoder and the second machine learning model; predicting, by the second machine learning model, a lexical token that occurs next in a new sequence of lexical tokens that represents a new lexical text that is syntactically invalid. On the other hand, DONG teaches deploying into a production environment, without the first machine learning model, the token sequence encoder and the second machine learning model; predicting, by the second machine learning model, a lexical token that occurs next in a new sequence of lexical tokens that represents a new lexical text that is syntactically invalid ([0026] However, in other examples, the system(s) may deploy only a portion of the machine learning model(s), such as the base model(s) and one or more of the domain specific parts. The examiner notes that DONG teaches deploying a portion of the ML models that are used to predict a next token in a sequence [0060]. The examiner further notes that PILAULT and DONG are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s machine learning model to incorporate deploying into a production environment, without the first machine learning model, the token sequence encoder and the second machine learning model; predicting, by the second machine learning model, a lexical token that occurs next in a new sequence of lexical tokens that represents a new lexical text that is syntactically invalid as taught by DONG [0026] to save computing resources. [0026]).
Claim 16 is rejected under 35 U.S.C 103 based upon the same rationale as claim 6 as it is the non-transitory computer readable media claim corresponding to the method claim.
Claims 9, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over PILAULT (Using Graph Algorithms to Pretrain Graph Transformers – 2023), in view of RETINRAJ (US20240160912A1), further in view of BALAKRISHNAN (US20230360557A1), further in view of FERNANDEZ (US20230366687A1), further in view of JIANG (TreeBERT A Tree-Based Pre-Trained Model for Programming - 2021).
Regarding claim 9, PILAULT teaches the method of claim 1. However, PILAULT is not relied upon to explicitly teach counting occurrences of a particular traversal path in the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens. On the other hand, JIANG teaches counting occurrences of a particular traversal path in the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens ([Page 56, Sec. 3.2] We use the set of paths from the root node to the terminal nodes to represent the AST, A = (p1; p2; .. ; pN), where N is the number of paths contained in the AST. The examiner further notes that PILAULT and JIANG are both directed to machine learning and are thus considered to be reasonably analogous. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified PILAULT’s graph model to incorporate counting occurrences of a particular traversal path in the plurality of traversal paths that represents the subsequence of the sequence of lexical tokens as taught by JIANG [Page 56, Sec. 3.2] to uniquely represent a tree compared to other traversal methods, such as pre-order traversal [Page 55, Sec. Motivation]).
Claim 19 is rejected under 35 U.S.C 103 based upon the same rationale as claim 9 as it is the non-transitory computer readable media claim corresponding to the method claim.
Conclusion
The following reference have been determined to be related to the application, but were not applied in any specific rejection. They are nonetheless listed below for reference.
SINGH (US 2023/0350657 Al)
“SINGH teaches a method for translating source code using sparse-self attention”
ALON (US 2015/0331416 Al)
“ALON teaches a method for general path-based representation for learning from programs”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAMCY ALGHAZZY whose telephone number is (571)272-8824. The examiner can normally be reached Monday-Friday between 9AM and 6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, OMAR FERNANDEZ RIVAS can be reached on (571)272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAMCY ALGHAZZY/Examiner, Art Unit 2128
/KYLE R STORK/Primary Examiner, Art Unit 2128
Prosecution Insights