DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/10/2026 has been entered.
Response to Arguments
1. Regarding the claim objections, Applicant has amended claims 21 and 22 to address the minor informalities. Accordingly, the objections have been withdrawn.
2. Regarding the rejection under 35 U.S.C. § 101, Applicant's arguments filed 03/10/2026 have been fully considered but they are not persuasive.
Applicant argues on pgs. 7-10 that the claims are eligible under 35 U.S.C. § 101.
Step 2A Prong 1:
Applicant first argues that the claims are not directed to abstract ideas and that amended claims 1 and 15 recite specific technical operations that cannot be performed practically in the human mind as a mental process (see pgs. 8-9 of Remarks). The Examiner respectfully disagrees with this argument. The claims as currently written recite abstract ideas under Step 2A Prong 1 (both mental processes and mathematical concepts), which both fall under the category of abstract idea.
A person can encode a pair of input sentences into a written representation to generate encoded samples using pen and paper. A person can further make a length of each embedding shorter or longer, can write down a mix of the two embeddings as an interpolated embedding (e.g. half of a first embedding and half of a second embedding for the sentence pair) and can then write down an original sentence corresponding to a mix of the two written representations, all using pen and paper.
Furthermore, the limitations which recite the location-based attention (e.g. calculating positional weights, calculating softmax function output, summing output values to obtain a weighted sum) as well as the causal attention amount to mathematical concepts.
Therefore, the claims as written recite abstract ideas.
Step 2A Prong 2:
Applicant further argues that the claims integrate any alleged abstract ideas into a practical application under Step 2A Prong 2 analysis. Specifically, Applicant argues that the claims reflect a technical improvement in the form of a specific technical solution of determining target length and adjusting encoded sample length using location-based attention for length normalization and is eligible for analogous reasons to USPTO SME Example 48 (see pg. 10 of Remarks). Applicant further argues that the claims do not recite generate computer components and instead recites a specific technical pipeline imposing meaningful limits on the claims (see also pg. 10 of Remarks). The Examiner respectfully disagrees with these arguments. The additional limitations present in claims 1 and 15 do not integrate the judicial exception into a practical application. The only additional limitations present in amended claims 1 and 15 are generic computer components. Specifically, “a memory”, “a processor”, “an encoder and an artificial intelligence model”, “a decoder configured with a causal attention” are recited, which are each recited at a high level of generality and amount to merely instructing the mental process and mathematical calculations be performed using computer components. The claims do not contain any detail as to the structure/machine learning architecture of any of the encoder, decoder, or artificial intelligence model, nor do the claims contain any limitations which demonstrate how the various components are interconnected within a specific technical system to accomplish a practical application. Furthermore, the claimed method stops after obtaining the original sentences from the decoder, with no additional limitations describing how these sentences are actually used for a practical application. Further claim limitations describing 1) more of the structure of the data augmentation system components themselves (e.g. machine learning architectures/components of the encoder/decoder/artificial intelligence model) and how they interconnect in a specific manner and/or 2) further limitations describing how the obtained sentences are used (e.g. reciting how these newly generated sentences are subsequently used as an augmented dataset for training a model), using language supported by Applicant’s specification, would help to make the claims eligible under Step 2A Prong 2.
Hence Applicant’s arguments are not persuasive.
3. Regarding the rejections under 35 U.S.C. § 102 and § 103, Applicant has amended independent claims 1 and 15 to overcome the prior art. Accordingly, the rejections are withdrawn.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
4. Claim 14 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 14 was amended to recite “and train, using the reconstructed sentence as training data, a classification model of a classifier”. This feature was not described in the specification in such a way as to reasonably convey to a person of ordinary skill in the art that the inventor(s) had possession of the claimed invention at the time the application was filed. The specification describes a classifier (para. 0025, 0060, 0115) and describes that “the existing classifier 160 learned with target data receives the newly generated sentence (interpolated hidden vector) from the generation part 130 as input, performs inference, and calculates softmax probabilities for each label to be used in the classification task. The label with the highest softmax probability is used as the correct label for the newly generated sentence” (para. 0115). In other words, the specification describes a classifier that was learned/trained using the target data (the base data on which data augmentation is being performed (para. 0114)) to perform inference on the reconstructed sentence, not a classification model of a classifier learned/trained using the reconstructed sentence. The specification does not describe a step to “train, using the reconstructed sentence as training data, a classification model of a classifier”. This limitation is therefore new matter and rejected for inadequate written description.
Claim Rejections - 35 USC § 101
5. Claims 1-7, 9-16, and 19-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, “An apparatus” is recited, which is directed to one of the four statutory categories of invention (machine) (Step 1: YES). However, the claims limitations, under their broadest reasonable interpretation, recite mental processes or mathematical concepts which fall into the category of abstract idea (Step 2A Prong 1: YES).
The following limitations, under their broadest reasonable interpretation, recite mental processes:
encode… a pair of input sentences of a plurality of input sentences: a person can write down an encoding for a plurality of input sentences using pen and paper.
output, based on the pair of encoded input sentences, encoded samples: a person can write down encoded samples corresponding to the encoded input sentences
adjust a length of each of the encoded samples to match a target length: a person can adjust a length of the encodings (i.e. make longer or shorter) to match a target length
mix the encoded samples having the adjusted length at a predetermined mixing ratio to generate an interpolated hidden vector of a newly generated sentence: a person can mix the samples at a predetermined ratio to generate a vector, using pen and paper.
reconstruct …each original sentence corresponding to the pair of input sentences in proportion to the predetermined mixing ratio based on the interpolated hidden vector: a person can map the hidden vector back to textual form to reconstruct an original sentence according to a mixing ratio (e.g. 50/50), using pen and paper
adjust the length of each of the encoded samples based on location-based attention, determine a positional weight of each word in the encoded samples based on a difference in length between a corresponding encoded sample and the target length, apply a softmax function to the positional weight of each word to represent the positional weight of each word as an output value between 0 and 1.0, and obtain, based on a sum of the output values, a weighted sum of hidden vectors corresponding to the encoded samples: this limitation amounts to a mathematical calculation which falls under the abstract idea grouping of mathematical concept.
Claim 1 does not contain any additional limitations which integrate the judicial exception into a practical application. The only additional limitations are ”a memory storing a data augmentation program; and a processor configured to execute the data augmentation program, wherein the processor is configured to execute the data augmentation program to cause the apparatus to”, “encode, using an encoder and an artificial intelligence model”, “reconstruct, using a decoder configured with a causal attention”, “wherein the processor is configured to execute the data augmentation program to cause the apparatus to…”. These limitations are recited at a high level of generality and amount to mere instructions to implement the judicial exception using a generic computer, which do not integrate the judicial exception into a practical application as they do not impose any meaningful limits on practicing the judicial exception. Therefore, claim 1 is directed to an abstract idea (Step 2A: YES).
Claim 1 does not contain any additional limitations which amount to significantly more than the judicial exception (Step 2B: NO). As discussed above, the only additional limitations amount to mere instructions to implement the judicial exception using a generic computer, which do not amount to significantly more than the judicial exception as they do not provide an inventive concept. Therefore, claim 1 is not patent eligible.
Regarding dependent claims 2-7, 9-14, and 22-23, “The apparatus” is recited, which is directed to one of the four statutory categories of invention (machine) (Step 1: YES). However, the claims limitations, under their broadest reasonable interpretation, recite mental processes or mathematical concepts which fall into the category of abstract idea (Step 2A Prong 1: YES).
The following limitations, under their broadest reasonable interpretation, recite mental processes or mathematical concepts:
Claim 2:
encode the plurality of input sentences using a byte pair encoding algorithm: this limitation amounts to a mathematical calculation which falls under the abstract idea grouping of mathematical concept
Claim 2 contains the additional limitation “wherein the encoder is configured to”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 3:
implemented as a bidirectional recurrent neural network or a transformer: this limitation amounts to a mathematical calculation which falls under the abstract idea grouping of mathematical concept
Claim 3 contains the additional limitation “wherein the encoder is… ”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 4:
receive each pair of sentences, among the plurality of sentences, and encode the each pair of sentences to output corresponding encoded samples represented as a set of hidden vectors: a person can receive each pair of sentences and write down an encoding for each set to create a set of hidden vectors.
Claim 4 contains the additional limitation “wherein the processor is further configured to cause the apparatus to…”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 5:
replace at least one token, of each of the plurality of input sentences, with a mask token: a person writes down a mask token for at least one token in each of the plurality of input sentences.
Claim 5 contains the additional limitation “wherein the processor is further configured to cause the apparatus to”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 6:
add Gaussian noise to the encoded samples: this limitation amounts to a mathematical calculation which falls under the abstract idea grouping of mathematical concept
Claim 6 contains the additional limitation “wherein the processor is further configured to cause the apparatus to”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 7:
wherein the original length of each of the plurality of input sentences remains unaltered while the plurality of input sentences are encoded: a person writes down an encoding for the plurality of input sentences, making it have the same length as the original length of each sentence
wherein the pair of input sentences have different lengths, and wherein the processor is further configured to cause the apparatus to: determine, based on a weighted combination of the different lengths of the pair of input sentences using the predetermined mixing ratio, the target length; and adjust different lengths of the encoded samples to match the target length: adjusting lengths based on calculated weighted combinations of lengths amounts to a mathematical calculation.
Claim 7 contains no additional limitations.
Claim 9:
linearly combine, according to the predetermined mixing ratio, the encoded samples having the length matched to the target length: this limitation amounts to a mathematical calculation which falls under the abstract idea grouping of mathematical concept
Claim 9 contains the additional limitation “wherein the processor is further configured to cause the apparatus to…” which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 10:
Claim 10 contains the additional limitation “wherein the decoder is implemented as a recurrent network or a transformer”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 11:
reconstruct the original sentence by receiving the interpolated hidden vector as an input; and reconstructing the original sentence in proportion to the predetermined mixing ratio: a person obtains a vector, and maps the vector to an original sentence using the predetermined mixing ratio
Claim 11 contains the additional limitation “wherein the decoder is configured to”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 12:
obtain a final loss by combining a learning objective function that obtains a loss through a difference between the reconstructed sentence and the original sentence with an L2 loss that applies an L2 regularization to the interpolated hidden vector: this limitation amounts to a mathematical calculation which falls under the abstract idea grouping of mathematical concept.
Claim 12 contains the additional limitation “wherein the processor is further configured to cause the apparatus to”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 13:
Claim 13 contains the additional limitation “wherein the encoder and the decoder are further configured to use a pre-learned model comprising a bidirectional auto-regressive transformer (BART)-large”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 14:
perform an inference on the reconstructed sentence; calculate a softmax probability for each label which is used in a classification task based on a result of the inference; and use a label with a highest probability as a correct label for the reconstructed sentence; and train, using the reconstructed sentence as training data, a classification model of a classifier: this limitation amounts to a mathematical calculation which falls under the abstract idea grouping of mathematical concept
Claim 14 contains the additional limitation “wherein the processor is further configured to cause the apparatus to”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 22:
training, based on training data, the artificial intelligence model; and control, using a regression model, weights of the hidden vectors for convergence…reconstruct the original sentence by reconstructing…the original sentence corresponding to the interpolated hidden vector: training and controlling weights of the hidden vectors for convergence amounts to mathematical concepts. Further, a person can reconstruct an original sentence corresponding to a hidden vector.
Claim 22 contains the additional limitation “wherein the processor is configured to cause the apparatus to reconstruct the original sentence by reconstructing, using the decoder and an artificial intelligence model…”, which amounts to mere instructions to implement the judicial exception using a generic computer.
Claim 23:
sample the predetermined mixing ratio from a uniform distribution between 0 and 1; down-sample or up-sample the encoded samples to match the target length; and generate a new sentence from the interpolated hidden vector using beam search: sampling from a uniform distribution, down/upsampling encoded samples, and performing beam search amount to mathematical concepts/calculations
Claims 2-7, 9-14, and 22-23 do not contain any additional limitations which integrate the judicial exception into a practical application (Step 2A Prong 2: NO). As discussed above, the only additional limitations are mere instructions to implement the judicial exception using a generic computer, which do not integrate the judicial exception into a practical application as they do not impose any meaningful limits on practicing the judicial exception. Therefore, claims 2-7, 9-14, and 22-23 are directed to an abstract idea (Step 2A: YES).
Claims 2-7, 9-14, and 22-23 do not contain any additional limitations which amount to significantly more than the judicial exception (Step 2B: NO). As discussed above, the only additional limitations amount to mere instructions to implement the judicial exception using a generic computer, which do not amount to significantly more than the judicial exception as they do not provide an inventive concept. Therefore, claims 2-7, 9-14, and 22-23 are not patent eligible.
Regarding claim 15, “A method” is recited, which is directed to one of the four statutory categories of invention (process) (Step 1: YES). However, the claims limitations, under their broadest reasonable interpretation, recite limitations similar to those recited in apparatus claim 1, and therefore also recites mental processes and mathematical concepts which fall into the category of abstract idea (Step 2A Prong 1: YES, see above for claim 1 analysis).
Claim 15 does not contain any additional limitations which integrate the judicial exception into a practical application (Step 2A Prong 2: NO). Claim 15 contains the additional limitations “executed by an apparatus comprising a memory and a processor”, “encoding, using an encoder and an artificial intelligence model”, “reconstructing, using a decoder configured with a causal attention”. These limitations are recited at a high level of generality and amounts to mere instructions to implement the judicial exception using a generic computer. Even when viewed in combination, mere instructions to implement the judicial exception using a generic computer do not integrate the judicial exception into a practical application as they do not impose any meaningful limits on practicing the judicial exception. Therefore, claim 15 is directed to an abstract idea (Step 2A: YES).
Claim 15 does not contain any additional limitations which amount to significantly more than the judicial exception (Step 2B: NO). As discussed above, the additional limitations are mere instructions to implement the judicial exception using a generic computer, which do not amount to significantly more than the judicial exception as they do not provide an inventive concept. Therefore, claim 15 is not patent eligible.
Regarding dependent claims 16 and 19-21, “The method” is recited, which is directed to one of the four statutory categories of invention (process) (Step 1: YES). However, the claims limitations, under their broadest reasonable interpretation, recite limitations similar to those in claims 5, 11-12, and 22, and thus also recite mental processes or mathematical concepts which fall into the category of abstract idea (Step 2A Prong 1: YES, see above analysis for claims 5, 11-12, and 22).
Claims 16 and 19-21 do not contain any additional limitations which integrate the judicial exception into a practical application (Step 2A Prong 2: NO). As discussed above, the only additional limitations are mere instructions to implement the judicial exception using a generic computer, which do not integrate the judicial exception into a practical application as they do not provide any meaningful limits on practicing the abstract idea. Therefore, claims 16 and 19-21 are directed to an abstract idea.
Claims 16 and 19-21 do not contain any additional limitations which amount to significantly more than the judicial exception (Step 2B: NO). As discussed above, only additional limitations amount to mere instructions to implement the judicial exception using a generic computer, which do not amount to significantly more than the judicial exception as they do not provide an inventive concept. Therefore, claims 16 and 19-21 are not patent eligible.
Allowable Subject Matter
6. Claims 1-7, 9-16, and 19-23 would be allowable if rewritten or amended to overcome the rejection under 35 U.S.C. § 101 and the rejection under 35 U.S.C. § 112(a).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Araki (US 2022/0269863 A1): sentence generator for data augmentation, for a given single sentence, generate several augmented data sentences (Fig. 1)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CODY DOUGLAS HUTCHESON whose telephone number is (703)756-1601. The examiner can normally be reached M-F 8:00AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre-Louis Desir can be reached at (571)-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CODY DOUGLAS HUTCHESON/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Prosecution Insights