DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to arguments
All previous objections/rejections not mentioned in this office action have been withdrawn by the examiner.
Applicant's arguments filed 10/02/2025 arguments have been fully considered, but they are not persuasive. Please see the following for further detail.
Regarding the rejection under 101, Examiner updated rejection based on amendment and the rejection maintained.
Applicant arguments with respect to claims 1, and 4-20 have been considered but are moot because the new ground of rejection (103) does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 4-20 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Applicant amended independent claims 1, 16 and 20. Also, canceled claims 2 and 3. Applicant incorporate claim 2 and 3 in all independent claims but defined it more specific way. For example
With respect to independent claims 1, 16, and 20, the claims have been amended to recite
training the neural network using a difference between the second target-side data and the first target-side data,
wherein the transformation model comprises an attention layer for the first embedding vector and an implicit layer, and
wherein the implicit layer is configured to receive the first embedding vector as input and output the second embedding vector and
implicit layer using an output vector as input until a preset condition is satisfied.” Para[001], mention about implicit layer and source and target side embedding vector. but there is no support for first and second embedding vector with implicit layer.
Newly amended all independent claims 1, 16 and 20 looks like “… wherein the transformation model comprises an attention layer for the first embedding vector and an implicit layer, and …”
Obviously, claim scope has changed.
Also claim 6 has training but not exactly like newly amended claims.
In newly amended independent claims define training: “training the neural network using a difference between the second target-side data and the first target-side data,”).
In the specification applicant provided training the model with difference between decoding result (para[0015], [0018], [0019], and [0080], difference between embedding vector (para[0027], [0079]).
But nowhere in the spec says training model using second and first target side data. “… using a difference between the second target-side data and the first target-side data,”).
(“training the neural network using a difference between the second target-side data and the first target-side data,”).
In specification there is no support for newly amended claims. Applicant needs to define all newly amended claims in the present disclosure. Also, claim scope has changed.
In the interest of compact prosecution, the Examiner is interpreting the “Second target side data” –, the second target-side data is the decoded embedding vector. Please see entire 103 rejection.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4–15, 16–19, and 20 are rejected under 35 U.S.C. § 101 because the claims are directed to a judicial exception (an abstract idea) and do not recite additional elements that amount to significantly more.
Summary of the statutory framework and guidance applied
The Office applies the two-step framework for subject matter eligibility consistent with current USPTO guidance.
Step 1: Determine whether the claim recites a statutory category (process, machine, manufacture, or composition of matter).
Step 2A, Prong One: If the claim recites a statutory category, determine whether the claim recites a judicial exception (abstract idea, law of nature, or natural phenomenon).
Step 2A, Prong Two: If a judicial exception is recited, determine whether the claim integrates the exception into a practical application.
Step 2B: If the claim does not integrate the exception into a practical application, determine whether the claim recites additional elements that amount to significantly more than the judicial exception.
Step 1 (statutory category)
Claims 1, 4–15 are directed to a method (process).
Claims 16–19 are directed to a system (machine).
Claim 20 is directed to a non-transitory computer-readable medium (manufacture). Each of these falls within a statutory category.
Step 2A, Prong One (judicial exception)
The claims recite mathematical concepts and data manipulation, including transforming text into embedding vectors, mapping vectors between embedding spaces, decoding vectors into text, and training neural networks based on calculated differences. These elements constitute a judicial exception in the form of abstract mathematical concepts and data processing.
Step 2A, Prong Two (integration into a practical application)
The claims do not integrate the judicial exception into a practical application. The additional elements recited—generic computing components (processors, memory, computing device), embedding models, attention layers, implicit/iterative layers, decoders, pretrained models, and training based on differences—are described at a high level and perform conventional functions of receiving, transforming, training, and outputting data. The claims do not describe a specific improvement to the functioning of the computer or another technological improvement. They instead use generic computer implementation to carry out the abstract mathematical operations.
Step 2B (significantly more)
The claims recite no additional elements or combination of elements that amount to significantly more than the abstract idea. The claimed processors, memories, models, layers, and training steps are well-understood, routine, and conventional activities and components in the field of machine learning and natural language processing. The claims do not recite a non-conventional arrangement of components or otherwise specify how the claimed elements effect a technological improvement.
Limitation-by-limitation analysis for independent claims
Claim 1 (method) – Limitation analysis
“Obtaining a pair of source-side data and first target-side data, the source-side data being text data in a first language and the first target-side data being text data in a second language” — This is data gathering/input identification and is a conventional, pre-solution activity.
“Transforming the source-side data into a first embedding vector located in a source-side embedding space of the first language through a source-side embedding model” — This recites converting text into a vector representation, a mathematical/data transformation.
“Transforming the first embedding vector into a second embedding vector located in a target-side embedding space of the second language through a transformation model” — This recites mapping between vector spaces, i.e., a mathematical manipulation.
“Decoding the second embedding vector through a target-side decoder, wherein the target-side decoder is a neural network configured to decode the second embedding vector into second target-side data in the second language” — This recites translating a vector back into text via a neural network; it is an abstract data transformation implemented on a generic computing substrate.
“Training the neural network using a difference between the second target-side data and the first target-side data” — This recites routine model training using an error/difference measure, a mathematical calculation and conventional ML practice.
“Wherein the transformation model comprises an attention layer for the first embedding vector and an implicit layer” — The recitation of model components at a high level (attention, implicit/iterative layer) is conventional and lacks a specific technical implementation or improvement.
“Wherein the implicit layer is configured to receive the first embedding vector as input and output the second embedding vector and repeatedly perform a layer operation based on a value of a weight parameter of the implicit layer using an output vector as input until a preset condition is satisfied” — This describes iterative computation within the model (a loop/recurrence) and is a mathematical/algorithmic operation implemented on generic hardware. The claim does not show this arrangement provides a specific technical improvement.
Conclusion for claim 1: The claim is directed to abstract mathematical/data transformations and recites only.
Conventional computing/ML elements; it does not integrate the judicial exception into a practical application and does not include additional elements that amount to significantly more.
Claim 16 (system) – Limitation analysis
The claim recites generic hardware components (“one or more processors,” “memory”) configured to execute instructions that perform the same abstract data transformations and training steps recited in claim 1. Implementing the abstract method on generic hardware does not provide a practical application or significantly more.
Conclusion for claim 16: The system claim is ineligible for the same reasons as claim 1.
Claim 20 (computer-readable medium) – Limitation analysis
The claim recites instructions stored on a non-transitory medium that cause a processor to perform the same abstract steps recited in claim 1. The storage of instructions to perform an abstract method on a computer-readable medium does not render the subject matter eligible. The specification at paragraph [0029] mentions computer readable storage medium which can be reasonably interpreted as a computer program being recorded or downloaded, but still a computer program, which is not patentable under the most recent 35 U.S.C. 101 guidelines
Conclusion for claim 20: The claim is ineligible for the same reasons as claim 1.
Dependent claims (claims 4–15, 17–19)
The dependent claims add details such as use of pretrained models, specific dataset composition, additional embedding vectors, freezing certain model weights during training, configuration to accept multiple inputs, provision of source/target identifiers, cross-modal mapping, and similar refinements. These added features are conventional activities, parameters, or configurations in machine learning and data processing. They do not integrate the abstract idea into a practical application, nor do they supply additional elements or an inventive concept sufficient to render the claims eligible.
Overall conclusion
Claims 1, 4–15, 16–19, and 20 are rejected under 35 U.S.C. § 101 as being directed to an abstract idea (mathematical concepts and data manipulation) and failing to recite additional elements or combinations that amount to significantly more than the judicial exception. The claims are therefore not directed to patent-eligible subject matter.
REJECTION
Claims 1, 4–15, 16–19, and 20 are rejected under 35 U.S.C. § 101 as being directed to non-statutory subject matter (an abstract idea) and lacking additional limitations that transform the abstract idea into patent-eligible subject matter.
Examiner’s Remarks
All claims should be conditioned for allowances, once all rejections, and conditions are resolved.
Claims 1, 4-20 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101 set forth in this Office action and 112(a)., but note that it may no longer be ASM once the new matter issue is resolved.
Also, note that changing the independent claims to language that are different from these independent claims, will result in an updated search where new ground of rejection may be presented. As such, to prevent this issue, applicant should provide language similar to the amended claims with proper support from the specification.
The claims recite detailed limitations by defining an embedding transformation method performed by at least one computing device, the method comprising: “obtaining a source-side embedding model; obtaining a pair of source-side data and first target-side data, the source-side data being text data in a first language and the first target-side data being text data in a second language; transforming the source-side data into a first embedding vector located in a source-side embedding space of the first language through a the source-side embedding model; and transforming the first embedding vector into a second embedding vector located in a target-side embedding space of the second language through a transformation model, decoding the second embedding vector through a target-side decoder, wherein the target- side decoder is a neural network configured to decode the second embedding vector into second target-side data in the second language; and training the neural network using a difference between the second target-side data and the first target-side data, wherein the transformation model comprises an attention layer for the first embedding vector and an implicit layer, and wherein the implicit layer is configured to receive the first embedding vector as input and output the second embedding vector and repeatedly perform a layer operation based on a value of a weight parameter of the implicit layer using an output vector as input until a preset condition is satisfied.”
After performing an extensive search, the examiner determined that prior art of the record does not anticipate or render obvious the claimed invention defined by each of independent claims.
Once all rejection, and conditions are resolved, all claims should be conditioned for allowances.
Allowable Subject Matter
Claims 1, 4-20 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101 set forth in this Office action and 112(a). Such amendments should further consider having all independent base claims the same in scope, with the exception of the embodiments. The combination of the cited references does not reasonably teach or suggest the pending claim features.
Claims 1, 16, and 20 is considered allowable, since certain key features of the claimed invention are not taught or fairly suggested by the prior art, particularly in ordered combination with the rest of the limitations recited within each of the independent claims.
The primary reason for considering Claim 1, 16, and 20 is the combination of limitations recited in Claim 1, “obtaining a pair of source-side data and first target-side data, the source-side data being text data in a first language and the first target-side data being text data in a second language; transforming the source-side data into a first embedding vector located in a source-side embedding space of the first language through a of the second language through a transformation model. decoding the second embedding vector through a target-side decoder, wherein the target- side decoder is a neural network configured to decode the second embedding vector into second target-side data in the second language; and training the neural network using a difference between the second target-side data and the first target-side data, wherein the transformation model comprises an attention layer for the first embedding vector and an implicit layer, and wherein the implicit layer is configured to receive the first embedding vector as input and output the second embedding vector and repeatedly perform a layer operation based on a value of a weight parameter of the implicit layer using an output vector as input until a preset condition is satisfied.”
The closest prior art is Lee et al. US 20220319500 A1- Lee teaches (“[0084] According to an embodiment, the at least one sub-network layer 530 may include a multi-head attention layer 531, a first layer 535a, a feed-forward layer 533, and/or a second layer 535b. The at least one sub-network layer 530 may receive an embedding vector, to which positional encoding is applied, from the positional encoding layer 520. The first linear normalization layer 532 may be included in an input terminal of the at least one sub-network layer 530. The embedding vector, to which positional encoding is applied in the positional encoding layer 520, may be branched and entered into the first linear normalization layer 532. Each embedding vector may be linearly normalized.”) (“[0150] According to an example embodiment, the memory may store one or more instructions that, when executed, cause the processor to: enter data into the language model, generate an embedding vector based on the data in the input embedding layer, add position information to the embedding vector in the positional encoding layer, branch the embedding vector based on domain information included in the embedding vector, normalize the branched embedding vectors, enter the normalized embedding vectors into the multi-head attention layer, enter output data of the multi-head attention layer into the first layer, normalize pieces of output data of the first layer, enter the normalized pieces of output data of the first layer into the feed-forward layer, enter output data of the feed-forward layer into the second layer and to normalize pieces of output data of the second layer, and enter the normalized pieces of output data of the second layer into the linearization layer and the softmax layer to obtain result data.”)
The second closest prior art is Lee et al. US 20180373704 A1 - Lee teaches (“[0099] The training apparatus may adjust an internal parameter of the encoders or the decoder to increase the accuracy in translation. The training apparatus may train the encoders or the decoder to decrease a difference between a target sentence and an actual translated sentence. The training apparatus may achieve an end-to-end training in response to training neural network models by reflecting a result of using a supplement sentence.”) (“[0109] The training apparatus may acquire a second vector value by inputting the supplement sentence to the first encoder 511. The first vector value may be acquired from the second vector value and the training source sentence 931 using the second encoder 513 based on a neural network model. The training apparatus may output a target sentence by inputting the first vector value to the decoder 515.” ) (“[0110] The training apparatus may train the decoder 515 and the integrated encoder 510 by evaluating an accuracy of the target sentence. The training apparatus may train the integrated encoder 510 and the decoder 515 to decrease a difference between the target sentence and an accurate result of translation. The training apparatus may train the integrated encoder 510 and the decoder 515 until a desired level of accuracy is achieved.”)
The next prior art is REZAGHOLIZADEH et al. US 20210390269 A1- REZAGHOLIZADEH teaches (“(“[0077] Then, finally, at task block 408A, second decoder submodule 2128 receives sentence embedding z.sub.y and decodes z.sub.y to generate a vector representation (e.g., one-hot vector representation) of text sequence y in second language L2. Further processing may be performed to convert the vector representation of text sequence y into textual representation.”) (“[0090] … the decoder submodule of the trained auto-encoder may be used to obtain a translation by decoding a sentence embedding from the latent code space to a text sequence in a desired target language.”) (“[0051] It will further be appreciated that any differences between the input one-hot vector representations of text sequences x, y, and the corresponding reconstructed output {circumflex over (x)}, ŷ (see FIG. 2A) from decoder submodules 202B, 212B may be quantified as “reconstruction losses”. In various contemplated embodiments, reconstruction losses may be minimized by implementing a “loss function” directed to optimizing the training processes of first and second auto-encoder modules 202, 212. The loss function may embody, for example, iterative gradient descent variant techniques that monitor the correlations between the input text sequences and the reconstructed output text sequences and provide iterative feed-back data to adjust and fine-tune various parameters (e.g., weights) of the encoder and decoder neural networks. That is, first and second auto-encoder modules 202A, 212A may be iteratively trained to learn the latent code representations of text sequences in the latent code space 210 until the quantified loss values, as determined by the loss function, are reduced to a degree where the quality of the reconstructed outputs {circumflex over (x)}, ŷ substantially resemble input text sequences x,y to an acceptable, predetermined threshold level.”) (“[0060] The SPN module 220 may be configured to operate under Gaussian distributions or any other suitable probability distribution over the real vector-based values of sentence embeddings z.sub.x, z.sub.y. Moreover, the SPN module 220 may be constructed or generated, at least in part, by automatic structure learning algorithms for SPNs. Furthermore, SPN module 220 may incorporate any suitable elements having adjustable parameters, which may be iteratively adjusted during training. In this manner, the parameters of SPN module 220 may be iteratively adjusted during the second training phase to maximize the determination of the joint probability distribution of pairs of sentence embeddings z.sub.x, z.sub.y, P(z.sub.x,z.sub.y), for an aligned multilingual text corpus of languages L1, L2.”)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FOUZIA HYE SOLAIMAN whose telephone number is (571)270-5656. The examiner can normally be reached M-F (8-5)AM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras D. Shah can be reached at (571) 270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/F.H.S./Examiner, Art Unit 2653
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659