DETAILED ACTION
This action is responsive to Remarks and Claim Amendments filed on August 05, 2026.
Claims 1-2 and 4-9 have been amended. Claim 3 has been canceled.
Claims 1-2 and 4-9 are pending and are presented to examination.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Response to Amendments
The objection of claims 1-9 is withdrawn in view of applicant’s amendments.
The objection to the abstract set forth in the prior action is withdrawn in view of the substitute abstract filed with the Amendment.
The objection to the title set forth in the prior action is withdrawn in view of the amended title filed with the Amendment.
Applicant argues at page 8 of the Remarks that the amended claims do not invoke 35 U.S.C. 112(f) because the nonce term "unit" has been removed and sufficient structure is now recited. Applicant's argument is persuasive as to claims 1-8. Amended claims 1-7 recite a processor and a memory storing instructions that configure the processor to perform the recited functions, and amended claim 8 is drafted in method form reciting affirmative steps. The interpretation of the limitations of claims 1-3 and 5-7 under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, set forth in the prior action is therefore withdrawn. Claim 3 has been canceled. Because 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is no longer invoked as to claims 1-8, the rejection of claims 1, 6 and 7 under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph, predicated on the failure of the written description to disclose a training algorithm corresponding to the claimed learning function (Aristocrat Techs. v. Int'l Game Tech., 521 F.3d 1328 (Fed. Cir. 2008); MPEP § 2181, subsection II(B)) is withdrawn. Applicant's argument is not persuasive as to claim 9. Amended claim 9 continues to recite "a masking unit that masks a specific type of token...." The term "unit" remains a generic placeholder modified by functional language and unmodified by sufficient structure. This limitation continues to be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, as set forth in the Claim Interpretation section below. Because corresponding structure for the masking function is disclosed and clearly linked at Specification paragraphs [0061]-[0070] and Figure 9, the Aristocrat-based rejection under 35 U.S.C. 112(b) is not maintained as to that limitation. Claim 9 is, however, rejected under 35 U.S.C. 112(b) for the separate reasons set forth below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claim 9 in this application is given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word "means," but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: "a masking unit that masks a specific type of token of first source code written in a first programming language second source code written in a second programming language with a different placeholder for each of a plurality of the-tokens, as learning data" in claim 9.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
A review of the specification shows that the following appears to be the corresponding structure described in the specification for the 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph limitation "masking unit" in paragraphs [0061]-[0070] and figure 9, which is described as an algorithm to: parse, iterate with nodes, check a map and assign placeholder.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 4-5 and 8-9 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 8 — lack of antecedent basis for "the input source code." Amended claim 8 recites creating a translation model "for converting the first source code written in the first programming language into output source code written in the second programming language," and thereafter recites "applying the translation model to the input source code and convert the input source code into the output source code." Claim 8 does not previously recite "input source code." Unlike amended claim 1, which introduces "input source code" in the model-creation step, claim 8 recites "the first source code" in that step. The term "the input source code" therefore lacks antecedent basis, and it is unclear whether "the input source code" is intended to be the same as, or different from, "the first source code." Claim 8 additionally recites "and convert the input source code," which is not in gerund form consistent with the remaining steps of the method claim.
Claims 4 and 5 — scope of "the second source code." Amended claim 1 recites "second source code written in a second programming language" as material that is masked and learned as training data, and separately recites the translated result as "output source code." Amended claim 4 recites that "the second source code masked with the placeholders is output upon determining there is no instruction for post-processing," and amended claim 5 recites outputting "the second source code for which post-processing is performed." Read literally against the antecedent established in claim 1, claims 4 and 5 require outputting the masked training corpus rather than the translated result. This appears inconsistent with Specification paragraphs [0090]-[0094], which describe post-processing of the translated program. It is therefore unclear whether "the second source code" in claims 4 and 5 refers to the second source code masked as learning data in claim 1 or to the output source code produced by the translation model. Claims 4 and 5 further recite the outputting function in the passive voice without identifying the structure that performs it.
Claim 2 — lack of antecedent basis for "the tokens." Amended claim 2 recites "masked with the placeholders for each of the tokens." Amended claim 1 recites "a specific type of token" and "a plurality of tokens." It is unclear whether "the tokens" refers to the plurality of tokens, to tokens of the specific type, or to some other set. Claim 2 further recites the masking function in the passive voice without identifying the structure that performs it.
Claim 9 — multiple grounds. First, claim 9 recites "a specific type of token of first source code written in a first programming language second source code written in a second programming language," omitting a conjunction between the two recited source codes, such that the relationship between them cannot be determined. Second, claim 9 recites "each of a plurality of the-tokens"; the hyphenated construction is unclear and "the tokens" lacks antecedent basis for the reasons given for claim 2. Third, claim 9 recites that "the preprocessing device includes a masking unit that masks...., learn the first source code masked with the placeholders and learn the second source code masked with the placeholders and create a translation model...., and apply the translation model...." The functions of learning, creating, and applying are recited without any structure to which they are attributed; it cannot be determined whether these functions are performed by the masking unit, by the preprocessing device, or by the translation model creation device. Fourth, claim 9 positively recites "a translation model creation device" but attributes no function whatsoever to that device, rendering its role in the claimed system indeterminate.
For purposes of applying prior art below, and consistent with MPEP § 2173.06, the claims are examined as best understood. Specifically, "the second source code" in claims 4 and 5 is interpreted as the output source code produced by the translation model, consistent with Specification paragraphs [0090]-[0094]; the omitted conjunction in claim 9 is interpreted as "and"; and the learning, creating, and applying functions of claim 9 are interpreted as performed by the recited system.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 and 8-9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Baptiste Roziere et al. (“DOBF: A Deobfuscation Pre-Training Objective for Programming Languages”, hereinafter “Roziere” – previously presented).
With respect to claim 1 (Currently Amended), Roziere teaches A programming language conversion device comprising: a processor; and a memory coupled to the processor, the memory storing instructions, that when executed by the processor, configures the processor to: (Roziere discloses a computer-implemented system executing stored instructions on hardware processors and associated memory. See Roziere § 4.3 ("We implement our models in PyTorch Paszke et al. [2019] and train them on 32 V100 GPUs for eight days. We use float16 operations to speed up training and to reduce the memory usage of our models.").)
mask a specific type of token of first source code written in a first programming language and second source code written in a second programming language with a different placeholder for each of a plurality of tokens, as learning data (Roziere discloses masking a specific type of token — namely class, function, and variable names (identifiers) — and expressly discloses doing so for source code in both of two programming languages, Python and Java, to produce the training dataset. As to the specific type of token, see Roziere § 3.2 ("We obfuscate code snippets by replacing class, function and variable names with special tokens, and train a model to recover the original names."). As to a different placeholder for each of a plurality of tokens, see Roziere § 3.3 ("Obfuscated classes, functions, and variables, are replaced with associated special tokens: CLASS_0 . . . CLASS_N, FUNC_0 . . . FUNC_N and VAR_0 . . . VAR_N.") and § 3.2 ("When an identifier is selected, all of its instances in the code are replaced by the same special token."), establishing that each distinct identifier receives its own distinct placeholder while every instance of that identifier maps to that same placeholder. See also Roziere Figure 1 (mapping the identifiers bfs, graph, root, visited, queue, neighbor, and node to FUNC_0 and V0-V5 respectively). As to masking source code in a first programming language and source code in a second programming language as learning data, see Roziere § 4.3, under the heading Training dataset ("We obfuscate each file and create the corresponding dictionary of masked identifier names, resulting in a parallel (obfuscated file - dictionary) dataset of 19 GB for Python and 26 GB for Java."). Roziere thus expressly obfuscates source code in both Python and Java and uses the resulting obfuscated files as the training dataset.)
learn the first source code masked with the placeholders and learn the second source code masked with the placeholders and create a translation model for converting input source code written in the first programming language into output source code written in the second programming language (Roziere discloses training the model on the masked source code of both languages, and discloses that the model so trained is the model that performs translation from the first programming language to the second. As to learning the masked source code of both languages, see Roziere § 4.3 ("During DOBF training, we alternate between batches of Java and Python composed of 3000 tokens per GPU."), i.e., the masked Java source code and the masked Python source code are both consumed as training input by the same model. As to creating a translation model for converting source code of the first programming language into source code of the second programming language, see Roziere § 4.2 ("TransCoder Roziere et al. [2020] is an unsupervised machine translation model which translates functions and methods between C++, Java, and Python. A single seq2seq model is trained for all languages."), § 5.2 ("For unsupervised translation, DOBF+DAE increases the computational accuracy by 1.9% when translating from Python to Java, and by 6.8% when translating from Java to Python"), and Table 2 (reporting CA@1 of 38.7/40.6 for Python to Java and 40.0/42.4 for Java to Python). See also Roziere Abstract ("models pre-trained with DOBF significantly outperform existing approaches on multiple downstream tasks, providing relative improvements of up to 12.2% in unsupervised code translation"). The translation model whose accuracy Roziere reports is the model produced by training on the masked Java and Python source code.) and
apply the translation model to the input source code and convert the input source code into the output source code (Roziere discloses applying the trained translation model to input source code written in a first programming language and obtaining output source code written in a second programming language. See Roziere § 4.2 ("TransCoder is evaluated using the Computational Accuracy metric, which computes the percentage of correct solutions according to series of unit tests. We only consider a single model output (CA@1), with beam sizes of 1 and 10."). The computational accuracy metric presupposes that the model has been applied to first-language input source code and has produced second-language output source code capable of being compiled and executed against unit tests. See also Roziere § 5.2 and Table 2 (reporting the Python to Java and Java to Python results so obtained).
With respect to claim 8, the claim is directed to a method that corresponds to the device recited in claim 1, respectively (see the rejection of claim 1 above).
With respect to claim 9, the claim recites limitations similar to claim 1 in system form and is rejected for the same reasons set forth for claim 1. As to the recitation of a preprocessing device and a translation model creation device, Roziere discloses both functional roles within its pipeline: an obfuscation and preprocessing stage that masks identifiers and produces the training dataset (Roziere §§ 3.2, 4.3), and a model-training stage that produces the translation model (Roziere §§ 3.3, 4.2, 4.3, Table 2). The recitation of these roles as residing on separate devices does not patentably distinguish the claimed system from Roziere's computer-implemented system, in which preprocessing and training are separate, sequentially executed software components running on a computing cluster of 32 V100 GPUs (Roziere § 4.3). As to the masking unit interpreted under § 112(f), Roziere's obfuscation procedure performs the identical algorithm identified as the corresponding structure at Specification paragraphs [0061]-[0070] and Figure 9: Roziere parses each source file, identifies identifier nodes, maintains a dictionary mapping each identifier to its assigned special token, and substitutes that token for every instance of the identifier (Roziere §§ 3.2, 3.3, 4.3).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Baptiste Roziere et al. (“DOBF: A Deobfuscation Pre-Training Objective for Programming Languages”, hereinafter “Roziere” – previously presented) in view of Marie-Anne Lachaux et al. ("Unsupervised Translation of Programming Languages", hereinafter “Lachaux”).
Examiner notes: This alternative ground is presented to address Applicant's contention at page 4 of the Remarks that Roziere does not disclose creating the translation model using both the masked first source code and the masked second source code. The rejection under § 102(a)(1) is maintained; this ground is presented in the alternative only.
With respect to claim 1 (Currently Amended), Roziere teaches A programming language conversion device comprising: a processor; and a memory coupled to the processor, the memory storing instructions, that when executed by the processor, configures the processor to: (Roziere discloses a computer-implemented system executing stored instructions on hardware processors and associated memory. See Roziere § 4.3 ("We implement our models in PyTorch Paszke et al. [2019] and train them on 32 V100 GPUs for eight days. We use float16 operations to speed up training and to reduce the memory usage of our models.").) mask a specific type of token of first source code written in a first programming language and second source code written in a second programming language with a different placeholder for each of a plurality of tokens, as learning data (Roziere discloses masking a specific type of token — namely class, function, and variable names (identifiers) — and expressly discloses doing so for source code in both of two programming languages, Python and Java, to produce the training dataset. As to the specific type of token, see Roziere § 3.2 ("We obfuscate code snippets by replacing class, function and variable names with special tokens, and train a model to recover the original names."). As to a different placeholder for each of a plurality of tokens, see Roziere § 3.3 ("Obfuscated classes, functions, and variables, are replaced with associated special tokens: CLASS_0 . . . CLASS_N, FUNC_0 . . . FUNC_N and VAR_0 . . . VAR_N.") and § 3.2 ("When an identifier is selected, all of its instances in the code are replaced by the same special token."), establishing that each distinct identifier receives its own distinct placeholder while every instance of that identifier maps to that same placeholder. See also Roziere Figure 1 (mapping the identifiers bfs, graph, root, visited, queue, neighbor, and node to FUNC_0 and V0-V5 respectively). As to masking source code in a first programming language and source code in a second programming language as learning data, see Roziere § 4.3, under the heading Training dataset ("We obfuscate each file and create the corresponding dictionary of masked identifier names, resulting in a parallel (obfuscated file - dictionary) dataset of 19 GB for Python and 26 GB for Java."). Roziere thus expressly obfuscates source code in both Python and Java and uses the resulting obfuscated files as the training dataset.) learn the first source code masked with the placeholders and learn the second source code masked with the placeholders [[and create a translation model for converting input source code written in the first programming language into output source code written in the second programming language]] (Roziere discloses training a single model on the masked source code of both languages. See Roziere § 4.3 ("During DOBF training, we alternate between batches of Java and Python composed of 3000 tokens per GPU.").)
apply the translation model to the input source code and convert the input source code into the output source code (Roziere discloses applying the trained translation model to input source code written in a first programming language and obtaining output source code written in a second programming language. See Roziere § 4.2 ("TransCoder is evaluated using the Computational Accuracy metric, which computes the percentage of correct solutions according to series of unit tests. We only consider a single model output (CA@1), with beam sizes of 1 and 10."). The computational accuracy metric presupposes that the model has been applied to first-language input source code and has produced second-language output source code capable of being compiled and executed against unit tests. See also Roziere § 5.2 and Table 2 (reporting the Python to Java and Java to Python results so obtained).
Roziere is silent to disclose and create a translation model for converting input source code written in the first programming language into output source code written in the second programming language; however, in an analogous art, Lachaux teaches: learn the first source code masked with the placeholders and learn the second source code masked with the placeholders and create a translation model for converting input source code written in the first programming language into output source code written in the second programming language (Lachaux discloses that a model is pre-trained by masking tokens in source code sequences drawn from multiple programming languages, and that the resulting pre-trained model is the model from which the translation model is created. As to masking and learning across both languages, see Lachaux § 3.1 ("For the masked language modeling (MLM) objective, at each iteration we consider an input stream of source code sequences, randomly mask out some of the tokens, and train TransCoder to predict the tokens that have been masked out based on their contexts. We alternate between streams of batches of different languages."). As to the pre-trained model becoming the translation model, see Lachaux § 3.2 ("We initialize the encoder and decoder of the seq2seq model with the XLM model pretrained in Section 3.1."). Lachaux thus expressly teaches that the model trained on the masked source code of the several languages is carried forward as, and becomes, the translation model. As to the translation function of the resulting model, see Lachaux Abstract ("We train our model on source code from open source GitHub projects, and show that it can translate functions between C++, Java, and Python with high accuracy.").) It would have been obvious to one of ordinary skill in the art at the time the invention was made before the effective filing date of the claimed invention to create the translation model of Roziere by initializing a sequence-to-sequence translation model with the model trained on the masked first and second source code, as taught by Lachaux. The motivation to do so is supplied by Roziere itself: Roziere expressly adopts Lachaux's TransCoder as the downstream translation task on which the model trained on masked Java and Python source code is fine-tuned and evaluated. See Roziere § 4.2 ("TransCoder Roziere et al. [2020] is an unsupervised machine translation model which translates functions and methods between C++, Java, and Python. A single seq2seq model is trained for all languages. In the original work, TransCoder is pre-trained with MLM, and trained with denoising auto-encoding and back-translation."). One of ordinary skill following Roziere is therefore directed to Lachaux for the specific mechanism by which a model trained on masked source code is converted into a translation model, and Lachaux supplies that mechanism expressly at § 3.2. The combination is the use of a known technique (Lachaux's initialization of a sequence-to-sequence translation model from a masked-token pre-trained model) to improve a similar device (Roziere's masked-token training pipeline) in the same way, yielding the predictable result of a translation model that converts source code of a first programming language into source code of a second programming language.
With respect to claim 8, the claim recites limitations similar to claim 1 in method form and is rejected for the same reasons set forth for claim 1. With respect to claim 9, the claim recites limitations similar to claim 1 in system form and is rejected for the same reasons set forth for claim 1, including the treatment of the preprocessing device, the translation model creation device, and the masking unit set forth in the rejection of claim 9 under 35 U.S.C. § 102(a)(1) above (Roziere §§ 3.2, 3.3, 4.2, 4.3; Lachaux §§ 3.1, 3.2, Abstract).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Baptiste Roziere et al. (“DOBF: A Deobfuscation Pre-Training Objective for Programming Languages”, hereinafter “Roziere” – previously presented) in view Tian et al. (“Using Latent Dirichlet Allocation for Automatic Categorization of Software”, hereinafter “Tian” – previously presented).
With respect to claim 2 (Currently Amended), Roziere teaches wherein the [[filtered]] first source code is masked with the placeholders for each of the tokens (Roziere's obfuscation procedure replaces each distinct identifier in the source code with its own distinct placeholder, with all instances of a given identifier replaced by the same placeholder. See Roziere § 3.2 ("We obfuscate code snippets by replacing class, function and variable names with special tokens, and train a model to recover the original names.") and § 3.3 ("Obfuscated classes, functions, and variables, are replaced with associated special tokens: CLASS_0 . . . CLASS_N, FUNC_0 . . . FUNC_N and VAR_0 . . . VAR_N.").) Roziere is silent to disclose filter the first source code based on a topic and is silent to disclose that the source code so masked is filtered; Roziere's preprocessing of the GitHub corpus is limited to license-based selection and de-duplication, and Roziere does not select source files for training on the basis of topical category (Roziere § 4.3). However, in an analogous art, Tian teaches: filter the first source code based on a topic (Tian's LACT system receives a corpus of source code, applies Latent Dirichlet Allocation to derive a topic-document distribution, and assigns each software system to one or more topic-based categories. Each software system is parsed and represented as a document composed of identifiers and comments extracted from its source code (Tian § 2, step 1; Table 1); LDA is then applied such that "each document (i.e., software system) is probabilistically associated with a set of topics" (Tian § 2, step 2); and each software system is assigned to a category where "one of the category topics belongs to a software system with a probability above a certain distribution threshold" (Tian § 2, step 4). The resulting categories are topical in nature, including problem-domain categories such as Game, Editor, Database, Terminal, E-mail, and Chat, and library or architecture based topics such as GTK, MFC, SSL, and XML (Tian §§ 3.1-3.2). The output of LACT is therefore a topic-categorized partition of the input source-code corpus, which operates to select source code according to topic.)
wherein the filtered first source code is masked with the placeholders for each of the tokens (Tian supplies the filtered antecedent. The topic-categorized partition produced by Tian's LACT system is the topic-filtered first source code, which in the combination is supplied as the input to Roziere's masking procedure (Tian § 2, steps 1-4; §§ 3.1-3.2).)
It would have been obvious to one of ordinary skill in the art at the time the invention was made before the effective filing date of the claimed invention to position Tian's topic-based filtering upstream of Roziere's masking and training pipeline, so as to filter the training corpus by topic before the source code is masked and learned. The motivation to do so arises from a problem acknowledged in the art and in Applicant's own disclosure: a translation model trained on source code drawn from heterogeneous topics learns inconsistent vocabulary and idioms across languages, which degrades translation quality. See Specification paragraph [0014] (stating that when topics or functions of both languages of learning data are different, translation quality of a translation model is significantly affected) and paragraph [0032] (acknowledging clustering by a statistical method such as Latent Dirichlet Allocation on source code as a known topic classification method). One of ordinary skill, faced with Roziere's broad GitHub-harvested corpus and seeking to improve translation quality, would have been led to apply Tian's LDA-based source-code categorization for the express purpose Tian describes, namely organizing a source-code corpus by topic, with the predictable result of yielding a topically consistent training set for the masking and training pipeline of Roziere.
Claims 4-6 are rejected under 35 U.S.C. 103 as being unpatentable over Baptiste Roziere et al. (“DOBF: A Deobfuscation Pre-Training Objective for Programming Languages”, hereinafter “Roziere” – previously presented) in view Le et al. (US Pub. No. 2019/0188268, hereinafter “Le” - previously presented).
With respect to claim 4 (Currently Amended), Roziere is silent to disclose wherein the second source code masked with the placeholders is output upon determining there is no instruction for post-processing. Roziere does not describe the inference-time disposition of placeholder tokens appearing in the translated output, and does not describe a conditional output mode in which the masked output is delivered when no post-processing instruction is given. However, in an analogous art, Le teaches: wherein the second source code masked with the placeholders is output upon determining there is no instruction for post-processing (Le discloses a neural translation system partitioned into a neural network translation model (component 120) that emits target sentences containing pointer (placeholder) tokens, and an architecturally separate rare word processing subsystem (component 130) that performs post-processing on those placeholder tokens using a word dictionary (component 140). See Le paragraphs [0015]-[0017] and Figure 1. Because the post-processing function is performed by a separate subsystem operating upon the translation model's native output, Le teaches that the translation model's output, prior to and absent invocation of that subsystem, contains the placeholder tokens. See Le paragraph [0021] ("Once the neural network translation model 120 has been trained, the rare word processing subsystem 130 can, for every pointer token in a target sentence emitted by the neural network translation model 120 from a source sentence, replace the pointer token according to the corresponding source word in the source sentence using the word dictionary 140"). Le further discloses that the post-processing is a discrete and separately performed step. See Le paragraph [0036] ("The system constructs the word dictionary for use in word translations in a post-processing step performed on a neural network generated machine translation including out-of-vocabulary tokens in a target sentence.") and Le paragraph [0046] and Figure 3 (showing the placeholder replacement as a discrete step 306, distinct from the model's sentence generation at step 304). Where Le's post-processing subsystem is not invoked, the translation output is delivered in its placeholder-bearing form.)
It would have been obvious to one of ordinary skill in the art at the time the invention was made before the effective filing date of the claimed invention to incorporate Le's separate post-processing subsystem architecture into the translation pipeline of Roziere, and to condition invocation of that subsystem upon an instruction. The motivation to do so arises from a circumstance acknowledged in Applicant's own disclosure: in some cases the placeholder-bearing output is the output the user requires, as where the masked character string is written in one natural language and is to be edited or corrected in another before the original strings are restored. See Specification paragraph [0092]. Conditioning execution of an existing post-processing step upon a supplied instruction is a routine programming choice well within the level of ordinary skill, and yields the predictable result of affording control over whether placeholders are retained in the output.
With respect to claim 5 (Currently Amended), Roziere is silent to disclose wherein the processor is configured to perform post-processing of returning the placeholders to an original state and output the second source code for which post-processing is performed upon determining there is an instruction for post-processing. Roziere's pipeline does not include a component that, at inference time, returns placeholder tokens appearing in the translated output to their original character strings, nor does Roziere describe outputting a post-processed result conditioned upon an instruction. However, in an analogous art, Le teaches:
wherein the processor is configured to perform post-processing of returning the placeholders to an original state and output the second source code for which post-processing is performed upon determining there is an instruction for post-processing (Le's rare word processing subsystem 130 operates upon the placeholder-bearing output of the neural network translation model and replaces each placeholder with its original value by reference to the word dictionary 140. See Le paragraphs [0021], [0023] ("replaces the pointer token in the target sentence with the corresponding source word from the source sentence"). The word dictionary 140 "maps words in the source language to translations of the words into the target language" and is constructed in conjunction with model training (Le paragraphs [0022], [0036]). The output of the subsystem is the final, post-processed result. See Le paragraph [0046] and Figure 3, step 306 ("The system replaces any pointer tokens in the target language sentence to generate the final translation for the source language sentence"). The conditional aspect is met by the same architectural separation set forth with respect to claim 4: the rare word processing subsystem 130 is a separately invoked component, executed when the replacement step is performed.) It would have been obvious to one of ordinary skill in the art at the time the invention was made before the effective filing date of the claimed invention to combine Le's rare word processing subsystem and associated word dictionary with the masking and training pipeline of Roziere. Roziere's masking procedure replaces each distinct identifier with its own distinct placeholder and necessarily creates a per-file mapping between placeholders and original identifiers, which Roziere expressly records as a dictionary of masked identifier names (Roziere § 4.3). Roziere is silent as to how placeholder tokens surviving into the translated output are to be handled at inference time. Le supplies the mechanism that Roziere's pipeline lacks, namely maintaining the placeholder-to-original mapping as a dictionary and using that dictionary at inference to replace each placeholder in the output with its corresponding original value. The motivation to combine is direct: a model that masks identifiers during training must, if its output is to be usable as source code, ultimately restore those identifiers in the translated output, as Applicant's own disclosure confirms at Specification paragraphs [0090]-[0094]. The combination yields the predictable result of a translated source file in which the placeholders have been returned to their original character strings.
With respect to claim 6 (Currently Amended), Roziere teaches wherein the processor is configured to: perform a process of converting the first source code masked with the placeholders into preliminary learning data, and create the translation model in which features of the first programming language and the second programming language are learned based on the preliminary learning data (Roziere discloses a multi-step data-preparation process that converts the masked source code into the corpus consumed during pre-training, and discloses a pre-training stage in which features of both programming languages are learned from that corpus.
As to performing a process of converting the masked source code into preliminary learning data, Roziere § 4.3, under the heading Training dataset, describes a sequence of operations applied to the masked files to produce the pre-training corpus: de-duplication and split assignment ("Following Lopes et al. [2017] and Allamanis [2019], we remove duplicate files. We also ensure that each fork belongs to the same split as its source repository."), masking and dictionary creation ("We obfuscate each file and create the corresponding dictionary of masked identifier names, resulting in a parallel (obfuscated file - dictionary) dataset of 19 GB for Python and 26 GB for Java."), and subword tokenization and length-based selection ("For comparison purposes, we apply either the BPE codes used by Roziere et al. [2020] or by Feng et al. [2020]. In practice, we train only on files containing less than 2000 tokens, which corresponds to more than 90% and 80% of the Java and Python files respectively."). See also Roziere § 4.3 ("We show some statistics about this dataset in Table 3 in the appendix."). The masked source code is thereby converted, through de-duplication, dictionary creation, byte-pair-encoding tokenization, and length filtering, into the corpus on which pre-training is performed. As to creating the model in which features of the first programming language and the second programming language are learned based on that preliminary learning data, Roziere § 4.3, under the heading Training details, discloses that both languages are learned from that corpus ("During DOBF training, we alternate between batches of Java and Python composed of 3000 tokens per GPU."), and further discloses initializing that training from a model pre-trained on both languages jointly ("We try different initialization schemes: training from scratch and with a Python-Java MLM model following Roziere et al. [2020]."). Roziere further discloses that the model so produced is the model carried forward to create the translation model ("In all cases, we initialize the encoders of these models with the encoder of DOBF and fine-tune all parameters."). See also Roziere Table 2 (reporting the resulting Python to Java and Java to Python translation accuracy).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Baptiste Roziere et al. (“DOBF: A Deobfuscation Pre-Training Objective for Programming Languages”, hereinafter “Roziere” – previously presented) in view Le et al. (US Pub. No. 2019/0188268, hereinafter “Le” - previously presented) and further in view of Marie-Anne Lachaux et al. ("Unsupervised Translation of Programming Languages", hereinafter “Lachaux”).
With respect to claim 7 (Currently Amended), Roziere teaches wherein the processor is configured to [[perform a process of converting the first source code masked with the placeholders into main learning data, and]] create the translation model based on the main learning data (Roziere discloses a second, distinct learning stage following pre-training, in which the model produced by pre-training is further trained on the translation task and thereby becomes the translation model. See Roziere § 4.3, under the heading Fine-tuning details ("In all cases, we initialize the encoders of these models with the encoder of DOBF and fine-tune all parameters.") and ("For TransCoder, we use a learning rate of 10⁻⁴ as in Roziere et al. [2020] and we train the models for 2 day on 32 Tesla V100 GPUs."). See also Roziere § 4.2 (identifying TransCoder as the translation task on which the model is fine-tuned and evaluated) and Table 2 (reporting the resulting Python to Java and Java to Python translation accuracy obtained from the model so created).) Roziere in view of Le is silent to disclose perform a process of converting the first source code masked with the placeholders into main learning data. Roziere identifies the corpus used for its fine-tuning stage only by reference to Roziere et al. [2020], and does not itself describe the operations by which that corpus is derived from the source code. However, in an analogous art, Lachaux teaches:
wherein the processor is configured to perform a process of converting the first source code masked with the placeholders into main learning data create the translation model based on the main learning data. (Lachaux discloses deriving two distinct training corpora from the same body of source code by an express extraction process, and using the second such corpus to train the translation model. As to the existence of a separate corpus for the translation-training stage, see Lachaux § 4.1 ("We pretrain TransCoder on all source code available, and train the denoising auto-encoding and back-translation objectives on functions only."). As to the process by which that corpus is produced, see Lachaux Appendix A.3, under the heading Function extraction ("We train and evaluate our translation model on functions only. We differentiate class functions and standalone functions.") and ("Thus, all the results in this work are given for models pretrained on all available data and trained on standalone functions only."). Lachaux thereby discloses converting the source code into a second, distinct body of learning data by extracting standalone functions from it, and creating the translation model from that second body of learning data. See also Lachaux Table 3 (statistics of the extracted function dataset).) It would have been obvious to one of ordinary skill in the art at the time the invention was made before the effective filing date of the claimed invention to derive the corpus used in the fine-tuning stage of Roziere by extracting functions from the source code, as taught by Lachaux. The motivation to do so is supplied by Roziere itself, which adopts Lachaux's TransCoder as the translation task on which the model is fine-tuned and expressly directs the reader to Lachaux for the corresponding training configuration. See Roziere § 4.3 ("For TransCoder, we use a learning rate of 10⁻⁴ as in Roziere et al. [2020] and we train the models for 2 day on 32 Tesla V100 GPUs."). One of ordinary skill implementing Roziere's fine-tuning stage is therefore directed to Lachaux for the preparation of the fine-tuning corpus, and Lachaux supplies that preparation expressly at § 4.1 and Appendix A.3. Lachaux additionally articulates the benefit of the extraction step, namely that functions are short enough to fit into a single batch and permit evaluation of the model with unit tests. See Lachaux § 4.1 ("Unlike files or classes, functions are short enough to fit into a single batch, and working at function level allows for a simpler evaluation of the model with unit tests (c.f. Section 4.4)."). The combination yields the predictable result of a translation model created from a function-level corpus extracted from the source code.
Response to Arguments
Applicant's arguments filed August 5, 2026 have been fully considered. The arguments directed to 35 U.S.C. § 112(f) are persuasive in part, as set forth in the Claim Interpretation section above. The arguments directed to the prior art are not persuasive, for the reasons set forth below.
A. Applicant argues at pages 9-11 of the Remarks that "Roziere does not disclose masking both first source code and second source code to be used as learning data and then generating the translation model using both the masked first source code and second source code."
This argument is not persuasive because it is contradicted by the express text of Roziere. Roziere states, under the heading Training dataset at § 4.3, that "We obfuscate each file and create the corresponding dictionary of masked identifier names, resulting in a parallel (obfuscated file - dictionary) dataset of 19 GB for Python and 26 GB for Java." Roziere therefore masks source code written in a first programming language (Python) and masks source code written in a second programming language (Java), and the masked files of both languages constitute the training dataset. Roziere further states, under the heading Training details at § 4.3, that "During DOBF training, we alternate between batches of Java and Python composed of 3000 tokens per GPU." The masked Java source code and the masked Python source code are thus both learned by the same model. The two facts Applicant asserts to be absent from Roziere — masking of source code in both languages, and learning from the masked source code of both languages — are each expressly stated in a single paragraph of the reference.
B. Applicant argues at pages 9-11 of the Remarks that Roziere's technique "relates to deobfuscation, i.e., 'recovering the original version of the obfuscated code,'" and that training DOBF to translate obfuscated files into lists of identifier names "involves a different purpose and different techniques than in the presently claimed invention."
This argument is not persuasive for two reasons. First, an assertion that a reference is directed to a different purpose does not distinguish the claim where the reference performs the recited steps. It is well settled that a statement of intended use or purpose does not impart patentable weight where the prior art structure or method is capable of performing the recited function. Claim 1 recites masking, learning, creating a translation model, and applying that model; Roziere performs each of these acts. Second, and independently, the premise is factually incorrect: Roziere is not confined to deobfuscation. Roziere expressly evaluates and reports source-code translation between the two languages whose masked source code was learned. See Roziere Abstract ("providing relative improvements of up to 12.2% in unsupervised code translation"), § 5.2, and Table 2 (reporting computational accuracy for Python to Java and Java to Python translation). Translation between the first and second programming languages is a reported result of Roziere, not a purpose foreign to it.
C. To the extent Applicant intends to argue that Roziere's masked source code is used only for pre-training, and that the translation model is instead created at a separate fine-tuning stage, that argument would not distinguish the claims either.
First, claim 1 does not require that the translation model be created exclusively from the masked source code. Claim 1 recites learning the masked first source code, learning the masked second source code, and creating a translation model. Roziere's model is trained on the masked Java and Python source code, and that trained model is the model that is carried forward and that produces the translation results Roziere reports at § 5.2 and Table 2. Roziere's Abstract attributes the translation improvement directly to the pre-training performed on the masked source code.
Second, and dispositively, Applicant's own claims recite the identical two-stage structure. Claim 6 recites converting the masked first source code into preliminary learning data and creating the translation model in which features of the first programming language and the second programming language are learned based on that preliminary learning data. Claim 7 recites converting the masked first source code into main learning data and creating the translation model based on that main learning data. Applicant therefore claims a pipeline in which masked source code is processed into a first body of learning data used to learn language features, and thereafter into a second body of learning data from which the translation model is created. That is the same architecture Roziere employs, in which the model trained on masked Java and Python source code is subsequently fine-tuned on the translation task (Roziere §§ 4.2, 4.3, Table 2). Applicant cannot claim a two-stage pre-training and fine-tuning pipeline in claims 6 and 7 while simultaneously contending that a two-stage pre-training and fine-tuning pipeline falls outside claim 1.
Third, and in any event, the alternative ground of rejection over Roziere in view of Lachaux set forth above addresses this contention on the merits. Lachaux expressly discloses that the model pre-trained by masking tokens in source code sequences drawn from multiple programming languages is the model from which the sequence-to-sequence translation model is initialized (Lachaux §§ 3.1, 3.2). Roziere itself directs one of ordinary skill to Lachaux for precisely this mechanism (Roziere § 4.2).
D. Applicant argues at pages 9-11 of the Remarks that the dependent claims are allowable by virtue of their dependency from an allowable base claim and for the additional features each recites. Because the independent claims are not allowable for the reasons set forth above, the dependent claims are not allowable by virtue of their dependency. Applicant has not identified any particular additional feature of any dependent claim said to distinguish over the applied art, and accordingly no further response is possible. The rejections of the dependent claims are maintained and restated above to reflect the amended claim dependencies.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure:
Di Toni et al. (US Pub. No. 2024/0184555) discloses iterative code generation using neural language models, including inserting masks into a translation of a source code snippet and reprocessing the masked translation to generate infills of corrected source code.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANIBAL RIVERACRUZ whose telephone number is (571)270-1200. The examiner can normally be reached Monday-Friday 9:30 AM-6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S Sough can be reached at 5712726799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANIBAL RIVERACRUZ/Primary Examiner, Art Unit 2192
Prosecution Insights