DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, 6-9, 11-15, 17-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Clement (US 20210357762 A1; hereinafter Clement) in view of Chen (US 20240201984 A1; hereinafter Chen), and further in view of Akbari (US 20240037335 A1; hereinafter Akbari).
Regarding Independent Claim 1, Clement teaches A method for training an encoder-decoder based framework for code related tasks, the method comprising (see, e.g., Clement paragraph [0067]: "FIG. 6 illustrates a method 600 for training a neural transformer model. The method 600 is used to train … a source code domain encoder-decoder neural transformer model with attention in several model sizes"):
receiving, via a communication interface, a first training dataset of unimodal code data (see, e.g., Clement paragraph [0090]: “Source code snippets are parsed to form a concrete syntax tree from which tokens/subtokens are extracted" [i.e., a training dataset is split and used to train models using source code data (unimodal code data)] and paragraph [0104]: "The computing devices 802, 804 may include one or more processors 808, 840, one or more communication interfaces 810, 842… A communication interface 810, 842 facilitates wired or wireless communications between the computing device 802, 804 and other devices" [i.e., the communication interface is coupled to the source code domain encoder-decoder neural transformer models (see, e.g., Fig. 8)]);
encoding, by an encoder, a first training input based on a first code sequence from the first training dataset into a first code representation (see, e.g., Clement paragraph [0090]: "The fine-tuning component generates the fine-tuning training dataset using the training dataset provided in the request. The training dataset is split into input sequences for training, testing and validation subsets. The input sequences are constructed in the same manner as the training dataset for the pre-trained model. Source code snippets are parsed to form a concrete syntax tree from which tokens/subtokens are extracted. Byte-level byte-pair encoding is used to form the subtokens" [i.e., a training dataset is split into sequences for training from source code snippets that are parsed into concrete syntax tree representations (code representations) that are encoded by an encoder] and paragraph [0092]: “In the case of fine-tuning a source code domain encoder neural transformer for a specific classification task, the output layer of the pre-trained model is replaced with a classification layer while reusing all encoder blocks” [i.e., the encoder blocks are used in the fine-tuning component for encoding a training input of a code sequence]);
generating, by a decoder, a code output from the first code representation (see, e.g., Clement paragraph [0042]: "The decoder block 240 predicts each token/subtoken ti in the target language one-by-one at each time step conditioned on all previously-generated target tokens/subtokens t1, . . .t−1" [i.e., the tokens/subtokens sequences are elements of code output generated by the decoder] and paragraph [0074]: "The set of hidden representations is passed onto each decoder block. (Collectively, block 612)" [i.e., the decoder takes the hidden code representation as input to generate output results]);
computing at least one unimodal training objective based on the code output and the first code sequence (see, e.g., Clement paragraph [0066]: "A noising transformation, such as a span masking function, is then applied to each sequence that randomly masks out a subset of subtokens and the masked span of subtokens is replaced with a mask subtoken, M. The model is trained with the masked sequences to learn to reconstruct the original sequence without the masked subtokens" [i.e., the act of matching the decoder's code output to the original code sequence (the first code sequence) during a masking reconstruction computation is computing a unimodal training objective]);
pretraining the encoder and the decoder according to the at least one unimodal training objective (see, e.g., paragraph [0067]: "The pre-training component then trains a neural transformer model in each configuration in each of the model sizes with the pre-training dataset (block 504)… The method 600 is used to train a source code domain encoder neural transformer model" [i.e., the encoder and decoder are trained to perform a unimodal training objective (code-to-code)] and paragraph [0070]: "(3) for the training procedure: denoising auto-encoder, with a cross-entropy loss optimization objective;" [i.e., a cross-entropy loss optimization objective for a training procedure]);
receiving, via a communication interface (see, e.g., Clement paragraph [0104]: "The computing devices 802, 804 may include one or more processors 808, 840, one or more communication interfaces 810, 842… A communication interface 810, 842 facilitates wired or wireless communications between the computing device 802, 804 and other devices" [i.e., the communication interface is coupled to the source code domain encoder-decoder neural transformer models (see, e.g., Fig. 8)]),
encoding, by the pretrained encoder, a second training input of a second code sequence and a text from the second training dataset into a second code representation and a text representation (see, e.g., Clement paragraph [0051]: "Turning to FIG. 4, there is shown an exemplary system 400 having one or more computing devices 402…, a fine-tuning component 422 that fine-tunes a pre-trained model for a target software engineering task, one or more source code repositories 424, and one or more pre-training training datasets 426" [i.e., there are multiple training datasets used for pretraining the encoder], paragraph [0063]: "This type of encoding does not rely on knowing the underlying language making it suitable for an input sequence of text strings that contain source code and/or natural language text" [i.e., the pretrained encoder can process code and test input sequences] and paragraph [0073]: "For the encoder neural transformer model, the multi-head self attention layer takes the context tensor as input and passes it through the multiple layers of multi-head attention, layer normalization and feed forward neural network of each encoder block to finally produce a set of hidden representations" [i.e., the encoder processes the concatenated code and text inputs (together or separately) to produce vector outputs of second code/text representations]);
generating, by the pretrained decoder, a training output from the second code representation and the text representation (see, e.g., paragraph [0024]: "The decoder-only neural transformer model is an auto-regressive model that produces an output one element at a time based on the outputs of previous time steps", paragraph [0048]: "The encoder-decoder multi-head attention layer 302 receives queries from the previous decoder layer 242 and the memory keys and values 217 from the output of the encoder block 212" [i.e., the pretrained decoder uses the task representation (encoded code and text representations) to perform sequence generation], paragraph [0063]: "This type of encoding does not rely on knowing the underlying language making it suitable for an input sequence of text strings that contain source code and/or natural language text" [i.e., the pretrained encoder can process code and test input sequences and transmits those values to the decoder] and paragraph [0074]: "The set of hidden representations is passed onto each decoder block. (Collectively, block 612)" [i.e., the hidden representations of each sequence in each batch (see, e.g., Fig. 6) serves as having second code and text representations that are received by the decoder which generates a training output]);
Although Clement substantially teaches the claimed invention, Clement is not relied upon to explicitly teach the limitations a second training dataset of bimodal code-text pair data;
computing at least one bimodal training objective based on the training output and the second training input;
and training the pretrained encoder and the pretrained decoder according to the at least one bimodal training objective.
In the same field, analogous art Chen teaches a second training dataset of bimodal code-text pair data (see, e.g., Chen paragraph [0039]: “a statement list corresponding to the annotations is obtained through position information and data flow information, and an <annotation, target code> pair is obtained” [i.e., code-text pairs of data are utilized] and paragraph [0051]: "Preferably, step SS6 specifically includes: the dataset obtained in step SS5 is randomly divided in a proportion of 80%: 100%: 10%, with the training set including 336457 pieces of data… meanwhile, the entire model is of an encoder-decoder structure; an encoder CodeBERT uses bimodal data instances (natural language NL and programming language PL) as an input for training in a pretraining stage, and a decoder employs 6 Transformer decoding layers to stack to construct the entire network" [i.e., CodeBERT uses a bimodal training objective of capturing both the syntax of code and the semantics of natural language descriptions for split (multiple) training datasets]);
computing at least one bimodal training objective based on the training output and the second training input (see, e.g., Chen paragraphs [0051-0052]: "an encoder CodeBERT uses bimodal data instances (natural language NL and programming language PL) as an input for training in a pretraining stage, and a decoder employs 6 Transformer decoding layers to stack to construct the entire network. The network structure is as shown in FIG. 3… training is performed by setting a target code length to be 100, a context code length to be 300, and an annotation length to be 30" [i.e., CodeBERT uses a bimodal training objective of capturing both the syntax of code and the semantics of natural language descriptions, based on a training output and second training input in the context of a Seq2Seq1 function (see, e.g., Fig. 3)]);
and training the pretrained encoder and the pretrained decoder according to the at least one bimodal training objective (see, e.g., Chen paragraph [0037]: " step SS7: the training set in division in step SS6 is enabled to be used for model training, and evaluation is performed on the validation set to obtain a model with the best effect on the validation set as a target model" and paragraph [0051]: " an encoder CodeBERT uses bimodal data instances (natural language NL and programming language PL) as an input for training in a pretraining stage, and a decoder employs 6 Transformer decoding layers to stack to construct the entire network" [i.e., the pretrained encoder (CodeBERT) and pretrained decoder are combined into the entire network in step SS7, the bimodal pairs from a second training dataset update weights of both encoder and decoder models based on the bimodal training objective (loss) in a fine-tuning process]).
Clement and Chen are analogous art because they are both directed to syntactic representation learning (see, e.g., Clement, paragraph [0021], Chen, paragraph [0005]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Clement to incorporate bimodal code-text pair data for a bimodal training objective of Chen. Doing so would have allowed Clement to use Chen’s method in order to “build a model by employing an encoder-decoder network in the field of deep learning, and use a pretrained language model CodeBERT as an encoder to improve the quality of annotation generation, assist developers in code understanding and reading, and improve code maintainability”, as suggested by Chen (see, e.g., Chen, paragraph [0005]).
Clement in view of Chen do not specifically disclose an encoder comprising a first encoder module and a second encoder module, a decoder connected to the first and second encoder modules; encoding, by the first encoder module, the text into the text representation; and encoding, by the second encoder, the second code sequence into the second code representation in parallel.
However Akbari teaches encoder comprising a first encoder module and a second encoder module, a decoder connected to the first and second encoder modules (see, e.g., Akbari, paragraph [0104], [0106], figure 2A, figure discloses encoder having two modules and they are connected to the decoder),
and encoding, by the first encoder module, the text into the text representation (see, e.g., Akbari paragraph [0106]: “The encoder 206 includes a text encoder 210 to process natural language information 202 to generate word embeddings” [i.e., the first encoder module is the text encoder 210 that converts text to word embeddings]);
and encoding, by the second encoder, the second code sequence into the second code representation in parallel (see, e.g., Akbari paragraph [0091]: “In some examples, a model trained in bi-modal generation of NL and NA can be deployed to perform tasks such as processing NL to perform generative inference tasks relating to NA and/or NL, such as NA answer generation in response to NL questions, text-based NA generation, architecture-based NL generation (e.g., NA captioning), NA translation, multi-modal NA translation assisted by NL information, NA completion, NA repair, and/or multi-modal NA completion or repair assisted by NL information” [i.e., NA tasks such as generation, captioning, completion and repair are tasks that are associated with text-to-code generation, code repair, and auto-completion prediction, functioning as code sequence input data], paragraph [0113]: “The cross transformer encoder 240 enables joint learning of NL (e.g., textual) and NA (i.e., architectural) embeddings, in this example represented as word embeddings and graph embeddings respectively, and sharing of learning signals between both modalities” [i.e., encoders 210 and 220 share their embeddings (parameters) into the cross transformer encoder for joint/parallel learning enabling the function of sharing same parameters operating in parallel] and paragraph [0140]: “In operation, the text encoder 210 processes the question 402 and the architecture encoder 220 processes the NA data sample 404. The NL encoding (e.g., word embeddings) and NA encoding (e.g., graph encoding)” [i.e., the second code sequence (NA data sample) is represented by the graph encoding after being processed by the Architecture encoder 220 (second encoder module), encoded in parallel with the text encoder 210 (first encoder module)]).
Clement, Chen and Akbari are analogous art because they are each directed to syntactic representation learning (see, e.g., Clement, paragraph [0021], Chen, paragraph [0005], Akbari, paragraph [0013]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Clement in view of Chen to incorporate the shared parameters between encoder modules and using encoder modules for parallel encoding text representations and code sequences of Akbari. Doing so would have allowed Clement in view of Chen to use Akbari’s method in order to create a “system that can perform time- and cost-efficient generative inference in response to a simple natural language query, with the potential to significantly improve usability, user engagement, user exploration, and user experience, especially for beginner and intermediate users and developers in the field of machine learning”, as suggested by Akbari (see, e.g., Akbari, paragraph [0027]).
Regarding claim 2, Clement further teaches wherein the first training input is generated by randomly replacing a portion of tokens in the first code sequence into indexed sentinel tokens (see, e.g., Clement paragraph [0058]: “The pre-training component generates a pre-training dataset from a diverse corpus of unlabeled source code programs or files” [i.e., the first code sequence is the source code input] and paragraph [0066]: “A noising transformation, such as a span masking function, is then applied to each sequence that randomly masks out a subset of subtokens and the masked span of subtokens is replaced with a mask subtoken, M" [i.e., the span masking process randomly replaces a portion of tokens with a single mask subtoken, M, functioning as an indexed sentinel token]),
and wherein the at least one unimodal training objective is computed by comparing a reconstructed code sequence, generated by the decoder with the first code sequence (see, e.g., Clement paragraph [0066]: "A noising transformation, such as a span masking function, is then applied to each sequence that randomly masks out a subset of subtokens and the masked span of subtokens is replaced with a mask subtoken, M. The model is trained with the masked sequences to learn to reconstruct the original sequence without the masked subtokens" [i.e., the act of matching the decoder's code output to the original code sequence (the first code sequence) during a masking reconstruction computation is computing a unimodal training objective], paragraph [0069]: “The model architecture, training procedure, data normalization and vocabulary encoding procedures are hyperparameters that are tailored to meet a particular objective” [i.e., the particular (singular) objective is unimodal] and paragraph [0070]: “for the training procedure: denoising auto-encoder, with a cross-entropy loss optimization objective” [i.e., the denoising autoencoder has a unimodal training objective of producing a cross entropy loss optimization])
Regarding claim 4, Chen teaches wherein the at least one bimodal training objective is computed based on a batch of code-text pairs from the second training dataset by (see, e.g., Chen paragraph [0039]: “a statement list corresponding to the annotations is obtained through position information and data flow information, and an <annotation, target code> pair is obtained” and paragraph [0051]: "meanwhile, the entire model is of an encoder-decoder structure; an encoder CodeBERT uses bimodal data instances (natural language NL and programming language PL) as an input for training in a pretraining stage, and a decoder employs 6 Transformer decoding layers to stack to construct the entire network" [i.e., CodeBERT uses a bimodal training objective of capturing both the syntax of code and the semantics of natural language descriptions]):
generating, by the encoder, a set of code representations and a set of text representations from the batch of code-text pairs (see, e.g., Chen paragraph [0051]: "the entire model is of an encoder-decoder structure; an encoder CodeBERT uses bimodal data instances (natural language NL and programming language PL) as an input for training in a pretraining stage, and a decoder employs 6 Transformer decoding layers to stack to construct the entire network" [i.e., the encoder (CodeBERT) processes the code/PL input to generate the required code representation]);
Although Clement in view of Chen substantially teaches the claimed invention, Clement in view of Chen is not relied upon to explicitly teach computing a set of code-to-text similarities and a set of text-to-code similarities between the set of code representations and the set of text representations;
computing a contrastive loss by comparing the set of code-to-text similarities and the set of text-to-code similarities with ground-truth one-hot similarities.
In the same field, analogous art Akbari teaches computing a set of code-to-text similarities and a set of text-to-code similarities between the set of code representations and the set of text representations (see, e.g., Akbari paragraph [0124]: "Optionally, at step 303, the training dataset is used to pre-train the encoder 206 portion of the generative bi-modal model 200 using supervised learning… a similarity evaluator 246 is used for processing encoded representations to determine a similarity measure using a cosine similarity metric" [i.e., a similarity metric is calculated between NL and NA data (textual and code representations)]);
computing a contrastive loss by comparing the set of code-to-text similarities and the set of text-to-code similarities with ground-truth one-hot similarities (see, e.g., Akbari paragraph [0125]: “The use of both positive and negative training data samples enables the encoder 206 to learn both similarities and dissimilarities between NA and NL information. In other words, during the training procedure, the encoder 206 learns to maximize the similarity measure (e.g., cosine similarity) generated between the neural network architecture information 204 and the natural language information 202 of the positive training samples, and to minimize the similarity measure generated between the neural network architecture information 204 and the natural language information 202 of the negative training samples. In some embodiments, a loss function may be computed based on the similarity measure and back-propagated through the encoder 206 to adjust the values of the learnable parameters thereof, for example using gradient descent” [i.e., the positive/negative data samples serve as ground-truth one-hot similarities (match or mismatch representing 1 and 0 values in the context of backpropagation) and the act of maximizing/minimizing the calculated similarity loss function based on this ground-truth serves as the contrastive loss]).
Regarding claim 6, Clement further teaches wherein the at least one bimodal training objective is computed by: generating, by the decoder, a predicted code sequence based on the text representation (see, e.g., Clement paragraph [0063]: “The byte-level subwords are generated using the Byte Pair Encoding (BPE) algorithm… an input sequence of text strings that contain source code and/or natural language text… which are vector representations of a source code fragment or natural language text” [i.e., the input text representation could be either language text or source code sequences (bimodal)], paragraph [0074]: “the output of each encoder block is passed onto the next encoder block with the output of the last encoder block producing the set of hidden representations. The set of hidden representations is passed onto each decoder block” and paragraph [0075]: “The decoder blocks of the neural transformer model take a shifted sequence of an output embedding as input” [i.e., The encoder processes the input text representation (source code sequences) and the decoder generates the output code sequence based on the encoders hidden representations]);
and computing a text-to-code loss by comparing the predicted code sequence and the second code sequence (see, e.g., Clement paragraph [0063]: “The byte-level subwords are generated using the Byte Pair Encoding (BPE) algorithm… an input sequence of text strings that contain source code and/or natural language text… which are vector representations of a source code fragment or natural language text” [i.e., the input text representation could be either language text or source code sequences], paragraph [0067]: “The method 600 is used to train a source code domain encoder neural transformer model with attention in several model sizes, a source code domain decoder neural transformer model with attention in several model sizes, and a source code domain encoder-decoder neural transformer model with attention in several model sizes” and paragraph [0076]: “the feed forward neural networks in the encoder blocks and the decoder blocks are trained iteratively, making multiple passes over the training dataset before converging to a minimum. Each training iteration includes forward propagation, loss calculation, backpropagation steps followed by updating the weights by calculating the weight gradients. The loss function estimates the loss or error which is used to compare how good or bad the predicted results are. In one aspect, a categorical cross-entropy loss function is used” [i.e., the encoder-decoder model uses a cross-entropy loss that compares a prediction to the ground truth code sequence, functioning as a text-to-code loss inputting language text to predict a code sequence]).
Regarding claim 7, Clement further teaches wherein the at least one bimodal training objective is computed by: generating, by the decoder, a predicted text based on the second code representation (see, e.g., Clement paragraph [0066]: "A noising transformation, such as a span masking function, is then applied to each sequence that randomly masks out a subset of subtokens and the masked span of subtokens is replaced with a mask subtoken, M. The model is trained with the masked sequences to learn to reconstruct the original sequence without the masked subtokens" [i.e., the decoder performs the reconstruction / denoising task, which is a sequence-generation task where it generates the missing subtokens (text/code) to reconstruct the original sequence into predicted text], paragraph [0075]: “The decoder blocks of the neural transformer model take a shifted sequence of an output embedding as input… The masking combined with the output embeddings shifted by one position ensures that the predictions to position T depend only on the known outputs at positions less than T… serving as the query for encoder-decoder attention, where the key and value pairs for the attention are the outputs of encoder” [i.e., the decoder outputs the predicted sequence token-by-token by the autoregressive generation process] and paragraph [0083]: "a sequence-to-sequence translation or machine translation task is often performed using an encoder-decoder neural transformer model");
and computing a code-to-text loss by comparing the predicted text and the text (see, e.g., Clement paragraph [0063]: "This type of encoding does not rely on knowing the underlying language making it suitable for an input sequence of text strings that contain source code and/or natural language text" [i.e. bimodal input/output is necessary for the code-to-text loss], paragraph [0070]: "(3) for the training procedure: denoising auto-encoder, with a cross-entropy loss optimization objective" [i.e., the denoising auto-encoder objective involves comparing the reconstructed output (the final predicted text) to the original input (the ground truth text/code) to calculate the loss serving as the code-to-text loss] and paragraph [0076]: "The loss function estimates the loss or error which is used to compare how good or bad the predicted results are. In one aspect, a categorical cross-entropy loss function is used" [i.e., cross-entropy loss is used for comparing a predicted token sequence against a ground truth sequence]).
Regarding Claim 8, Clement further teaches further comprising: jointly training the pretrained encoder and the pretrained decoder according to a sum of multiple bimodal training objectives (see, e.g., Clement paragraph [0058]: "The pre-training component generates a pre-training dataset from a diverse corpus of unlabeled source code programs or files. In some aspects, the pre-training dataset may also include natural language text that pertains to a source code file such as source code summaries which describe the operation of a source code construct" [i.e., bi-modal data (code and text) is used in the training objectives], paragraph [0064]: "It should be noted that in bi-modal training, a model having been trained on English language text may be reused to train on source code. In this situation, the source code training would have to augment the vocabulary with indent and dedent tokens to account for large spans of white spaces that is used in some programming language, such as Python. A further discussion of this issue is described in more detail below", paragraph [0067]: "The method 600 is used to train a source code domain encoder neural transformer model with attention in several model sizes, a source code domain decoder neural transformer model with attention in several model sizes, and a source code domain encoder-decoder neural transformer model with attention in several model sizes" [i.e., training the encoder-decoder neural transformer model requires joint training of the pretrained encoder and decoder models as a single unit (see, e.g., Fig. 4)], paragraph [0076]: "The loss function estimates the loss or error which is used to compare how good or bad the predicted results are. In one aspect, a categorical cross-entropy loss function is used. Once the loss is calculated, it is propagated backwards to the hidden layer that contributed directly to the output” and paragraph [0086]: "The total training loss is a linear combination of the distillation loss L1, the loss for a particular pretraining task L2 (cross-entropy loss for autoregressive language modeling task in the case of decoder-only model), and the cosine embedding loss L3 which tends to align the directions of the student and teacher output hidden states" [i.e., training is based on a combination (linear combination) of multiple objectives serving as multiple bimodal training objectives in the context of bi-modal training]).
Regarding Claim 9, Akbari teaches wherein the first encoder module and the second encoder module share the same parameters (see, e.g., Akbari, paragraph [0104]: “The generative bi-modal model 200 includes an encoder 206 for receiving input information 201 (which may include NL information 202 and/or NA information 204)”, paragraph [0106]: “FIG. 2B illustrates the functional modules of an example encoder 206 of the generative bi-modal model 200. The encoder 206 includes a text encoder 210 to process natural language information 202 to generate word embeddings, a neural network architecture encoder 220 to process neural network architecture information 204 to generate graph encodings, and a cross transformer encoder 240 to process word embeddings and graph encodings to generate joint embeddings 242” [i.e., the text encoder 210 and the NN encoder 220 function as first and second encoder modules], paragraph [0128]: “In some examples, the generative bi-modal model 200 may be trained to generate NL information 202 and NA information 204 in parallel, for example by providing training data samples having both NL and NA information as input information 201, and computing the loss based on both the NL and NA information of the input information 203” [i.e., the encoder modules 210 and 220 process NL information 202 and NA information 204 in parallel] and paragraph[0113]: “The cross transformer encoder 240 enables joint learning of NL (e.g., textual) and NA (i.e., architectural) embeddings, in this example represented as word embeddings and graph embeddings respectively, and sharing of learning signals between both modalities” [i.e., encoders 210 and 220 share their embeddings (parameters) into the cross transformer encoder for joint/parallel learning enabling the function of sharing same parameters operating in parallel]).
Regarding Claim 11, Clement further teaches generating, by the trained encoder only, a code-related task output in response to a code-related task input according to a specific code-related task (see, e.g., Clement paragraph [0083]: "For example, a classification task is often performed using an encoder neural transformer model", paragraph [0092]: "In the case of fine-tuning a source code domain encoder neural transformer for a specific classification task, the output layer of the pre-trained model is replaced with a classification layer while reusing all encoder blocks" [i.e., the classification layer is the mechanism for generating a specific task output] and paragraph [0093]: "For example, an exemplary software engineering classification task is a software bug classification task where an encoder neural transformer model can identify whether a code snippet is likely to have a particular type of source code bug" [i.e., the code snippet bug identification is a code-task and the software bug classification is the code-related task output for a specific code-related task]).
Regarding Claim 12, Clement further teaches generating, by the trained decoder only, a code-related task output in response to a code-related task input according to a specific code-related task (see, e.g., Clement paragraph [0042]: "The decoder block 240 predicts each token/subtoken ti in the target language one-by-one at each time step conditioned on all previously-generated target tokens/subtokens t1, . . .t−1", paragraph [0075]: "The decoder blocks of the neural transformer model take a shifted sequence of an output embedding as input" [i.e., a sequence of an output embedding (from code-related data) is used as input according to a specific task], paragraph [0083]: "an auto-regressive task is often performed using a decoder neural transformer model" [i.e., the decoder architecture is linked to an auto-regressive task (such as code-completion) generating an output sequence] and paragraph [0097]: "In the case of a source code domain decoder neural transformer model, the architecture of the pre-trained mode does not need to be altered to be fine-tuned on auto-regressive software engineering tasks" [i.e., the decoder performs auto-regressive software engineering tasks (such as code prediction/completion)]).
Regarding Claim 13, Clement further teaches encoding, by the trained encoder, a code-related task input into a task representation (see, e.g., Clement paragraph [0074]: "the first encoder block of the neural transformer model takes the context tensor as input and passes it through the multiple layers of multi-head attention, layer normalization and feed-forward neural network to finally produce a set of hidden representations" [i.e. the context tensor (derived from the tokenized input) is a code-related task input and the set of hidden representations (final output of the encoder) is a task representation]);
and generating, by the trained decoder, a code-related task output from the task representation according to a specific code-related task (see, e.g., Clement paragraph [0048]: " The encoder-decoder multi-head attention layer 302 receives queries from the previous decoder layer 242 and the memory keys and values 217 from the output of the encoder block 212" [i.e., the decoder takes a task representation (encoders output/memory) as input and generates a sequential output for a code related task], paragraph [0074]: "The set of hidden representations is passed onto each decoder block. (Collectively, block 612)", paragraph [0083]: "… a sequence-to-sequence translation or machine translation task is often performed using an encoder-decoder neural transformer model. (Collectively, block 702)" [i.e., translation tasks are code-related tasks that are done by the encoder-decoder mechanism] and paragraph [0093]: "For example, an exemplary software engineering classification task is a software bug classification task where an encoder neural transformer model can identify whether a code snippet is likely to have a particular type of source code bug" [i.e., the code snippet bug identification is a code-task and the software bug classification is the code-related task output for a specific code-related task]).
Regarding Independent Claim 14, claim is a system claim having similar limitations as of method claim 1 above and therefore it is rejected under the same rational. The additional limitations of the claim are addressed below.
Clement teaches A system for training an encoder-decoder based framework for code related tasks, the system comprising (see, e.g., Clement paragraph [0004]:):
a memory that stores an encoder, a decoder and a plurality of processor executable instructions (see, e.g., Clement paragraph [0114]);
a communication interface that receives a first training dataset of unimodal code data (see, e.g., Clement paragraph [0090] and paragraph [0104], figure 8);
and one or more hardware processors that read and execute the plurality of processor- executable instructions from the memory to perform operations comprising (see, e.g., Clement paragraph2 [0114-0115]: “):
encoding, by the encoder, a first training input based on a first code sequence from the first training dataset into a first code representation (par. 90, 92, 104 and figure 8);
Regarding Independent Claim 17, claim is a program product claim having similar limitations as of method claim 1 above and therefore it is rejected under the same rational. The additional limitations of the claim are addressed below.
Clement teaches A non-transitory machine-readable medium comprising a plurality of machine-executable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform operations comprising (see, e.g., Clement paragraph [0104]: “The computing devices 802, 804 may include one or more processors 808, 840, one or more communication interfaces 810, 842” and paragraphs [0105-0106]: “A memory device 816, 848 may be any non-transitory computer-readable storage media that may store executable procedures, applications, and data. The computer-readable storage media does not pertain to propagated signals, such as modulated data signals transmitted through a carrier wave… The memory device 848 of computing device 804 may contain instructions, components, and data. A component is a software program that performs a specific function and is otherwise known as a module, program, component, and/or application. The memory device 848 may include an operating system 850, a pre-training component 852, a fine-tuning component 854, one or more source code domain encoder neural transformer models 856, one or more source code domain” [i.e., the non-transitory computer-readable storage media coupled to the processors contains instructions that are executed by processors to perform operations]):
Regarding Claims 15 and 18, they are system and program product claims having similar limitations as of method claim 9 above. Therefore they are rejected under the same rational as of claim 9 above.
Regarding Claim 20, as discussed above, Clement further teaches wherein the operations further comprise: generating, by the trained encoder only, the trained decoder only, or both the trained encoder and decoder (see, e.g., Clement paragraph [0042]: "The decoder block 240 predicts each token/subtoken ti in the target language one-by-one at each time step conditioned on all previously-generated target tokens/subtokens t1, . . .t−1", paragraph [0075]: "The decoder blocks of the neural transformer model take a shifted sequence of an output embedding as input" [i.e., a sequence of an output embedding (from code-related data) is used as input according to a specific task], paragraph [0097]: "In the case of a source code domain decoder neural transformer model, the architecture of the pre-trained mode does not need to be altered to be fine-tuned on auto-regressive software engineering tasks" [i.e., the decoder performs auto-regressive software engineering tasks (such as code prediction/completion)] and paragraph [0083]: "For example, a classification task is often performed using an encoder neural transformer model", paragraph [0092]: "In the case of fine-tuning a source code domain encoder neural transformer for a specific classification task, the output layer of the pre-trained model is replaced with a classification layer while reusing all encoder blocks" [i.e., the classification layer is the mechanism for generating a specific task output]),
a code-related task output in response to a code-related task input according to a specific code-related task (see, e.g., Clement paragraph [0083]: "an auto-regressive task is often performed using a decoder neural transformer model" [i.e., the decoder architecture is linked to an auto-regressive task (such as code-completion) generating an output sequence] paragraph [0093]: "For example, an exemplary software engineering classification task is a software bug classification task where an encoder neural transformer model can identify whether a code snippet is likely to have a particular type of source code bug" [i.e., the code snippet bug identification is a code-task and the software bug classification is the code-related task output for a specific code-related task]).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Clement, Chen in view of Akbari and further in view of Allamanis (US 20220374208 A1; hereinafter Allamanis).
Regarding claim 3, as discussed above, Clement in view of Chen and in view of Akbari teaches the method of claim 1.
Clement further teaches wherein the first training input is generated by (see, e.g., Clement paragraph [0067]: “The method 600 is used to train a source code domain encoder neural transformer model with attention”):
randomly selecting a pivot location in the first code sequence (see, e.g., Clement paragraph [0066]: A noising transformation, such as a span masking function, is then applied to each sequence that randomly masks out a subset of subtokens… The number of text spans and the span lengths are randomly generated and each span is replaced with a single mask subtoken” [i.e., the random masking process identifies the specific subtoken location to be reconstructed, and a single mask token replacing the span acts as a pivot location for the reconstruction task]);
Clement in view of Chen and in view of Akbari do not disclose truncating a first portion of the first code sequence before the pivot location as the first training input;
and wherein the at least one unimodal training objective is computed by comparing a predicted remaining portion of code by the decoder with a second portion of the first code sequence after the pivot location.
However Allamanis teaches and truncating a first portion of the first code sequence before the pivot location as the first training input (see, e.g., Allamanis paragraph [0059]: “The training dataset generation engine extracts partial code states from the concrete syntax tree. A partial code state represents a linearization of the leaves of a partially-expanded syntax tree. A partial code state contains at least one non-terminal symbol that has yet to be expanded“, paragraph [0066]: “for each partial code state, an input tuple is formed consisting of the partial code state, a non-terminal expansion index” [i.e., the partial code state (first portion) of the code is formed by traversing the syntax tree up to a non-terminal expansion index (pivot) which effectively truncates the sequence by stopping before the unexpanded non-terminal symbol] and paragraph [0082]: “The decoder blocks of the neural transformer models take a shifted sequence of an output embedding as input. The masking in the masked multi-head attention layer is used to prevent positions from attending to subsequent positions in the future. The masking combined with the output embeddings shifted by one position ensures that the predictions to position T depend only on the known outputs at positions less than T” [i.e., preventing attention to subsequent tokens functions as truncation, and the preceding context is the first portion to predict the next token at a pivot position]);
and wherein the at least one unimodal training objective is computed by comparing a predicted remaining portion of code by the decoder with a second portion of the first code sequence after the pivot location (see, e.g., Allamanis paragraph [0056]: “Referring to FIG. 3, there is shown an exemplary method 300 of the code completion system. The training dataset generation engine generates a training dataset of tuples which is used to train the neural transformer model to learn to predict the expansion of a non-terminal symbol”, paragraph [0066]: “for each partial code state, an input tuple is formed consisting of the partial code state, a non-terminal expansion index, and a true non-terminal expansion”, paragraph [0083]: “The loss function estimates the loss or error which is used to compare how good or bad the predicted results are. In one aspect, a categorical cross-entropy loss function is used” [i.e., the unimodal training objective (cross-entropy loss) of code sequences, is calculated by comparing the model's predicted non-terminal expansion (predicted remaining portion) against the true non-terminal expansion (second portion) contained within the input tuple]).
Clement, Chen, Akbari and Allamanis are analogous art because they are each directed to syntactic representation learning (see, e.g., Clement, paragraph [0021], Chen, paragraph [0005], Allamanis, paragraph [0037]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Clement, Chen in view of Akbari to incorporate the truncation of a code sequence before a pivot point at a first training input and a unimodal training objective computed by comparing portions of a code sequence and a predicted portion of code by a decoder of Allamanis. Doing so would have allowed Clement, Chen in view of Akbari to use Allamanis’ method in order to “find the most likely candidate to complete a partially-formed source code snippet by performing the most likely expansions predicted by the neural transformer model with attention”, as suggested by Allamanis (see, e.g., Allamanis, paragraph [0007]).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Clement, Chen, Akbari, in view of Hu (US 12079106 B1; hereinafter Hu), and further in view of Zhao (US 20210133535 A1; hereinafter Zhao).
Regarding claim 5, as discussed above, Clement, Chen in view of Akbari teaches the method of claim 1.
Akbari teaches a prediction on whether the second code sequence and the text are a matching pair (see, e.g., Akbari paragraph [0063]: “The byte-level subwords are generated using the Byte Pair Encoding (BPE) algorithm… an input sequence of text strings that contain source code and/or natural language text… which are vector representations of a source code fragment or natural language text” [i.e., the input text representation could be either language text or source code sequences] and paragraph [0124]: "a similarity evaluator 246 is used for processing encoded representations to determine a similarity measure using a cosine similarity metric" [i.e., the similarity measure calculated between the code and the text representations is the prediction of whether they are a matching pair]);
Clement, Chen in view of Akbari do not specifically disclose wherein the at least one bimodal training objective is computed by: generating, by a classification head of the decoder, and computing a matching loss by comparing the prediction and a ground-truth one-hot label corresponding to the second code sequence and the text
However, Hu teaches wherein the at least one bimodal training objective is computed by: generating, by a classification head of the decoder (see, e.g., Hu Col. 2 lines 14-21: "Detailed herein are embodiments of a model that finds corrupt code and repairs it using a neural-symbolic edit-based architecture to learn code representations (also called NSEDIT). NSEDIT proposes an innovative way of addressing the bug fix problem as a sequence of text edits based on an encoder/decoder architecture such as the transformer architecture such as using aspects of a bidirectional encoder representations from transformers (BERT) mode" [i.e., for code generation, the model incorporates BERT which encompasses a bi-modal training objective] and Col. 15 lines 16-24: "the reranking problem is formulated as a classification problem and train the reranker to identify the correct sequence among the K candidates. In some embodiments, the reranker is a Siamese model. Cross entropy loss is used to train the reranker. FIG. 10 illustrates embodiments of a reranker architecture. As shown, buggy code 1001 and edited code 1003 (e.g., [CLS]<buggy>[SEP]<edited>[SEP]) is input into the decoder 521. The reranker 1007 then computes a rerank score" [i.e., the reranker functions as a classification head of the decoder 521, to produce a match/mismatch prediction score]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Clement, Chen in view of Akbari to incorporate the classification head of a decoder generating a bimodal training objective of Hu. Doing so would have allowed Clement, Chen in view of Akbari to use Hu’s method in order to create a system that “finds corrupt code and repairs it using a neural-symbolic edit-based architecture to learn code representations”, as suggested by Hu (see, e.g., Hu, Col. 2 lines 15-16).
Although Clement, Chen, Akbari in view of Hu do not specifically disclose computing a matching loss by comparing the prediction and a ground-truth one-hot label corresponding to the second code sequence and the text
In the same field, analogous art Zhao teaches computing a matching loss by comparing the prediction and a ground-truth one-hot label corresponding to the second code sequence and the text (see, e.g., Zhao paragraph [0036]: "The optimization objective is to minimize the cross entropy between expected outputs and the actual final output 290 of PSDP 200" [i.e., the use of cross-entropy loss on the final prediction is the functional equivalent of matching loss, the mathematical function of cross-entropy loss inherently includes and requires the comparison of a model's predicted probability distribution against a ground-truth label, which is formatted as a one-hot encoded vector in the context of machine learning] and paragraph [0044]: “and the output is evaluated versus ground truth sequences using correlation techniques such as Bland-Altman method and the Spearman's rank correlation coefficients and calculating performance metrics such as the error, accuracy, precision, recall, receiver operating characteristic curve (ROC), etc.” [i.e., the correlation techniques used incorporate matching a ground truth values against model output values, which is formatted as a one-hot encoded vector in the context of machine learning]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Clement, Chen, Hu and Akbari to incorporate the computation of a matching loss comparing a prediction to the ground-truth one-hot label of Zhao using the code sequence and text of Akbari. Doing so would have allowed the combination of Clement, Chen, Hu and Akbari to use Zhao's method for “auto composing using a transformer-based language model having a PSDP that reduces the number of parameters of a model and at the same time maintains the capability of generating understandable and reasonable compositions”, as suggested by Zhao (see, e.g., Zhao, paragraph [0031]).
Claims 10, 16 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Clement, Chen, in view of Akbari and further in view of Zhao.
Regarding Claim 10, as discussed above, Clement, Chen and in view of Akbari teaches the method of claim 1.
Chemens, Chen and Akbari do not specifically disclose wherein the decoder comprises one or more decoder modules that share same parameters except a last feed forward layer adapted as a different decoder head for different bimodal training objectives,
and wherein the generating the training output comprises: generating, by the one or more decoder modules, respective decoding outputs according to the different bimodal training objectives in parallel.
However Zhou teaches wherein the decoder comprises one or more decoder modules that share same parameters except a last feed forward layer adapted as a different decoder head for different bimodal training objectives (see, e.g., Zhao paragraph [0031]: "the parameter sharing decoder pair comprises: a first decoder comprising N layers, each layer comprising a first masked multi-head attention block and a first feed forward network, wherein parameters of the first decoder are shared across all N layers of the first decoder; and a second decoder comprising N layers" [i.e., the parameter sharing decoder pair serves as one or more decoder modules sharing parameters], paragraph [0034]: "The PSDP comprises two smaller modified transformer decoders, each transformer decoder having its own set of parameters for both the masked multi-head attention and the feedforward network, with the parameters being shared across all the layers within the transformer decoders" [i.e., the two decoders learn two separate representations optimizing towards a single cross-entropy loss which functions as a bimodal training objective], paragraphs [0035-0036]: “The first decoder 205 further comprises a feed forward network 235… The second decoder 210 further comprises a feed forward network 265… The second normalization from each of the first decoder 205 and the second decoder 210 are fed into a concatenation layer 275 and concatenated to generate a concatenation of the normalized outputs from the first decoder 205 and the second decoder 210. The concatenation is then fed into a mapping layer 280 and a subsequent normalization layer 285, which ultimately generate the final output 290” [i.e., the last feed forward layer (mapping layer 280), of the decoder modules containing feed forward networks, is adapted into a combined decoder head for a combined (different) bimodal training objective] and paragraph [0037]: "The masked multi-head attention block 215, the first normalization layer 230, the feed forward network 235, and the second normalization layer 240 of each N layer of the first decoder 205 all share parameters across all layers of the decoder 205" [i.e., parameter sharing between multiple decoder modules]),
and wherein the generating the training output comprises: generating, by the one or more decoder modules, respective decoding outputs according to the different bimodal training objectives in parallel (see, e.g., Zhao paragraph [0036]: "The second normalization from each of the first decoder 205 and the second decoder 210 are fed into a concatenation layer 275" [i.e., both decoders operate simultaneously on the input sequence (in parallel) to produce their respective decoding outputs which are then combined via concatenation before the final prediction] and paragraph [0125] "Although some flowcharts describe operations as a sequential process, many of the operations can be performed in parallel or concurrently").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Clement, Chen in view of Akbari to incorporate the sharing of parameters between decoder modules, except a last feed forward layer adapted as a different decoder head for different bimodal training objectives, and the generation of respective decoding outputs according to the various bimodal training objectives performed in parallel of Zhou. Doing so would have allowed Clement, Chen in view of Akbari to use Zhao's method for “auto composing using a transformer-based language model having a PSDP that reduces the number of parameters of a model and at the same time maintains the capability of generating understandable and reasonable compositions”, as suggested by Zhao (see, e.g., Zhao, paragraph [0031]).
Regarding Claims 16 and 19, they are system and program product claims having similar limitations as of method claim 10 above. Therefore they are rejected under the same rational as of claim 10 above.
Response to Arguments
Applicant's arguments filed 3/31/2006 regarding 103 rejection has been fully considered but they are not persuasive.
Argument regarding 103 Rejection:
a) In remarks page 12, applicant argues that Clement fails to teach encoder comprising a first encoder module and second encoder module operating in parallel and Akbari fails to teach a decoder connected to the first and second encoder module of the encoder.
Response to argument:
Regarding argument (a) examiner respectfully disagrees with the applicant. The limitation was previously presented in claim 9 except disclosing “a decoder connected to the first encoder and the second encoder module”. Akbari was cited to teach the limitation of claim 9 which discloses in par. 128 and figure 2A having encoder 206 which includes encoder module 210 and 220 that are trained tother which implies both encoder modules are being operated in parallel. Additionally figure 2A also clearly discloses encoder 206 having encoder module 210 and 220 are connected to decoder module 208. The claim language is broad and does not specifically disclose that the encoder modules are directly connected to the decoder or the decoder receiving each encoder output directly at the decoder. Moreover, applicant’s specification or drawing do not specifically disclose that there are any direct communication between the encoder modules and the decoder (see applicant drawing 3A, 3B). Therefore Akbari as cited clearly discloses the claim language as argued. Accordingly applicant’s argument is not persuasive.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDULLAH AL KAWSAR whose telephone number is (571)270-3169. The examiner can normally be reached M-F 7:30am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Wiley can be reached at (571) 272-4150. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDULLAH AL KAWSAR/ Supervisory Patent Examiner, Art Unit 2127
1 The training objective (loss function) in Seq2Seq shown in Fig. 3, requires comparing the predicted annotation (training output) against the corresponding ground-truth annotation (target sequence / second training input dataset) to measure error and update model weights.
Prosecution Insights