DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiners Remarks
The current claims are noted to not be rejected under 101 for not currently containing any abstract ideas.
Response to Arguments
Remarks page 7-10, Applicant contends:
Amended claim limitations, such as claims 1, 8, and 15, do not appear taught by prior art, as amended claims recite a first and second quantization learning step as recited by the claims.
Response:
Applicant’s arguments with respect to claim(s) 1, 8, and 15 have been considered but are moot because the new ground of rejection contain elements that have not been previously examined or does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Remarks page 10-11, Applicant contends:
Claims 5, 12, and 19 as amended do not recite mathematical concepts, thus should not be rejected under 101.
Response:
Applicant’s arguments with respect to claim(s) 5, 12, and 19 have been considered but are moot because the new ground of rejection contain elements that have not been previously examined or does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 5, 6, 8, 9, 12-16, 19, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zagoruyko et al (“Paying More Attention to Attention: Improving The Performance Of Convolutional Neural Networks Via Attention Transfer”), referred to as Zagoruyko in this document, and further in combination with Mishra et al (“Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy”), referred to as Mishra in this document, and further in combination with Akino (US 20230081531 A1), referred to as Akino in this document, and further in combination with Vaswani et al (“Attention Is All You Need”), referred to as Vaswani in this document, and further in combination with Goutam et al (“LayerOut: Freezing Layers in Deep Neural Networks”), referred to as Goutam in this document.
Regarding Claim 1:
Zagoruyko teaches:
performing a first quantization learning by inserting a first self-attention map of a teacher model into a second self-attention map of the student model;
and performing a second quantization learning using a knowledge distillation method so that the second self-attention map of the student model follows the first self-attention map of the teacher model
[Zagoruyko Introduction page 2]: “To that end, we propose several novel ways of transferring attention [performing a first quantization learning by inserting a first self-attention map of a teacher model into a second self-attention map of the student model to replace a respective portion of the student model] from a powerful teacher network to a smaller student network with the goal of improving the performance of the latter (Fig. 1). To summarize, the contributions of this work are as follows: -We propose attention as a mechanism of transferring knowledge from one network to another -We propose the use of both activation-based and gradient-based spatial attention maps -We show experimentally that our approach provides significant improvements across a variety of datasets and deep networks architectures, including both residual and non-residual networks -We show that activation-based attention transfer gives better improvements than full-activation transfer, and can be combined with knowledge distillation”
[Zagoruyko 3 Attention Transfer page 4]: “In attention transfer, given the spatial attention maps of a teacher network (computed using any of the above attention mapping functions), the goal is to train a student network that will not only make correct predictions but will also have attentions maps that are similar [and performing a second quantization learning step using a knowledge distillation method so that the second self-attention map of the student model follows the first self-attention map of the teacher model] to those of the teacher.”
The performing of the first and second quantization learning are interpreted under BRI to be elements of knowledge distillation involving attention.
Zagoruyko does not explicitly teach:
A method for quantization learning by a model quantizer operating in a computer system
and compressing a transformer model, the method comprising: generating a student model through quantization of the transformer model
first self-attention map
second self-attention map
wherein parameter learning of a first parameter part of the student model is blocked in the first quantization learning step
wherein parameter learning of the first parameter part of the student model occurs in the second quantization learning step
Mishra teaches:
and compressing a transformer model, the method comprising: generating a student model through quantization of the transformer model;
[Mishra Introduction page 2]: “In this paper, we study the combination of network quantization with model compression and show that the accuracies of low-precision networks can be significantly improved by using knowledge distillation techniques. Previous studies on model compression use a large network as the teacher network and a small network as the student network. The small student network learns from the teacher network using the distillation process. The network architecture of the student network is typically different from that of the teacher network – for e.g. Hinton et al. (2015) investigate a student network that has fewer number of neurons in the hidden layers compared to the teacher network. In our work, the student network has similar topology as that of the teacher network, except that the student network has low-precision neurons [and compressing a transformer model, the method comprising: generating a student model through quantization of the transformer model;] compared to the teacher network which has neurons operating at full-precision.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Zagoruyko and Mishra. Zagoruyko and Mishra are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Zagoruyko and Mishra in order to create smaller/compressed models that provide good performance models on less powerful hardware ([Mishra Introduction page 2]: “Overall, the contributions of this paper are the techniques to obtain low-precision DNNs using knowledge distillation technique. Each of our scheme produces a low-precision model that surpasses the accuracy of the equivalent low-precision model published to date. One of our schemes also helps a low-precision model converge faster. We envision these accurate low-precision models to simplify the inference deployment process on resource constrained systems and even otherwise on cloud-based deployment systems.”). Zagoruyko even notes the idea of using what is taught with knowledge distillation ([Zagoruyko introduction page 2]: “We show that activation-based attention transfer gives better improvements than full-activation transfer, and can be combined with knowledge distillation”).
Akino teaches:
A method for quantization learning by a model quantizer operating in a computer system
[Akino 0014]: “According to some embodiments of the present invention, a computer-implemented [A method for quantization learning by a model quantizer operating in a computer system] method is provided for training a set of artificial neural networks. The method may be performed by one or more computing processors [a processor] in association with a memory [a memory] storing computer-executable programs [computer readable instructions that include an executable].”
The quote above shows the use of a computer system, as well as notes elements of a computer system that are relevant to the current application.
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Zagoruyko and Akino. Zagoruyko and Akino are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Zagoruyko and Akino in order to create a computer implemented version of the invention or application ([Akino 0014]: “According to some embodiments of the present invention, a computer-implemented method is provided for training a set of artificial neural networks. The method may be performed by one or more computing processors in association with a memory storing computer-executable programs.”).
Vaswani teaches:
a transformer model
the transformer model
[Vaswani Introduction page 2]: “In this work we propose the Transformer [a transformer model][the transformer model], a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output. The Transformer allows for significantly more parallelization and can reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs”
first self-attention map
second self-attention map
[Vaswani 2 Background page 2]: “To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention [first self-attention map][second self-attention map] to compute representations of its input and output without using sequence aligned RNNs or convolution. In the following sections, we will describe the Transformer, motivate self-attention and discuss its advantages over models such as [17, 18] and [9].”
The above mapping is to teach that the attention type can be self-attention, as the primary ref Zagoruyko doesn’t explicitly note the attention utilized can be self-attention. The above mapping also supports the idea of utilizing self-attention with transformer models by noting that transformer models rely on self-attention.
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Zagoruyko and Vaswani. Zagoruyko and Vaswani are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Zagoruyko and Vaswani in order to incorporate transformer models, as transformer models provide benefits of parallelization and are good for translations ([Vaswani Introduction page 2]: “The Transformer allows for significantly more parallelization and can reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs”).
Goutam teaches:
wherein parameter learning of a first parameter part of the student model is blocked in the first quantization learning step
wherein parameter learning of the first parameter part of the student model occurs in the second quantization learning step
[Goutam Proposed Method – LayerOut page 3]: “LayerOut proposes a simple modification to the backpropagation algorithm by requiring that only a few randomly chosen layers be updated. For a given architecture and a given epoch during train phase, a uniform random vector of prob abilities is sampled wherein the ith component represents the probability of binary decision to freeze [wherein parameter learning of a first parameter part of the student model is blocked in the first quantization learning step] or update layer [wherein parameter learning of the first parameter part of the student model occurs in the second quantization learning step] i of the architecture. When a decision to freeze a layer is made, the parameters corresponding to that layer are omitted from being updated during the backpropagation. We refer to these decisions as freezing strategy.”
The interpretation of needing a first parameter part that is blocked during the first step and the learning of the first parameter part occurs in the second step is interpreted as indicating that parts of the model are frozen for a step or part of training but not in another step. Goutam indicates this in the above quote by noting layers/parts of a model are frozen or updated during training where what is updated or frozen changes during epochs of the training phase, thus showing steps where a part is frozen or the part is updated.
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Zagoruyko and Goutam. Zagoruyko and Goutam are in the same field of endeavor of machine learning. One of ordinary skill would have combine Zagoruyko and Goutam to incorporate freezing parts of a model during training in order to encourage more robust learning ([Goutam Why LayerOut Should Work? Page 4]: “The conventional regularization techniques achieves generalization of any DNNs by preventing the hidden layers to co-adapt to certain specific features. In LayerOut when the layers are frozen, the hidden nodes are not activated non deterministically. The implicit source of randomness or noise means that every layer has to learn to be more robust to a lot of variation in its input thus preventing the co-adaptation of hidden layers to specific features.”).
Regarding Claim 2:
The method of claim 1 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam
Mishra teaches:
wherein the first self-attention map of the teacher model that is inserted into the second self-attention map of the student model corresponds to a third self-attention map of the transformer model prior to the quantization of the transformer model
[Mishra Introduction page 2]: “In the third scheme, we start with a trained full-precision large network and an apprentice network that has been initialised with full-precision weights. The apprentice network’s precision is lowered [wherein the first self-attention map of the teacher model that is inserted into the second self-attention map of the student model corresponds to a third self-attention map of the transformer model prior to the quantization of the transformer model where aspects of a transformer model and self-attention are taught in claim 1] and is fine-tuned using knowledge distillation techniques. We find that the low-precision network’s accuracy marginally improves and surpasses the accuracy obtained via the first scheme. This scheme then sets the new state-of-the-art accuracies for the ResNet models at ternary and 4-bit precision”
This limitation is interpreted under BRI to mean the teacher model is the same model as the student model before quantization (which aligns with [Current Application 0026]: “In the first step quantization learning, the self-attention map for each layer of the teacher model that is already well-trained by the model quantizer 1250 is inserted into the self-attention map portion of the student model. In this case, the self-attention map of the student model may be changed to the state before quantization.”), meaning the teacher model and the transformer model are interpreted as being the same model in combination with the teachings of claim 1 noting a transformer model is quantized to become a student model. The models containing similar topology (as noted by claim 1 mapping for the compressing for the student model in Mishra Introduction page 2) and student being a full-precision like the teacher before being quantized teaches that the teacher model fits the description for being a version of the student model before quantization. The teacher model “teaching” the student model also indicates the models are to perform similar tasks, thus providing further evidence of the unquantized student model being akin to the teacher model in combination with the other evidence.
Motivation to combine with Mishra is the same motivation to combine with Mishra in claim 1.
Regarding Claim 5:
The method of claim 1 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam
Zagoruyko teaches:
wherein the second quantization learning step further comprises determining a loss value between the second self-attention map of the student model and the first self-attention map of the teacher model
[Zagoruyko 3 Attention Transfer page 6]: “Attention transfer [wherein the second quantization learning further comprises calculating a loss value between the second self-attention map of the student model and the first self-attention map of the teacher model as this quote is noting that loss involves the parts for attention transfer] can also be combined with knowledge distillation Hinton et al. (2015), in which case an additional term (corresponding to the cross entropy between softened distributions over labels of teacher and student) simply needs to be included to the above loss. When combined, attention transfer adds very little computational cost, as attention maps for teacher can be easily computed during forward propagation, needed for distillation”
Regarding Claim 6:
The method of claim 5 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam
Zagoruyko teaches:
wherein the loss value corresponds to a probability distribution distance between parameters of the second self-attention map of the student model and parameters of the first self-attention map of the teacher model
[Zagoruyko 3 Attention Transfer page 6]: “Attention transfer can also be combined with knowledge distillation Hinton et al. (2015), in which case an additional term (corresponding to the cross entropy between softened distributions [wherein the loss value corresponds to a probability distribution distance between parameters of the second self-attention map of the student model and parameters of the first self-attention map of the teacher model softened distributions are interpreted as noting the element of knowledge distillation of probability distributions] over labels of teacher and student) simply needs to be included to the above loss. When combined, attention transfer adds very little computational cost, as attention maps for teacher can be easily computed during forward propagation, needed for distillation”
Regarding Claim 8:
Claim 8 is analogous to claim 1.
Regarding Claim 9:
The computer system of claim 8 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam.
Claim 9 is analogous to claim 2.
Regarding Claim 12:
The computer system of claim 8 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam.
Claim 12 is analogous to claim 5.
Regarding Claim 13:
The computer system of claim 12 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam.
Claim 13 is analogous to claim 6.
Regarding Claim 14:
The computer system of claim 13 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam.
Zagoruyko teaches:
wherein the processor is further configured to execute the model quantizer software to perform the second quantization learning by activating a gradient value of query weights, key weights, and value weights so that learning of parameters related to the second self-attention map of the student model occurs
[Zagoruyko 3 Attention Transfer page 6]: “Attention transfer can also be combined with knowledge distillation Hinton et al. (2015), in which case an additional term (corresponding to the cross entropy between softened distributions over labels of teacher and student) simply needs to be included to the above loss [wherein the processor is further configured to execute the model quantizer software to perform the second quantization learning by activating a gradient value of query weights, key weights, and value weights so that learning of parameters related to the second self-attention map of the student model occurs]. When combined, attention transfer adds very little computational cost, as attention maps for teacher can be easily computed during forward propagation, needed for distillation”
The above limitations are interpreted under BRI to mean that aspects of the model that are related to attention are trained. The loss in the above teaching is noted to involve the teaching of attention. Support for this interpretation is given by the specification noting that elements are trained related to the attention maps during training ([Current Specification 0036]: “In the second step quantization learning, the model quantizer 1250 performs quantization learning on the self-attention map related parameter (SA-GEN) and the remaining parameters (PROP) intensively trained in the first step quantization learning. At this time, quantization learning is performed so that the self-attention maps (SAMs) of the student model 1440 can follow the self-attention maps (SAMt) of the teacher model 1420.”).
Zagoruyko does not explicitly teach:
query weights, key weights, and value weights
Vaswani teaches:
query weights, key weights, and value weights
[Vaswani 3.2 Attention page 3]: “An attention function can be described as mapping a query and a set of key-value pairs [query weights, key weights, and value weights] to an output, where the query, keys, values, and output are all vectors. The output is computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility function of the query with the corresponding key.”
Attention is taught in the primary reference, and Vaswani indicates that attention utilizes query, key, and value like the claim limitation notes. Thus the motivation to combine the teaching of query, key, and value with Zagoruyko is considered strong, as the elements required are noted to be in the primary reference according to Vaswani.
The motivation to combine Zagoruyko with Vaswani is the same for the motivation to combine with Vaswani in claim 4.
Regarding claim 15:
Claim 15 is analogous to claim 1.
Regarding claim 16:
The method of claim 15 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam.
Claim 16 is analogous to claim 2.
Regarding claim 19:
The method of claim 15 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam.
Claim 19 is analogous to claim 5.
Regarding claim 20:
The method of claim 19 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam.
Claim 20 is analogous to claim 6.
Allowable Subject Matter
Claims 3 and 7 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is an examiner's statement of reasons for allowance:
The prior art of record teaches limitations as noted by the previous office action.
However, the claim 3 in the application is deemed to be directed to a nonobvious improvement over the prior art of record.
Regarding Claim 3:
The method of claim 2 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam
Prior art of record does not explicitly teach:
wherein the first quantization learning step further comprises not providing a gradient value of at least one weight to a second parameter part so that the parameter learning of the first parameter part related to the second self-attention map is blocked
Zagoruyko teaches aspects related to attention transfer, but does not give any indication of freezing the transferred attention. The above limitation from claim 3 is seen as indicating freezing/blocking at least part of the transferred attention, as claim 1 recites “performing a first quantization learning step by inserting a first self-attention map of a teacher model into a second self-attention map of the student model to replace a respective map portion of the student model, wherein parameter learning of a first parameter part of the student model is blocked in the first quantization learning step”. The claim 1 limitations don’t indicate that the frozen/blocked portion is related to any particular part of the student model. Claim 1 does note that the attention map of the student is replaced, which when combined with the recitation of claim 3 indicating that the first parameter part that is blocked is “related to the second self-attention map”.
Closet prior art of record teaches:
As taught in claim 1, Goutam teaches the freezing of parts of a model to help create more robust training, but provides no indication of intentionally freezing the attention or a part that is transferred from another model. The recitation in claim 1 appeared more as a combination of existing elements without much connection between the elements (as nothing in claim 1 indicates what a first parameter part should be in particular), but claim 3 indicates a particular connection in that the first parameter part is not just any part of the model but refers to the attention map which was replaced.
Wiegreffe et al (“Attention is not Explanation”) teaches freezing the attention of a model, but does it to test the effects of attention on the model, not to freeze/block an element that was transferred, thus the motivation and use of the blocking of attention doesn’t appear to overlap with what is taught by the claims.
[Weigreffe et al Introduction page 1-2]: “We apply a more model-driven approach to this question, beginning (§3.2) with testing attention modules’ contribution to a model by ap plying a simple baseline where attention weights are frozen to a uniform distribution. We demonstrate that for some datasets, a frozen attention distribution performs just as well as learned attention weights, concluding that randomly- or adversarially-perturbed distributions are not evidence against attention as explanation in these cases.”
Serra et al (“Overcoming Catastrophic Forgetting with Hard Attention to the Task”) teaches that a form of masking relating to attention can be used to help prevent catastrophic forgetting during training/fine tuning, but provides no indication of masking/blocking elements that are related to a transferred portion of a model. The teaching for Serra et al are also for relations to new tasks, where as the current application is using a student that is a quantized version of the teacher, thus would both be for the same task.
[Serra et al Introduction page 2]: “In this paper, we propose a task-based hard attention mechanism that maintains the information from previous tasks without affecting the learning of a new task… Since masks are almost binary, a portion of the weights remains static while the rest adapt to the new task.”
[Serra et al 2.3 page 3]: “This preserves the attention values for units that were important for previous tasks, allowing them to condition the training of future tasks.”
Aspects of the interpretation are done in light of the specification, such as paragraph 26 of the current application, which indicates a more detailed description of what the training referenced in the claims can be.
[Current Application 0026]: "The model quantizer 1250 may be driven by the CPU 1100 to perform a two-step quantization learning operation of the transformer model. In the first step quantization learning, the self-attention map for each layer of the teacher model that is already well-trained by the model quantizer 1250 is inserted into the self-attention map portion of the student model. In this case, the self-attention map of the student model may be changed to the state before quantization. In this state, through quantization learning of the student model, it may be possible to learn comparatively intensely the remaining parameters (PROP), rather than the parameters related to the self-attention map (SA-GEN). At this time, the parameter (SA-GEN) related to the self- attention map of the student model does not affect the learning operation of the student model. Therefore, the gradient value flowing from the parameter (SA-GEN) related to the self-attention map to the remaining parameters (PROP) may be cut off or blocked."
Thus while elements of the current application appear in prior art, the aspects recited in prior art do not appear to provide a motivation or complete teaching for the elements recited in claim 3. As a result, claim 3 is seen as allowable subject matter for being non-obvious under the prior art.
Analogous claim 10 and 17 are also allowable subject matter for the reasons cited for claim 3.
Regarding Claim 7:
The method of claim 5 is taught by Zagoruyko, Mishra, Akino, Vaswani, Goutam
Prior art of record does not explicitly teach:
wherein the second quantization learning step further comprises providing gradient values of weights to a second parameter part unrelated to the second self-attention map of the student model so that the parameter learning of the first parameter part related to the second self-attention map of the student model occurs
Claim 7 is noted to be seen as allowable subject matter for the same reason as claim 3. Providing the gradient values of weights is noted as taught by Zagoruyko in the previous office action mapping for claim 7, but claim 7 in combination with the amended claim 1 now indicates that a first parameter part is blocked that is related to the second self-attention map, thus providing the indication of the blocked/frozen part should contain at least a part of the attention map that was transferred in the first quantization step. As explained in the reasons for allowable subject matter of claim 3, the indication of freezing/blocking a transferred attention map for a step and then not blocking the map in another step does not appear taught by prior art.
The dependent claims of allowable subject matter claims are allowable subject matter because they depend on one of the allowable subject matter claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Jin et al (“KDLSQ-BERT: A Quantized Bert Combining Knowledge Distillation with Learned Step Size Quantization”) is relevant art that teaches quantizes transformer models like BERT and then distilling to compress transformer models.
Zhang et al (“TernaryBERT: Distillation-aware Ultra low Bit BERT”) is relevant art that notes the elements of transformers like BERT related to processes like knowledge distillation for creating better quantized models.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER D DEVORE whose telephone number is (703)756-1234. The examiner can normally be reached Monday-Friday 7:30 am - 5 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.D.D./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights