DETAILED ACTION
In view of the Appeal Brief filed on 03/02/2026, PROSECUTION IS HEREBY REOPENED. A new ground of Rejection is set forth below.
To avoid abandonment of the application, appellant must exercise one of the following two options:
(1) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply under 37 CFR 1.113 (if this Office action is final); or,
(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41.31 followed by an appeal brief under 37 CFR 41.37. The previously paid notice of appeal fee and appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth in 37 CFR 41.20 have been increased since they were previously paid, then appellant must pay the difference between the increased fees and the amount previously paid.
A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by signing below:
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-15 are currently pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/21/2026 was filed after the mailing date of the Final Office Action on 04/29/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s arguments, see Pages 5-11, filed 03/02/2026, with respect to the 35 U.S.C. 101 Rejection of Claims 1-15 have been fully considered and are persuasive. The 35 U.S.C. 101 Rejection of Claims 1-15 has been withdrawn.
Applicant’s arguments with respect to the prior art Rejection(s) of claim(s) 1-15 under 35 U.S.C. 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“A device configured to…” in claim 14
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 4-6, and 8-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (Learning Efficient Convolutional Networks through Network Slimming, 2017), hereinafter Liu; Molchanov et al. (Variational Dropout Sparsifies Deep Neural Networks, 2017), hereinafter Molchanov; and Prokhorov et al. (US 10139823 B2), hereinafter Prokhorov.
In regards to claim 1: The present invention claims: “A computer-implemented method for compressing a neural network,” Liu teaches “In this paper, we propose a novel learning scheme for CNNs to simultaneously 1) reduce the model size; 2) decrease the run-time memory footprint; and 3) lower the number of computing operations, without compromising accuracy.” (Abstract), and highlights the necessity to compress neural networks on said limited hardware (Page 2736).
“the neural network including at least one sequence of a first layer, which carries out a weighted summation of input variables of the first layer as a function of a multitude of weightings,” Liu teaches their method is for slimming a neural network, containing at least one layer taking input variables (Figure 1), with a plurality of weightings (2738, right column). Examiner’s Note: Weighted summation is demonstrably one of the most standard means for the input layer of a neural network to process input data (see References Not Cited below). The Examiner considers this high-level recitation of “a first layer, which carries out a weighted summation of input variables” inherent to a common neural network a POSITA would have been aware of at the time of the Applicant’s filing, in the absence of further detail differentiating the recited feature from the common understanding of the term.
“and a second layer, which carries out an affine transformation as a function of scaling factors of input variables of the second layer…” Liu Equation 1 (Page 2738) reasonably reads on a layer calculating an output based on weightings and a scaling factor. Liu also teaches “γ and β are trainable affine transformation parameters (scale and shift) which provides the possibility of linearly transforming normalized activations back to any scales.” (Page 2739). Examiner’s Note: An affine transformation is demonstrably one of the most standard means for intermediate or activation layer of a neural network to process data (see References Not Cited below). The Examiner considers this high-level recitation of “a second layer, which carries out an affine transformation” inherent to a common neural network a POSITA would have been aware of at the time of the Applicant’s filing, in the absence of further detail differentiating the recited feature from the common understanding of the term.
“the method comprising the following steps: defining a maximum complexity, the complexity characterizing a consumption of computer resources of the first layer;” Liu teaches “We prune channels with a global threshold across all layers, which is defined as a certain percentile of all the scaling factor values. For instance, we prune 70% channels with lower scaling factors by choosing the percentile threshold as 70%. By doing so, we obtain a more compact network with less parameters and run-time memory, as well as less computing operations.” (Page 2739).
“ascertaining a first cost function which characterizes a deviation of ascertained output variables of the neural network in relation to predefined output variables from training data;” Liu teaches “Specifically, the training objective of our approach is given by [Equation (1)] where (x, y) denote the train input and target, W denotes the trainable weights, the first sum-term corresponds to the normal training loss of a CNN…” (Page 2738). The first part of Liu’s Equation 1 is the typical training loss of the neural network, which reads on the “deviation of ascertained output variables of the neural network in relation to predefined output variables from training data”
“ascertaining a second cost function which characterizes a deviation of a current complexity of the neural network in relation to the maximum complexity, the current complexity being ascertained as a function of a number of the scaling factors which have an absolute value greater than a predefined threshold value;” Liu teaches “Specifically, the training objective of our approach is given by [Equation (1)] … g(·) is a sparsity-induced penalty on the scaling factors, and λ balances the two terms. In our experiment, we choose g(s) = |s|, which is known as L1-norm and widely used to achieve sparsity.” (Page 2738-2739) and “For instance, we prune 70% channels with lower scaling factors by choosing the percentile threshold as 70%.” (Page 2739). The second portion of Liu’s Equation 1 is the scaling factors of the channels, ultimately pruned as the scaling factors fall below the predefined complexity threshold.
“training the neural network in such a way that a sum of the first cost function and the second cost function is optimized as a function of the weightings and the scaling factors of the neural network;”. Liu teaches Our idea is introducing a scaling factor γ for each channel, which is multiplied to the output of that channel. Then we jointly train the network weights and these scaling factors, with sparsity regularization imposed on the latter. Finally we prune those channels with small factors, and fine-tune the pruned network… where (x, y) denote the train input and target, W denotes the trainable weights, the first sum-term corresponds to the normal training loss of a CNN, g(·) is a sparsity-induced penalty on the scaling factors, and λ balances the two terms.” (Page 2738)
While Liu essentially teaches the above algorithmic process, Liu focuses on channel-level pruning (which the Examiners notes may reasonably read on the forthcoming recitation of “weightings” as a channel would be a set of individual weights), rather than weight-level pruning, and therefore fails to explicitly teach:
“…weightings of the first layer each being assigned a scaling factor from the second layer,” However, Molchanov, in a similar field of endeavor, teaches “In this paper, we study Variational Dropout (Kingma et al., 2015) in the case when each weight of a model has its individual dropout rate. We propose Sparse Variational Dropout that extends Variational Dropout to all possible values of dropout rates and leads to a sparse solution.” (Page 1) and “High dropout rate…corresponds to a binary dropout rate that approaches p = 1. It effectively means that the corresponding weight or neuron is always ignored and can be removed from the model. In this work, we consider the case of individual [dropout rate] for each weight of the model.” (Page 4)
“removing those weightings of the first layer whose assigned scaling factor has an absolute value smaller than the predefined threshold value;” Liu teaches “We prune channels with a global threshold across all layers, which is defined as a certain percentile of all the scaling factor values. For instance, we prune 70% channels with lower scaling factors by choosing the percentile threshold as 70%. By doing so, we obtain a more compact network with less parameters and run-time memory, as well as less computing operations.” (Page 2739, mapping to the predetermined threshold value) and Molchanov teaches “As it is impossible for the weights to converge exactly to zero in a stochastic setting, we explicitly put weights with high corresponding dropout rates to 0 during testing. In our experiments with neural networks, we use the value log α = 3 as a threshold. This value corresponds to a Binary Dropout rate p > 0.95.” (Page 6, this is the dropout rate applied to each weight individually).
While Molchanov’s parameter trained on is dropout rate, rather than a scaling factor, as taught by Liu in the “Slimming” paper, it highlights that training and/or optimizing a parameter relevant to individual weight removal would have been known in the art at the time of the Applicant’s filing. Molchanov also highlights the limits of complex networks on limited hardware and the need to sparsify or compress the models (Page 1). Further, in a subsequent paper by Liu, cited here for reference and not part of the rejection (RETHINKING THE VALUE OF NETWORK PRUNING, 2019), Liu compares structured pruning (channel-level, for example) as in the original 2017 paper, and unstructured pruning (weight-level). Figures 3-4, and Table 7 of the “Rethinking” paper show as good or better performance pruning weights than pruning the entire channel(s). All of Liu’s “Slimming” paper, Molchanov, and Liu’s “Rethinking” paper highlight the need to prune or compress neural networks on computationally limited environments. It would have been obvious to one of ordinary skill in the art at the time of the Applicant’s filing to take the multiple function portions of Liu’s “Slimming” paper, and optimize over both the prediction loss, as well as a weight-level pruning optimization over a scaling factor, knowing that Liu’s “Rethinking” paper demonstrates benefits to weight-level pruning over channel-level pruning.
The combination of Liu and Molchanov fails to explicitly teach:
“and providing an actuator configured to control, in accordance with a control variable that is based on an output of the compressed neural network, a physical operation of a technical system.” Prokhorov teaches, in at least Fig. 4, the conversion of a machine learning model’s output into operational data and actuator control signals for the operation of a vehicle.
Prokhorov shows that the conversion of a machine learning model’s output into operational data and actuator control signals for the operation of a vehicle would have been known in the art at the time of Liu and Molchanov’s writing and before the Applicant’s filing date. Using known machine learning methods such as a combination of Liu and Molchanov to produce the output for a computationally limited environment such as an actuator would have been obvious to a person skilled in the art.
In regards to claim 4: The present invention claims: “wherein the first layer is a convolutional layer, and the weightings are filters, each of the scaling factors being assigned to a respective filter of the convolutional layer.” Section 4.4 (Pag 5) of Molchanov refers to a filter tensor in their calculations.
In regards to claim 5: The present invention claims: “wherein the complexity is defined as a function of an architecture of a processing unit on which the compressed neural network is to be executed.” Based on Applicant’s Specification, which recites the aforementioned limitation may apply to the entire network, the Examiner submits Liu’s threshold pertaining to the amount a model may be pruned sufficiently reads on this generic limitation.
In regards to claim 6: The present invention claims: “wherein the predefined threshold value is t = 10-4.” Neither Liu nor Molchanov teaches this particular value. It would have been obvious to one having ordinary skill in the art at the time the invention was made to optimize a threshold for pruning, since it has been held that discovering an optimum value of a result effective variable involves only routine skill in the art. In re Boesch, 617 F.2d 272,265 USPQ 215 (CCPA 1980).
In regards to claim 8: The present invention claims: “wherein the second cost function is scaled with a factor, the factor being selected in such a way that a value of the scaled second cost function corresponds to an ascertained value of the first cost function at the beginning of the training.” See Liu, Equation 1, which includes a value to balance the prediction loss portion of the function and the complexity portion.
In regards to claim 9: The present invention claims: “wherein, at the beginning of the training, the factor of the second cost function is initialized using a value 1 and, during repeated execution of the step of training, the factor is steadily increased until the factor corresponds to the ascertained value of the first cost function at the beginning of the training.” See the above rejections of Claims 1 and 8. While Liu teaches learning on the scaling factor (Page 2739), Liu does not explicitly teach the values claimed. It would have been obvious to one having ordinary skill in the art at the time the invention was made to optimize a threshold for pruning, since it has been held that discovering an optimum value of a result effective variable involves only routine skill in the art. In re Boesch, 617 F.2d 272,265 USPQ 215 (CCPA 1980).
In regards to claim 10: The present invention claims: “wherein, after the step of removing the weightings, the neural network is partially subsequently trained as a function of the first cost function.” Liu teaches “We can also extend the proposed method from single-pass learning scheme (training with sparsity regularization, pruning, and fine-tuning) to a multipass scheme. Specifically, a network slimming procedure results in a narrow network, on which we could again apply the whole training procedure to learn an even more compact model.” (Page 2739) and Molchanov makes reference to backward passes in “Both forward and backward passes through Sparse VD layers take twice as much time as passes through original layers.” (Page 6), either of which reads on the generic recitation of training the model further on the prediction loss after pruning.
In regards to claim 11: The present invention claims: “wherein the complexity characterizes a number of multiplications of the first layer or a number of parameters of the first layer or a number of output variables of the first layer.” Both the channel-level pruning of Liu and the weight-level pruning of Molchanov would impact the number of operations or outputs from a given layer; therefore, the Examiner submits the generic recitation of the claim is read on by a pruning threshold as taught by either.
In regards to claim 12: The present invention claims: “wherein the complexity characterizes a number of multiplications and parameters, the second cost function characterizing a sum of the deviation of the current complexity and predefined complexity with respect to the number of parameters and the number of multiplications.” See the rejections of claims 1 and 11 for how a combination of Liu and Molchanov reads on a second cost function (Liu, Equation 1, second portion) pertaining to the complexity (pruning threshold) of the neural network.
In regards to claim 13: The present invention claims: “using the compressed neural network as an image classifier.” Liu trains their model extensively on image datasets (Section 4.1).
In regards to claim 14: Claim 14 recites similar limitations as claim 1, save for the recitation of “A device configured to compress a neural network,” therefore, claim 14 is rejected with the same rationale as claim 1 above.
In regards to claim 15: Claim 15 recites similar limitations as claim 1, save for the recitation of “A non-transitory machine-readable memory medium on which is stored a computer program for compressing a neural network,” therefore, claim 15 is rejected with the same rationale as claim 1 above.
Claim(s) 2, 3, and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu, Molchanov, and Prokhorov as applied to claim 1 above, and further in view of Gordon et al. (MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks, 2018), hereinafter Gordon.
In regards to claim 2: While Liu teaches scaling factors, it fails to explicitly teach “wherein a current number of scaling factors is ascertained using a sum of indicator functions, applied to each scaling of the scaling factors,” However, Gordon teaches a sparsifying regularizer with indicator functions based on whether a scale has been zeroed out (Page 1590, Sections 4.2 and 4.3).
“the indicator function outputting a value 1 when an absolute value of the scaling factor is greater than the threshold value, and otherwise outputting a value 0,” Gordon teaches the indicator function outputting 1 if the node is not zeroed-out (1590, Section 4.2).
“the current complexity being ascertained as a function of the sum of the indicator functions, standardized to a number of the calculated weightings of the first layer, multiplied with a number of parameters or multiplications of the first layer.” See Gordon Section 4.2 for a constraint being calculated with either FLOPs (mapping operations like FLOPs to multiplications) or with the model size (number of parameters), multiplied by the sum of the indicator functions.
Gordon highlights that their system allows for shrinking large scale networks of varying resource constraints and improves the overall performance of the neural networks (Abstract, Section 5). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s filing to combine the shrinkage functions of Gordon with a combination of Liu and Molchanov to arrive at more efficient, better-performing networks.
In regards to claim 3: While Liu teaches scaling factors and multiple layers, it fails to explicitly teach “wherein the neural network includes a multitude of sequences of the first and second layers, the complexity of the first layers being ascertained as a function of the sum of the indicator functions, standardized to a number of the calculated weightings of the first layer, the current complexity being ascertained as the sum across the complexities of the first layers, which is multiplied in each case with a complexity of an immediately preceding first layer of the respective first layer, and multiplied with the number of parameters or multiplications from the respective first layer.” However, see above Rejection of claim 2 where Gordon teaches the indicator functions, summation, and multiplication by either operations (multiplications) or model size (number of parameters). See also Section 4.2, Equation 7 for the summation being applied across all layers. See above rejection how the combination of Liu, Molchanov, and Gordon would have been obvious to one of ordinary skill in the art at the time of the applicant’s filing.
In regards to claim 7: While Liu teaches scaling factors and multiple layers, it fails to explicitly teach “wherein one of the first layers is connected via a bridging connection to a further preceding layer of the neural network,” However, Gordon teaches their method accounts for various network topologies, (Page 1588, Section 3.1).
“the indicator function being applied to a sum of the scaling factors of two preceding layers.” See above Rejection of claim 3 how Gordon teaches all layers being used in the summation. See above rejection how the combination of Liu, Molchanov, and Gordon would have been obvious to one of ordinary skill in the art at the time of the applicant’s filing.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
ML Glossary, 2017: Demonstrates weighted summation of input variables as known in the art at the time of the Applicant’s filing. https://ml-cheatsheet.readthedocs.io/en/latest/nn_concepts.html#weighted-input
Forum Discussion of affine transformations, 2016-2018: Demonstrates the common nature of affine transformation within neural network processing, and how a basic affine transformation is essentially a linear function with translation. https://datascience.stackexchange.com/questions/13405/what-is-affine-transformation-in-regard-to-neural-networks
Liu, Zhuang, et al. "Rethinking the value of network pruning." arXiv preprint arXiv:1810.05270 (2018). Highlights the similarity on performance between channel- and weight-level pruning. Further demonstrates both would have been state of the art and their combination obvious.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRIFFIN T BEAN whose telephone number is (703)756-1473. The examiner can normally be reached M - F 7:30 - 4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRIFFIN TANNER BEAN/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights