Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
The amendment filed 03/24/2026 has been entered.
Claims 1, 3, 10-12, and 15-17 have been amended.
Claims 13 and 14 have been cancelled.
Claims 1-12 and 15-18 remain pending in the application.
The amendment filed 03/24/2026 is sufficient to overcome the claim objection of claim 14. The previous objection has been withdrawn.
The amendment filed 03/24/2026 is sufficient to overcome the 35 U.S.C. 112(f) claim interpretation. The previous claim interpretation has been withdrawn.
The amendment filed 03/24/2026 is sufficient to overcome the 35 U.S.C. 112(a) claim rejections. The previous rejections have been withdrawn.
The amendment filed 03/24/2026 is sufficient to overcome the 35 U.S.C. 112(b) claim rejections. The previous rejections have been withdrawn.
The amendment filed 03/24/2026 is sufficient to overcome the 35 U.S.C. 101 claim rejections. The previous rejections have been withdrawn.
Response to Arguments
Argument 1, regarding the claim objections, applicant argues that the objections should be withdrawn in view of claim 14 being cancelled. Examiner agrees and the objection has been withdrawn.
Argument 2, regarding the 112(f) claim interpretation, applicant argues that the 112(f) claim interpretation of claim 15 should be withdrawn in view of amendments made to claim 15. Examiner agrees and the 112(f) claim interpretation has been withdrawn.
Argument 3, regarding the 112(a) claim rejections, applicant argues that the 112(a) rejections should be withdrawn in view of amendments made to claim 15. Examiner agrees and the rejections have been withdrawn.
Argument 4, regarding the 112(b) claim rejections, applicant argues that the 112(b) rejections should be withdrawn in view of amendments made to claim 15. Examiner agrees and the rejections have been withdrawn.
Argument 5, regarding the 101 rejections, applicant argues that the 101 rejections should be withdrawn in view of claim 14 being cancelled and amendments clarifying the improvement of optimized computational effort and/or optimized energy consumption resulting from applying the optimized model compression technique. Examiner agrees and the 101 rejections have been withdrawn.
Argument 6, regarding the prior art rejections, applicant argues that Li does not teach a process of determining metrics for a plurality of different model compression techniques via generation of a compressed model for each compression technique as clarified in amended claims 1 and 15. Applicant argues that the cited paragraphs of Li do not suggest different pruning processes separate from each other to generate a plurality of different compressed models to subsequently choose between for identifying an optimized model.
Examiner respectfully disagrees because Li teaches applying, by the at least one processor, the selected optimized model compression technique to the artificial intelligence-based model to generate a compressed artificial intelligence-based model (Li P.5, Sec.3.4, Para.1 “After pruning the filters, the performance degradation should be compensated by retraining the network.”; Li produces a compressed network after pruning and retrains it to recover accuracy (corresponds to generating a compressed artificial intelligence-based model using the optimized model compression technique). Li P.4-5, Sec 3.3 and Li P.4, figure 2, different pruning strategies result in different models, with different filters being pruned for optimal computing efficiency); and deploying the compressed artificial intelligence-based model in an industrial system to execute an artificial intelligence-based task with optimized computational effort and/or optimized energy consumption within the industrial system (Li P.3, Sec.3.1, Para.1 “Our method prunes the less useful filters from a well-trained model for computational efficiency while minimizing the accuracy drop”. Li P.3, section “Pruning Filters and Feature Maps”, a convolutional neural network is used to prune less useful filters in order to reduce operations and computation cost for layers of the network).
The full prior art rejections are outlined below.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-15 and 17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li et al. (“Pruning Filters for Efficient ConvNets” (Published 2017); hereinafter Li).
Regarding Claim 1,
Li discloses a computer-implemented method for automated determination of a model compression technique for a compression of an artificial intelligence-based model, the method comprising:
automatically providing, by at least one processor, a set of model compression techniques using an expert rule (Li P.3, Sec.3.1, Para.1 “filters from a well-trained model for computational efficiency while minimizing the accuracy drop. We measure the relative importance of a filter in each layer by calculating the sum of its absolute weights … we find l1-norm is a good criterion for data-free filter selection”; P.3, Sec.3.1, Para.2 “The procedure of pruning m filters from the ith convolutional layer is as follows: 1. For each filter … calculate the sum of its absolute kernel weights … 3. Pune m filters with the smallest sum values and their corresponding feature maps”; Li discloses a heuristic criterion (l1-norm of filter weights) to decide which filters to prune (corresponds to compression techniques) which constitutes an expert rule because it is a human-designed method that guides which filters are removed (corresponds to using an expert rule). This expert rule is applied across layers and filters, effectively providing a selection mechanism for compression actions (corresponds to automatically providing a set of model compression techniques));
determining, by the at least one processor, metrics for each model compression technique of the set of model compression techniques based on weighted constraints by generating a respective compressed model for each model compression technique (Li P.3, Sec.3.1, Para.1 “Our method prunes the less useful filters from a well-trained model for computational efficiency while minimizing the accuracy drop”; Li evaluates candidate pruning actions by computing their effect on accuracy and computational cost (corresponds to determining metrics for model compression techniques of the set of model compression techniques). These metrics represent weighted constraints insofar as the process of balancing accuracy and computational efficiency is effectively weighted (corresponds to determining metrics … based on weighted constraints). Li P.5, Sec.3.4, Para.1 “After pruning the filters, the performance degradation should be compensated by retraining the network.”; Li produces a compressed network after pruning and retrains it to recover accuracy (corresponds to generating a compressed artificial intelligence-based model using the optimized model compression technique)); selecting, by the at least one processor, an optimized model compression technique based on the determined metrics (Li P.3, Sec.3.1, Para.1 “We find that pruning the smallest filters works better in comparison with pruning the same number of random or largest filters”; discloses selecting a pruning technique that optimizes the trade-off between reduced computation and accuracy loss (corresponds to selecting an optimized model compression technique based on the determined metrics). Li P.13, Sec.6.2, Para.2 “The evaluation is conducted in Torch7 with Titan X (Pascal) GPU and cuDNN v5.1”; discloses conducting model evaluation using a Titan X GPU which includes memory which stores instructions to be executed by a computer (corresponds to a computer program product comprising instructions which, when executed by a computer, cause the computer)) applying, by the at least one processor, the selected optimized model compression technique to the artificial intelligence-based model to generate a compressed artificial intelligence-based model (Li P.5, Sec.3.4, Para.1 “After pruning the filters, the performance degradation should be compensated by retraining the network.”; Li produces a compressed network after pruning and retrains it to recover accuracy (corresponds to generating a compressed artificial intelligence-based model using the optimized model compression technique). Li P.4-5, Sec 3.3 and Li P.4, figure 2, different pruning strategies result in different models, with different filters being pruned for optimal computing efficiency); and deploying the compressed artificial intelligence-based model in an industrial system to execute an artificial intelligence-based task with optimized computational effort and/or optimized energy consumption within the industrial system (Li P.3, Sec.3.1, Para.1 “Our method prunes the less useful filters from a well-trained model for computational efficiency while minimizing the accuracy drop”. Li P.3, section “Pruning Filters and Feature Maps”, a convolutional neural network is used to prune less useful filters in order to reduce operations and computation cost for layers of the network).
Regarding Claim 2,
Li discloses the method of claim 1, wherein the weighted constraints reflect hardware or software constraints of an executing system for execution of a compressed model of the artificial intelligence-based model compressed with the model compression technique (Li P.3, Sec.3.1, Para.1 “Our method prunes the less useful filters from a well-trained model for computational efficiency”; Li’s pruning method considers computational efficiency which is disclosed to be measured in FLOPs (see Li P.6, Table 1) which reflects hardware/software constraints (corresponds to the weighted constraints reflect hardware or software constraints of an executing system for execution of a compressed model of the artificial intelligence-based model compressed with the model compression technique)).
Regarding Claim 3,
Li discloses the method of claim 1, wherein the expert rule relates the artificial intelligence-based model to each model compression technique of the set of model compression techniques based on a condition of the artificial intelligence-based model or data needed for training or executing the artificial intelligence-based model (Li P.4, Sec.3.2, Para.1 “We empirically determine the number of filters to prune for each layer based on their sensitivity to pruning”; Li applies expert rules that are layer-specific where sensitivity analysis guides the pruning amount per layer, linking the expert rules to the conditions of the artificial intelligence-based model (corresponds to the expert rule relates an artificial intelligence-based model to the model compression techniques of the set of model compression techniques based on a condition of the artificial intelligence-based model … or executing the artificial intelligence-based model)).
Regarding Claim 4,
Li discloses the method of claim 1, wherein the metrics are functions in dependence of respective values representing respective constraints, and wherein the respective values are weighted with respective weighting factors (Li P.2, Para.3 “The number of pruned filters correlates directly with acceleration by reducing the number of matrix multiplications”; Li’s metrics relate pruning actions to computational cost savings and model accuracy which are quantifiable metrics (corresponds to the metrics are functions in dependence of respective values representing respective constraints). The pruning decisions effectively combine multiple constraints (accuracy and FLOPs) to produce metrics which consider balancing accuracy and computational efficiency and are effectively weighted (corresponds to the respective values are weighted with respective weighting factors)).
Regarding Claim 5,
Li discloses the method of claim 4, wherein the functions describe linear, exponential, polynomial, fitted, or fuzzy relations (Li P.2, Para.3 “We conduct sensitivity analysis for convolutional layers”; P.4, Sec.3.2, Para.1 “layers that maintain their accuracy as filters are pruned away correspond to layers with larger slopes”; Li measures how accuracy varies with pruning, effectively capturing a fitted functional relationship between pruning extent and resulting performance (corresponds to the functions describe … fitted … relations)).
Regarding Claim 6,
Li discloses the method of claim 4, wherein the functions vary depending on the weighted constraints (Li P.4, Sec.3.2, Para.1 “For layers that are sensitive to pruning, we prune a smaller percentage of these layers or completely skip pruning them.”; Li’s metric relationship of accuracy vs. pruning amount varies across layers depending on sensitivity which serves as a weighted constraint. More sensitive layers are pruned less; less sensitive layers are pruned more (corresponds to the functions vary depending on the weighted constraints)).
Regarding Claim 7,
Li discloses the method of claim 1, wherein the metrics are relative to a reference metric of the artificial intelligence-based model (Li P.6, Table 1 discloses evaluating metrics such as accuracy and FLOPs relative to the original uncompressed model (corresponds to the metrics are relative to a reference metric of the artificial intelligence-based model; for more please see Li P.6, Table 1)).
Regarding Claim 8,
Li discloses the method of claim 1, wherein the weighted constraints for building the metrics depend on hardware and software framework conditions of a system or a device the artificial intelligence-based model is used in (Li P.2, Para.3 “Compared to pruning weights across the network, filter pruning is a naturally structured way of pruning without introducing sparsity and therefore does not require using sparse libraries or any specialized hardware”; P.1, Abstract “this approach does not need the support of sparse convolution libraries and can work with existing efficient BLAS libraries for dense matrix multiplications”; Li’s constraint evaluation accounts for hardware and software compatibility, e.g., choosing pruning strategies that optimize runtime using existing libraries (corresponds to the weighted constraints for building the metrics depend on hardware and software framework conditions of a system or a device the artificial intelligence-based model is used in)).
Regarding Claim 9,
Li discloses the method of claim 1, wherein a respective weighting factor for a respective weighted constraint of the weighted constraints depends on an analysis type the artificial intelligence-based model is used in (Li P.4, Sec.3.2, Para.1 “We empirically determine the number of filters to prune for each layer based on their sensitivity to pruning”; Lie varies the pruning amount based on layer sensitivity which is a property of the model and its intended performance context. Layer-dependent pruning serves as an analysis type, that is, adjustment of pruning decisions based on the model’s layers’ sensitivity demonstrates an analysis type, e.g., more sensitive layers are pruned less and less sensitive layers are pruned more (corresponds to a respective weighting factor for a respective weighted constraint of the weighted constraints depends on an analysis type the artificial intelligence-based model is used in)).
Regarding Claim 10,
Li discloses the method of claim 1, wherein the selecting of the optimized model compression technique further comprises optimizing the metrics for each model compression technique of the set of model compression techniques over the weighted constraints (Li P.3, Sec.3.1, Para.1 “We find that pruning the smallest filters works better in comparison with pruning the same number of random or largest filters”; Li evaluates different pruning strategies (smallest filters, random filters, largest filters) using accuracy as a metric. The selection of the best performing strategy constitutes an optimization over competing objectives (accuracy vs. computational reduction). This comparison effectively selects and optimizes the best tradeoff between accuracy and computational efficiency (corresponds to the selecting of the optimized model compression technique further comprises optimizing the metrics for each model compression technique of the model compression techniques over the weighted constraints)).
Regarding Claim 11,
Li discloses the method of claim 10, wherein, in the optimizing of the metrics for each model compression technique of the set of model compression techniques over the weighted constraints, at least one weighted constraint is fixed Li P.3, Sec.3.1, Para.1 “Our method prunes the less useful filters from a well-trained model for computational efficiency while minimizing the accuracy drop”; Li treats accuracy as a constraint that must be maintained while pruning for computational efficiency. I.e., one constraint (accuracy) is fixed, and the pruning decision are made to satisfy it while optimizing another metric, e.g., FLOPs (corresponds to in the optimizing of the metrics for each model compression technique of the model compression techniques over the weighted constraints, at least one weighted constraint is fixed)).
Regarding Claim 12,
Li discloses the method of claim 10, wherein, in the optimizing of the metrics for each model compression technique of the set of model compression techniques over the constraints, an optimization method is used (Li P.4, Sec.3.2, Para.1 “To understand the sensitivity of each layer, we prune each layer independently and evaluate the resulting pruned network’s accuracy on the validation set … We empirically determine the number of filters to prune for each layer based on their sensitivity to pruning”; Li’s sensitivity analysis is the method used to guide pruning decisions which serves the function of determining how to select pruning levels to maximize computational efficiency without violating the accuracy constraint (corresponds to the optimizing of the metrics for each model compression technique of the model compression techniques over the constraints, an optimization method is used)).
Regarding Claim 15,
Claim 15 is rejected on the same grounds as claim 1.
Per claim 15, Li discloses an apparatus of an automation environment, the apparatus comprising:
a logic component1 (Li P.13, Sec.6.2, Para.2 “The evaluation is conducted in Torch7 with Titan X (Pascal) GPU and cuDNN v5.1”; discloses conducting model evaluation using a Titan X GPU which includes a memory and processor executes instructions stored by the memory (corresponds to a logic component)).
Regarding Claim 17,
Li discloses the method of claim 10, wherein the optimizing of the metrics for each model compression technique of the set of model compression techniques over the weighted constraints is over respective values representing the respective constraints or over parameters of the respective model compression techniques influencing the respective value representing the respective constraint (Li P.3, Sec.3.1, Para.1 “We measure the relative importance of a filter in each layer by calculating the sum of its absolute weights … Filters with smaller kernel weights tend to produce feature maps with weak activations”; discloses calculating l1-norm values of each filter which determines a filter’s impact on output accuracy and determines which filters can be pruned (corresponds to optimizing of the metrics for each model compression technique of the model compression techniques over the weighted constraints is over respective values representing the respective constraints); P.3, Sec.3.1, Para.2 “The procedure of pruning m filters from the ith convolutional layer is as follows: 1. For each filter … calculate the sum of its absolute kernel weights … 3. Pune m filters with the smallest sum values and their corresponding feature maps”; discloses that the parameters of Li’s model compression technique are the filters themselves. These parameters directly influence the computed l1-norm values, which represent the effect on accuracy or constraint. Li’s process prunes filters based on their calculated impact (corresponds to over parameters of the respective model compression techniques influence the respective value representing the respective constraint)).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (“Pruning Filters for Efficient ConvNets” (Published 2017); hereinafter Li) in view of Cella et al. (US 20190121348 A1 (Published 2019); hereinafter Cella).
Regarding Claim 16,
Li discloses the apparatus of claim 15, but appears to not disclose explicitly the limitations of claim 16.
However, Cella teaches wherein the apparatus is an edge device, and wherein the automation environment is an industrial automation environment (Cella Specification [0054] “These methods and systems include methods, systems, components, devices, workflows, services, processes, and the like that are deployed in various configurations and locations, such as: (a) at the ‘edge’ of the Internet of Things, such as in the local environment of a heavy industrial machine”; discloses a machine learning process which is performed on an edge device such as a heavy industrial machine (corresponds to the apparatus is an edge device of an industrial automation environment; for more please see Cella Specification [0055])).
Li and Cella are considered to be analogous to the claimed invention because they are in the same field of utilizing machine learning models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Cella. Doing so could assist in extending the application of compression techniques disclosed by Li to models stored on edge devices in industrial environments, as suggested by Cella (Cella Specification [0054] “These methods and systems include a range of ways for providing improved data include a range of methods and systems for providing improved data collection, as well as methods and systems for deploying increased intelligence at the edge, in the network, and in the cloud or premises of the controller of an industrial environment”).
Regarding Claim 18,
Li discloses the method of claim 12, but appears to not disclose explicitly the remaining limitations of claim 18.
However, Cella teaches wherein the optimization method comprises a gradient descent method, a genetic algorithm based method, or a machine learning classification method (Cella Specification [0886] “A further embodiment of any of the foregoing embodiments of the present disclosure may include situations wherein the swarm optimization algorithm is one or more types of Genetic Algorithms”; discloses performing a swarm optimization algorithm which applies a genetic algorithm (corresponds to the optimization method comprises …, or a machine learning classification method)).
Li and Cella are considered to be analogous to the claimed invention because they are in the same field of utilizing machine learning models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li to incorporate the teachings of Cella. Doing so could improve optimization techniques when applying the compression and optimizations techniques disclosed by Li in a system using genetically optimized sensors to gather data, as suggested by Cella (Cella Specification [0873] “Self-organizing the distribution of the mobile data collector unit and the one or more other mobile data collector units at the target location, in some implementations, can comprise utilizing a swarm optimization algorithm to allocate areas of sensor responsibility amongst the mobile data collector unit and the one or more other mobile data collector units”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHAN MOUNDI whose telephone number is (703)756-1547. The examiner can normally be reached 8:30 A.M. - 5 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.M./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
1 For the purposes of further examination, a “logic component” is being interpreted as part of a computer processor.
Prosecution Insights