SHARPNESS-AWARE MINIMIZATION FOR ROBUSTNESS IN SPARSE NEURAL NETWORKS

Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Status of Claims Applicant’s submission filed on 08.24.2026 has been entered. The status of claims is as follows: Claims 1-5, 7-22 remain pending in the application. Claim 6 is cancelled. Claims 21-22 are new. Claims 1, 10-17 and 19-20 are amended. Response to Arguments In reference to the claim rejection under 35 U.S.C 101: Applicant’s arguments, see Remarks, filed 08.24.2026, with respect to the rejections under 35 U.S.C 101 have been fully considered and are persuasive. The rejections of claims under 35 U.S.C 101 has been withdrawn. In reference to the claim rejection under 35 U.S.C 103: Argument 1: Applicant asserts in Remarks pg. 5 that neither Tian nor Foret teaches the claimed sequence of SAM-based finetuning, removal of updated trained parameters to form a sparse neural network, and subsequent pruning-free finetuning of that sparse neural network until convergence. According to Applicant, Foret merely discloses a generic SAM optimization loop, while Tian separately discloses pruning, and their combination does not establish the claimed functional and temporal relationship between parameter updating, pruning and post-pruning convergence. Response to Argument 1: Applicant’s argument is not persuasive because a rejection under 35 U.S.C 103 does not require every claimed limitation to be disclosed by a single reference. Rather, the rejection relies on the combined teachings of the cited references. In particular, Na was relied upon for the disputed pruning-related limitations, including removal of updated trained parameters to produce parameters of a sparse neural network and subsequent finetuning of the sparse neural network, while Foret was relied upon for the SAM-related limitations that it expressly teaches. Accordingly, Applicant’s argument that Foret alone does not disclose the entire claimed sequence does not address the rejection as actually presented. See detailed claim mappings for the specific teachings relied upon from each reference. Argument 2: Applicant asserts in Remarks pg. 7 that Na fails to disclose the amended limitations because its unstructured IMP experiments prune individual weights rather than remove neurons to produce a sparse neural network with fewer neurons. Applicant further contends that Na’s rewinding implementation resets model parameters after pruning, rather than continuously updating the same parameters, and that Na’s structured pruning experiments use SAM only for teacher-model fine-tuning rather than for the pruning procedure itself. Applicant also asserts that Na does not disclose the final fine-tuning of the sparse neural network without further pruning until convergence, but instead uses predetermined fixed training schedules. Response to Argument 2: Applicant’s arguments regarding Na have been considered but are not persuasive to the extend they contend that Na fails to disclose continued training of the remaining parameters after pruning. Applicant’s argument regarding resetting of parameters relies on Na’s rewinding implementation while overlooking Na’s expressly disclosed standard pruning implementation. In particular, Na teaches both pruning with and without rewinding and states that, in the standard pruning setting, training continues immediately after each pruning iteration without resetting the remaining weights to the pretrained initialization. Na further teaches continued optimization of the remaining parameters following pruning rather than requiring that the parameters be reset after each pruning operation. (See detailed mappings below.) Applicant’s arguments regarding Na are not persuasive because they address limitations for which Na was not relied upon in the rejection. The rejection does not require Na, by itself, to disclose every limitation of the claimed invention. Rather, Na was cited for the limitations that it expressly teaches, including continued training or optimization in connection with pruning and use of SAM during the pruning process. A new reference, Zhang et al. (“MetaSelection: Metaheuristic Sub-Structure Selection for Neural Network Pruning Using Evolutionary Algorithm”) was separately cited for the claimed final fine-tuning procedure performed after pruning, including fine-tuning the resulting sparse neural network without further pruning. Accordingly, Applicant’s arguments that Na alone fails to disclose the final fine-tuning stage, or other limitations mapped to Zhang, do not address the rejection as actually presented. The cited references must be considered in combination, with each reference relied upon for the particular teachings identified in the claim mapping. Argument 3: Applicant argues that Hendrycks does not cure the deficiencies of Tian and Foret because Hendrycks merely provides the ImageNet-C corruption benchmark for evaluating robustness, rather than teaching the claimed SAM-based pruning and subsequent fine-tuning process. Applicant further argues that Hendrycks does not disclose claim 12’s comparative accuracy limitation because its CondenseNet results do not compare a sparse neural network generated using SAM during pruning with a baseline sparse neural network generated without SAM, nor establish that the compared models originate from the same dense neural network or are evaluated on the same acquired inputs. According to Applicant, Hendrycks therefore fails to provide the claimed baseline sparse network comparison. Response to Argument 3: Applicant’s arguments have been fully considered but are not persuasive. Applicant is attempting to argue the references in a vacuum. Hendrycks was not cited for all the limitations applicant discussed and has no requirement to meet these limitations. The Hendrycks reference was used to show that the acquired inputs include one or more non-adversarial corruptions not present in the training inputs. This is all that Hendrycks needs to disclose and it is not required to meet limitations it has not been cited for. Regarding the disputed limitations, Na was cited to teach them (see detailed mappings below). Therefore, the rejection is maintained as shown in the previous office action. Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. Claim(s) 1-5, 7, 12-21 are rejected under 35 U.S.C. 103 as being unpatentable over Tian et al. (“Heuristic Method for Minimizing Model Size of CNN by Combining Multiple Pruning Techniques”) in view of Foret et al. (“SHARPNESS-AWARE MINIMIZATION FOR EFFICIENTLY IMPROVING GENERALIZATION”) and further in view of Na et al. (“Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models”) and Zhang et al. (“MetaSelection: Metaheuristic Sub-Structure Selection for Neural Network Pruning Using Evolutionary Algorithm”) Regarding claim 1, Tian discloses: applying, by a dense neural network, trained parameters to training inputs to predict outputs for a task; (Tian, Pg. 8, Algorithm 1: PNG media_image1.png 852 950 media_image1.png Greyscale Pg. 7, Section 3.2: “The algorithm first loads the pre-trained model init_model as the input for the algorithm. It initializes the reduction ratios of two pruning methods of the network slimming NS_ratio and the deep compression DC_ratio to the corresponding step_width values (lines 1–2). The lines 3–4 describe the procedures of Step 1 in Figure 5 while iterating the network slimming until the model’s accuracy Acc becomes lower than the target accuracy Acc_th. The algorithm derives the reduction ratio for the network slimming method in NS_ratio. Here, the model NS_model derived from the iterations above is the minimal compressed model by the network slimming.”) [Examiner’s note: Tian discloses in Algorithm 1 Lines 3-4 the concept of applying, by a dense neural network (i.e., the fully connected neural network before slimming process), trained parameters (i.e., the pre-trained network model init_model) to training inputs (i.e., the init_model, NS_ratio) to predict outputs (i.e., the result slimmed model) for a task] Tian fails to disclose: performing one or more finetuning training iterations by: computing a sharpness-aware minimization loss function to produce a gradient that reduces differences between the predicted outputs and ground truth labels corresponding to the training inputs; updating the trained parameters based on the gradient; and performing one or more additional finetuning training iterations while continuing to compute the sharpness-aware minimization loss function to produce a respective gradient and update the trained parameters based on the respective gradient, wherein the performing of at least one iteration of the one or more additional finetuning training iterations comprises removing at least a portion of the updated trained parameters to produce second parameters for a sparse neural network comprising a reduced number of neurons compared with the dense neural network; and after the additional finetuning training iterations are completed, finetuning the sparse neural network without further pruning until convergence is achieved. However, Foret explicitly discloses: computing a sharpness-aware minimization loss function to produce a gradient that reduces differences between the predicted outputs and ground truth labels corresponding to the training inputs; (Foret, Pg. 3, Theorem 1, ¶[1]: “we propose to select parameter values by solving the following Sharpness- Aware Minimization (SAM) problem: PNG media_image2.png 63 799 media_image2.png Greyscale .”, Pg. 18, Section C.6: “we trained a WideResNet on the CIFAR datasets using a variant of SAM that performs several iterations of projected gradient ascent to estimate PNG media_image3.png 25 161 media_image3.png Greyscale . We report the evolution of PNG media_image4.png 25 237 media_image4.png Greyscale during training (where L stands for the training error rate computed on the current batch) in Figure 7”, Pg. 20, Figure 7: “Evolution of PNG media_image4.png 25 237 media_image4.png Greyscale vs. training step, for different numbers of inner projected gradient steps.” PNG media_image5.png 289 968 media_image5.png Greyscale ) [Examiner’s note: Foret teaches computing the Sharpness-Aware Minimization (SAM) using projected gradient steps to evaluate PNG media_image4.png 25 237 media_image4.png Greyscale , which is a loss-based sharpness measure L, where L is the training error computed on the current batch. This training error reflects the discrepancy between the model’s prediction and the ground truth labels. Therefore, the gradient computed from the SAM loss is used to update the model parameters in a direction that reduces that prediction error] updating the trained parameters based on the gradient; and (Foret, Pg. 4, Algorithm 1: PNG media_image6.png 450 660 media_image6.png Greyscale ) [Examiner’s note: “the trained parameters” is being interpreted as the weights] It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Tian and Foret. Tian teaches an algorithm to combine both the structured and the unstructured pruning methods while maintaining the target accuracy that is configured by its application. Foret teaches sharpness-aware minimization for efficiently improving generalization. One of ordinary skill would have motivation to combine Tian and Foret to provide a new efficient, scalable, and effective approach to improving model generalization ability that directly leverages the geometry of the loss landscape and its connection to generalization, and is powerfully complementary to existing techniques. However, Na explicitly discloses: performing one or more finetuning training iterations by: (Na, Pg. 4, Section 3.1: “Given a network N and weights w, we wish to prune a subset of individual weights to leave only a subset, w’. A successfully pruned model has PNG media_image7.png 29 96 media_image7.png Greyscale while retaining good performance on the task(s) of interest.”, Pg. 4, Section 3.1.1: “Typically, w’ is found in an iterative process where N is trained and pruned incrementally either until some target sparsity is reached or until some larger-than-desired performance drop is observed, and a common criterion picks weights with the smallest absolute magnitude to be pruned at each iteration”) performing one or more additional finetuning training iterations while continuing to compute the sharpness-aware minimization loss function to produce a respective gradient and update the trained parameters based on the respective gradient, (Na, Pg. 4, Section 3.1: “Given a network N and weights w, we wish to prune a subset of individual weights to leave only a subset, w’. A successfully pruned model has PNG media_image7.png 29 96 media_image7.png Greyscale while retaining good performance on the task(s) of interest.”, Pg. 4, Section 3.1.1: “Typically, w’ is found in an iterative process where N is trained and pruned incrementally either until some target sparsity is reached or until some larger-than-desired performance drop is observed, and a common criterion picks weights with the smallest absolute magnitude to be pruned at each iteration.”, Pg. 4, Section 3.2.1: “Sharpness-Aware Minimization uses a minimax objective to push models into flat minima during gradient descent. At each step of optimization, we estimate gradients based on the parameters yielding the highest loss in an lp neighborhood around the current parameters. Specifically, we look for: PNG media_image8.png 50 329 media_image8.png Greyscale where the maximization region is an lp ball with radius p for p = 2 in Equation (1). The SAM minimax objective can then be defined as: PNG media_image9.png 55 377 media_image9.png Greyscale ”) wherein the performing of at least one iteration of the one or more additional finetuning training iterations comprises removing at least a portion of the updated trained parameters to produce second parameters for a sparse neural network comprising a reduced number of neurons compared with the dense neural network; and (Na, Pg. 4, Section 3.1: “Given a network N and weights w, we wish to prune a subset of individual weights to leave only a subset, w’. A successfully pruned model has PNG media_image7.png 29 96 media_image7.png Greyscale while retaining good performance on the task(s) of interest.” Pg. 4, Section 3.1.1: “Typically, w’ is found in an iterative process where N is trained and pruned incrementally either until some target sparsity is reached or until some larger-than-desired performance drop is observed, and a common criterion picks weights with the smallest absolute magnitude to be pruned at each iteration”, Pg. 5, Section 4.3.1: “We investigate the SAM procedure’s effectiveness in uncovering winning tickets (Frankle and Carbin, 2019). The IMP section of Table 1 shows that optimizing with SAM throughout iterative magnitude pruning allows pruned models to retain higher performance at reference sparsity levels when compared to models trained with vanilla Adam optimizer.”) The combination of Tian and Na are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Tian and Na before them, to modify the teachings of Tian to include the teachings of Na to improve the generalization and accuracy of the neural network while retaining the computational and storage benefits associated with sparsity, with a reasonable expectation of success because both references concern training and pruning neural networks to obtain efficient sparse models. However, Zhang explicitly discloses: after the additional finetuning training iterations are completed, finetuning the sparse neural network without further pruning until convergence is achieved. (Zhang, Pg. 2810, Fig. 2: “The pipeline of our proposed MetaSelection. At each iteration, a sub-structure is searched via evolutionary algorithm for block/channel selection. If the constraint is not satisfied, we only finetune the pruned network for several epochs to regain the accuracy and continue to prune it. Once the selected sub-structure meets the constraint, it will be finetuned until convergence (i.e., long-term finetune) and returned as the final pruned network.”, FIG.2: PNG media_image10.png 110 892 media_image10.png Greyscale ) The combination of Tian and Zhang are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Tian and Zhang before them, to modify the teachings of Tian to include the teachings of Zhang to improve and stabilize model performance after parameters have been removed. Regarding claim 2, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Na, Zhang and Foret further discloses: wherein the sharpness-aware minimization loss function produces a flat region including a minimum loss value compared with a loss function used to train the dense neural network. (Na, Pg. 4, Col. 2, Section 3.2: “To obtain models converged to flatter minima, we use Sharpness-Aware Minimization (SAM) procedure which optimizes for flatness directly.”, Pg. 4, Col. 2, Section 3.2.1: “Sharpness-Aware Minimization uses a minimax objective to push models into flat minima during gradient descent.”, Pg. 5, Col. 1, ¶[1]: “The SAM minimax objective can then be defined as: PNG media_image9.png 55 377 media_image9.png Greyscale ”) Regarding claim 3, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Foret, Na and Zhang further discloses: wherein structured pruning is used to produce the sparse neural network. (Tian, Pg. 2, ¶[3]: “In this paper, we propose a heuristic method that combines the structured and unstructured pruning methods. In our method, we control the compression ratios of the original structured and unstructured pruning methods by repeating these methods under a target accuracy.”, Pg. 4, ¶[4]: “In this paper, for the structure pruning, we focus on network slimming.”, Pg. 5, Figure 2: PNG media_image11.png 405 964 media_image11.png Greyscale ) Regarding claim 4, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Foret, Na and Zhang further discloses: wherein unstructured pruning is used to produce the sparse neural network. (Tian, Pg. 2, ¶[3]: “In this paper, we propose a heuristic method that combines the structured and unstructured pruning methods. In our method, we control the compression ratios of the original structured and unstructured pruning methods by repeating these methods under a target accuracy.”, Pg. 4, ¶[4]: “For the unstructured pruning, we focus on the weight pruning method utilized in the deep compression.”, Pg. 5, Figure 3: PNG media_image12.png 255 795 media_image12.png Greyscale ) Regarding claim 5, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Foret, Na and Zhang further discloses: wherein the training inputs are images and the dense neural network and the sparse neural network perform a classification task. (Foret, Pg. 4, Section 3: “In order to assess SAM’s efficacy, we apply it to a range of different tasks, including image classification from scratch (including on CIFAR-10, CIFAR-100, and ImageNet), finetuning pretrained models, and learning with noisy labels”) Regarding claim 7, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Foret, Na and Zhang further discloses: applying the second parameters, by the sparse neural network, to acquired inputs to predict deployed outputs for the task. (Tian, Pg. 8, Algorithm 1: PNG media_image13.png 840 980 media_image13.png Greyscale Pg. 7, Section 3.2: “The lines 8–9 correspond to Step 2 in Figure 5 where the procedure relates to the margin. Through the margin calculation on this reduction ratio, we obtain a compressed model drawn back due to the margin value and the algorithm updates NS_model to the model. The lines 10–12 correspond to Step 3 in Figure 5 iterating the deep compression until the model accuracy Acc becomes lower than the target accuracy acc_th again. Thus, the minimal model is derived. Finally, the algorithm returns the f inal_model as the output of the compression process (line 15). Moreover, we also assign multiple values to the margin by gradually decreasing the initial_margin. Furthermore, through repeating the procedures of lines 8–13, we will achieve the model final_model as the minimized model regarding the size.”) [Examiner’s note: Tian discloses in Algorithm 1 Lines 10-11 the concept of applying, by a sparse neural network (i.e., the slimmed network), the second parameters (i.e., NS_model, DC_ratio) to second training inputs to predict second outputs (i.e., the final model final_model)] Regarding claim 12, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 7 (as shown in the rejections above). Tian in view of Foret, Na and Zhang further discloses: wherein accuracy of the deployed outputs is greater than an accuracy of baseline outputs predicted by a baseline sparse neural network processing the acquired inputs, and wherein the baseline sparse neural network is generated by pruning the dense neural network without using the sharpness-aware minimization loss function to train or finetune parameters during pruning. (Na, Pg. 6, Section 4.3.2: “We also investigate SAM and vanilla Adam optimizers in the standard pruning setting, where we continue training immediately after pruning in each iteration, without resetting remaining weights to pre-trained BERT-base weights.”, Pg. 7, Section 4.3.3, Table 2: “We report evaluation metrics comparing pruned models obtained using teacher models finetuned with Adam and SAM optimizers. Pruned are trained to 95% sparsity using optimal values for ג and finetuning learning rate from Xia et al. (2022)’s structured pruning setting.”, Table 2: PNG media_image14.png 118 424 media_image14.png Greyscale , Pg. 7, Col. 2, ¶[2]: “SAM outperforming a vanilla Adam optimizer in this setting is a particularly desirable goal from a practical standpoint”) Regarding claim 13, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Foret, Na and Zhang further discloses: wherein at least one of the steps of applying, computing, updating, or removing is performed on a server or in a data center and the second parameters are streamed to a user device. (Tian, Pg. 3, ¶[2]: “To overcome the overhead issue, there are three options of the accelerators for the implementation of the convolution process in CNN: CPUs (central processing units) that include multiply-and-add instructions [27–29], GPUs (graphics processing units) that execute massively parallel operations [30,31], and FPGAs (field programmable gate arrays) that implement multiple operators in hardware… Therefore, we need some technique to provide greater energy-efficiency for the platform. In comparison to GPUs, FPGAs are considered an energy-efficient platform for the CNN. However, FPGAs can have relatively limited amount of memory and hardware resources for the computation. Thus, it is a critical problem to minimize the CNN model before implementation while maintaining high accuracy.”) Regarding claim 14, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Na, Zhang and Foret further discloses: wherein at least one of the steps of applying, computing, updating, or removing is performed within a cloud computing environment. (Foret, Pg. 5, Section 3.1, ¶6]: “We train all models on ImageNet for up to 400 epochs using a Google Cloud TPUv3 and report top-1 and top-5 test error rates for each experimental condition (mean and 95% confidence interval across 5 independent runs).”) Regarding claim 15, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Na, Zhang and Foret further discloses: wherein the sparse neural network is employed in a machine, robot, or autonomous vehicle. (Tian, Pg. 2, ¶[1]: “CNN-based applications need to challenge availability for their implementation on edge devices where the amount of resources are constrained [17,18] such as smartphones or vehicles.”, Pg. 2, Section 2.1: “Object detection has been a long-standing and fundamental research theme in the computer vision field that detects object instances of a certain class such as a person, animal or vehicle in digital images.”) Regarding claim 16, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Na, Zhang and Foret further discloses: wherein at least one of the steps of applying, computing, updating, or removing is performed on a virtual machine comprising a portion of a graphics processing unit. (Foret, Pg. 5, Section 3.1, ¶[3]: “Our implementations utilize JAX (Bradbury et al., 2018), and we train all models on a single host having 8 NVidia V100 GPUs5”) Regarding independent claims 17 and 19, they are rejected under the same rationale with claim 1 because they are analogous claims. Dependent claims 18 and 20 are rejected under the same rationale with claim 2 because they are analogous claims. Regarding claim 21, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 1 (as shown in the rejections above). Tian in view of Na, Zhang and Foret further discloses: wherein removal of at least the portion of the updated trained parameters does not cause a loss to increase by more than a threshold amount. (Na, Pg. 2, Col. 2, ¶[1]: “Intuitively, model parameters in neighborhoods having uniformly low loss values should be more robust to perturbations, such as model compression, compared to models in sharper regions, since changes to the parameters should lead to minimal change in loss with respect to the main training objective”, Pg. 2, Col. 2, ¶[4]: “In order to optimize for flat minima we employ sharpness-aware minimization (SAM; Foret et al. (2021)). SAM optimizes for parameters that lie in uniformly low-loss neighborhoods (i.e., flat minima), rather than those that have low loss with respect to the main training objective but that may otherwise lie in sharp regions of the loss landscape”, Pg. 4, Section 3.1: “Given a network N and weights w, we wish to prune a subset of individual weights to leave only a subset, w’. A successfully pruned model has PNG media_image7.png 29 96 media_image7.png Greyscale while retaining good performance on the task(s) of interest.”, Pg. 4, Section 3.1.1: “Typically, w’ is found in an iterative process where N is trained and pruned incrementally either until some target sparsity is reached or until some larger-than-desired performance drop is observed, and a common criterion picks weights with the smallest absolute magnitude to be pruned at each iteration”) Claim(s) 8-11 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Tian et al. (“Heuristic Method for Minimizing Model Size of CNN by Combining Multiple Pruning Techniques”) in view of Foret et al. (“SHARPNESS-AWARE MINIMIZATION FOR EFFICIENTLY IMPROVING GENERALIZATION”), Na et al. (“Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models”) and Zhang et al. (“MetaSelection: Metaheuristic Sub-Structure Selection for Neural Network Pruning Using Evolutionary Algorithm”) and further in view of Hendrycks & Dietterich (“BENCHMARKING NEURAL NETWORK ROBUSTNESS TO COMMON CORRUPTIONS AND PERTURBATIONS”) Regarding claim 8, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 7 (as shown in the rejections above). Tian in view of Na, Zhang and Foret fails to disclose: wherein the acquired inputs include corruptions that are not present in the training inputs. However, Hendrycks explicitly discloses: wherein the acquired inputs include corruptions that are not present in the training inputs. (Hendrycks, Pg. 1, ¶[3]: “To create IMAGENET-C, we introduce a set of 75 common visual corruptions and apply them to the ImageNet object recognition challenge”, Pg. 3, Section 4.1: “The IMAGENET-C benchmark consists of 15 diverse corruption types applied to validation images of ImageNet. The corruptions are drawn from four main categories— noise, blur, weather, and digital—as shown in Figure 1… IMAGENET-C images are saved as lightly compressed JPEGs; this implies an image corrupted by Gaussian noise is also slightly corrupted by JPEG compression. Our benchmark tests networks with IMAGENET-C images, but networks should not be trained on these images. Networks should be trained on datasets such as ImageNet and not be trained on IMAGENET-C corruptions”) [Examiner’s note: Hendrycks teaches the acquired inputs include corruptions (i.e., the IMAGENET-C) that are not present in the training inputs (i.e., “Networks should be trained on datasets such as ImageNet and not be trained on IMAGENET-C corruptions”)] It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Tian, Foret and Hendrycks. Tian teaches an algorithm to combine both the structured and the unstructured pruning methods while maintaining the target accuracy that is configured by its application. Foret teaches sharpness-aware minimization for efficiently improving generalization. Hendrycks teaches benchmarks for image classifier robustness. One of ordinary skill would have motivation to combine Tian, Foret and Hendrycks to enhance corruption and perturbation robustness, which is an important goal for computer vision and machine learning. It is also essential for creating deep learning systems that can be deployed in safety-critical applications. Regarding claim 9, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 7 (as shown in the rejections above). Tian in view of Na, Zhang and Foret fails to disclose: wherein the acquired inputs comprise images that are not included in the training inputs. However, Hendrycks explicitly discloses: wherein the acquired inputs comprise images that are not included in the training inputs. . (Hendrycks, Pg. 1, ¶[3]: “To create IMAGENET-C, we introduce a set of 75 common visual corruptions and apply them to the ImageNet object recognition challenge”, Pg. 3, Section 4.1: “The IMAGENET-C benchmark consists of 15 diverse corruption types applied to validation images of ImageNet. The corruptions are drawn from four main categories— noise, blur, weather, and digital—as shown in Figure 1… IMAGENET-C images are saved as lightly compressed JPEGs; this implies an image corrupted by Gaussian noise is also slightly corrupted by JPEG compression. Our benchmark tests networks with IMAGENET-C images, but networks should not be trained on these images. Networks should be trained on datasets such as ImageNet and not be trained on IMAGENET-C corruptions”) [Examiner’s note: Hendrycks teaches the acquired inputs include corruptions (i.e., the IMAGENET-C) that are not present in the training inputs (i.e., “Networks should be trained on datasets such as ImageNet and not be trained on IMAGENET-C corruptions”)] Regarding claim 10, the combination of Tian and Foret Tian, Na, Zhang and Foret discloses all the limitations of claim 7 (as shown in the rejections above). Tian in view of Na, Zhang and Foret fails to disclose: wherein the acquired inputs are captured images and include corruptions result from at least one of camera effects or environmental conditions. However, Hendrycks explicitly discloses: wherein the acquired inputs are captured images and include corruptions result from at least one of camera effects or environmental conditions. (Hendrycks, Pg. 3, Figure 1: “Our IMAGENET-C dataset consists of 15 types of algorithmically generated corruptions from noise, blur, weather, and digital categories. Each type of corruption has five levels of severity, resulting in 75 distinct corruptions. See different severity levels in Appendix B” PNG media_image15.png 557 788 media_image15.png Greyscale Pg. 4, ¶[1]: “The first corruption type is Gaussian noise. This corruption can appear in low-lighting conditions. Shot noise, also called Poisson noise, is electronic noise caused by the discrete nature of light itself. Impulse noise is a color analogue of salt-and-pepper noise and can be caused by bit errors. Defocus blur occurs when an image is out of focus. Frosted Glass Blur appears with “frosted glass” windows or panels. Motion blur appears when a camera is moving quickly. Zoom blur occurs when a camera moves toward an object rapidly. Snow is a visually obstructive form of precipitation. Frost forms when lenses or windows are coated with ice crystals. Fog shrouds objects and is rendered with the diamond-square algorithm. Brightness varies with daylight intensity. Contrast can be high or low depending on lighting conditions and the photographed object’s color”) Regarding claim 11, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 7 (as shown in the rejections above). Tian in view of Na, Zhang and Foret fails to disclose: wherein the acquired inputs are images that include at least one of Gaussian noise, shot noise, impulse noise, defocus blur, transmissive material blur, motion blur, brightness, contrast, pixelation, compression artifacts. However, Hendrycks explicitly discloses: wherein the acquired inputs are images that include at least one of Gaussian noise, shot noise, impulse noise, defocus blur, transmissive material blur, motion blur, brightness, contrast, pixelation, compression artifacts. (Hendrycks, Pg. 3, Figure 1: “Our IMAGENET-C dataset consists of 15 types of algorithmically generated corruptions from noise, blur, weather, and digital categories. Each type of corruption has five levels of severity, resulting in 75 distinct corruptions. See different severity levels in Appendix B” PNG media_image15.png 557 788 media_image15.png Greyscale Pg. 4, ¶[1]: “The first corruption type is Gaussian noise. This corruption can appear in low-lighting conditions. Shot noise, also called Poisson noise, is electronic noise caused by the discrete nature of light itself. Impulse noise is a color analogue of salt-and-pepper noise and can be caused by bit errors. Defocus blur occurs when an image is out of focus. Frosted Glass Blur appears with “frosted glass” windows or panels. Motion blur appears when a camera is moving quickly. Zoom blur occurs when a camera moves toward an object rapidly. Snow is a visually obstructive form of precipitation. Frost forms when lenses or windows are coated with ice crystals. Fog shrouds objects and is rendered with the diamond-square algorithm. Brightness varies with daylight intensity. Contrast can be high or low depending on lighting conditions and the photographed object’s color”) Regarding claim 22, the combination of Tian, Na, Zhang and Foret discloses all the limitations of claim 7 (as shown in the rejections above). Tian in view of Na, Zhang and Foret fails to disclose: wherein the acquired inputs include one or more non- adversarial corruptions not present in the training inputs, and However, Hendrycks explicitly discloses: wherein the acquired inputs include one or more non- adversarial corruptions not present in the training inputs, and (Hendrycks, Abstract: “Then we propose a new dataset called IMAGENET-P which enables researchers to benchmark a classifier’s robustness to common perturbations. Unlike recent robustness research, this benchmark evaluates performance on common corruptions and perturbations not worst-case adversarial perturbations”, Pg. 3, Section 4.1: “The IMAGENET-C benchmark consists of 15 diverse corruption types applied to validation images of ImageNet. The corruptions are drawn from four main categories—noise, blur, weather, and digital—as shown in Figure 1… These algorithmically generated corruptions are applied to the ImageNet (Deng et al., 2009) validation images to produce our corruption robustness dataset IMAGENET-C…. Networks should be trained on datasets such as ImageNet and not be trained on IMAGENET-C corruptions.”) wherein accuracy of the deployed outputs on the acquired inputs is greater than accuracy of outputs produced on the acquired inputs by a baseline sparse neural network generated by pruning the dense neural network without using the sharpness-aware minimization loss function to train or finetune parameters during pruning. (Na, Pg. 6, Section 4.3.2: “We also investigate SAM and vanilla Adam optimizers in the standard pruning setting, where we continue training immediately after pruning in each iteration, without resetting remaining weights to pre-trained BERT-base weights.”, Pg. 7, Section 4.3.3, Table 2: “We report evaluation metrics comparing pruned models obtained using teacher models finetuned with Adam and SAM optimizers. Pruned are trained to 95% sparsity using optimal values for ג and finetuning learning rate from Xia et al. (2022)’s structured pruning setting.”, Table 2: PNG media_image14.png 118 424 media_image14.png Greyscale , Pg. 7, Col. 2, ¶[2]: “SAM outperforming a vanilla Adam optimizer in this setting is a particularly desirable goal from a practical standpoint”) Conclusion Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMY TRAN whose telephone number is (571)270-0693. The examiner can normally be reached Monday - Friday 7:30 am - 5:00 pm EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /AMY TRAN/Examiner, Art Unit 2126 /DAVID YI/Supervisory Patent Examiner, Art Unit 2126