Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3, 8, 10, 14, 16, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mehnert (US 2024/0020512) in view of Saniee et al. (Saniee) (US 2022/0318631).
Regarding claims 1 and 14, Mehnert discloses an apparatus for pruning a neural network model, comprising:
one or more processors ([0029], [0030] a stored program is executed by a computer); and
one or more memories coupled with the one or more processors and storing processor-executable code that, when executed by the one or more processors ([0029], [0030] a stored program is executed by a computer), is configured to cause the apparatus to:
iteratively remove, via structural pruning, one or more nodes and one or more layers of the neural network model ([0016], structural pruning is performed, [0026] iterative processing is performed); and
remove, via weight pruning after the structural pruning, one or more weights of the neural network model by iteratively masking connections of a group of connections ([0019], weight masking is performed).
Merhnert is silent about remove, via weight pruning, one or more weights of the neural network model by iteratively masking each of a group of connections with a smallest weight based on a respective absolute value of each connection.
Saniee from the same or similar field of endeavor discloses remove, via weight pruning, one or more weights of the neural network model by iteratively masking each of a group of connections with a smallest weight based on a respective absolute value of each connection ([0008], the pruning process is performed iteratively, [0073] weights with the lowest absolute value are removed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Saniee into the teachings of Mehnert for reducing the parameter count for processing by the neural network.
Regarding claims 3 and 16, Mehnert discloses wherein the structural pruning is performed at each iteration of a group of structural pruning iterations ([0016], structural pruning is performed, [0026] iterative processing is performed).
Mehnert is silent about execution of the processor-executable code further causes the apparatus to reset network weights to seed values at each iteration of the group of structural pruning iterations.
Saniee from the same or similar field of endeavor discloses execution of the processor-executable code further causes the apparatus to reset network weights to seed values at each iteration of the group of structural pruning iterations ([0007], [0016], weights are reset for subsequent pruning and model refinement, [0062], connections are removed (i.e. structural pruning is performed)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Saniee into the teachings of Mehnert for reducing the parameter count for processing by the neural network.
Regarding claim 8, Sahniee further discloses wherein the neural network model is a fully connected neural network model (FIG. 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Saniee into the teachings of Mehnert for reducing the parameter count for processing by the neural network.
Regarding claim 10, Sahniee further discloses resetting un-pruned weights to seed values at a beginning of each iteration of the group of weight pruning iterations ([0007], [0016], weights are reset for subsequent pruning and model refinement, [0062], connections are masked (i.e. weight pruning is performed)).
Regarding claim 20, the limitations of claim 20 are rejected in the analysis of claim 14 (see claim 14 above). Mehnert further discloses a non-transitory computer-readable medium having program code recorded thereon for pruning a neural network model, the program code executed by one or more processors ([0029], [0030] a stored program is executed by a computer).
Claim(s) 2 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mehnert (US 2024/0020512) in view of Saniee et al. (Saniee) (US 2022/0318631), and further in view of Wang et al. (Wang) (US 2019/0286989).
Regarding claims 2 and 15, Mehnert in view of Saniee further discloses the method of claim 1 (see claim 1 above).
Mehnert in view of Saniee is silent about wherein each of the one or more nodes is removed based on a respective mean or respective maximum outward connection weight.
Wang from the same or similar field of endeavor discloses wherein each of the one or more nodes is removed based on a respective maximum outward connection weight ([0019], connections are removed based on a connection weight threshold by providing a measure in weights of the connections).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Wang into the teachings of Mehnert in view of Saniee for reducing connections for processing by the neural network.
Claim(s) 4, 7, 9, 18, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mehnert (US 2024/0020512) in view of Saniee et al. (Saniee) (US 2022/0318631), and further in view of Subbaraya et al. (Subbaraya) (US 2021/0256381).
Regarding claim 4, Mehnert in view of Saniee further discloses the method of claim 3 (see claim 3 above).
Mehnert in view of Saniee is silent about training the neural network model on a training dataset after each iteration of the group of structural pruning iterations; and testing the neural network model on a test dataset after each iteration of the group of structural pruning iterations.
Subbaraya from the same or similar field of endeavor discloses training the neural network model on a training dataset after each iteration of the group of structural pruning iterations ([0024], layers are removed; [0042], model generating process is iterative; [0023], the neural network is trained using a training dataset); and testing the neural network model on a test dataset after each iteration of the group of structural pruning iterations ([0023], the neural network is tested using a testing dataset).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Subbaraya into the teachings of Mehnert in view of Saniee for more efficient, accurate, and faster model training.
Regarding claims 7 and 18, Mehnert in view of Saniee further discloses the method of claim 1 (see claim 1 above).
Mehnert in view of Saniee is silent about wherein each layer of the one or more layers is removed based on a respective number of nodes associated with the layer being less than a node threshold.
Subbaraya from the same or similar field of endeavor discloses wherein each layer of the one or more layers is removed ([0024], a layer is removed) based on a respective number of nodes associated with the layer being less than a node threshold ([0024], [0037], a set of removable layers is identified based on a value for the number of activated neurons being less than a threshold value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Subbaraya into the teachings of Mehnert in view of Saniee for more efficient, accurate, and faster model training.
Regarding claims 9 and 19, Mehnert in view of Saniee further discloses the method of claim 1 (see claim 1 above), wherein the weight pruning is performed at each iteration of a group of weight pruning iterations (see claim 1 above).
Mehnert in view of Saniee is silent about the method further comprises: training the neural network model on a training dataset after each iteration of the group of pruning iterations ([0023], the neural network is trained using a training dataset); and testing the neural network model on a test dataset after each iteration of the group of pruning iterations ([0023], the neural network is tested using a testing dataset).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Subbaraya into the teachings of Mehnert in view of Saniee for more efficient, accurate, and faster model training. The examiner notes while the teachings of Subbaraya are not directed toward weight pruning operations, the process of iterative training and testing of a neural network model is well known the art and would have been obvious to one of ordinary skill in the art to apply to the process of weight pruning for training a model.
Claim(s) 5, 11, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mehnert (US 2024/0020512) in view of Saniee et al. (Saniee) (US 2022/0318631), and further in view of Subbaraya et al. (Subbaraya) (US 2021/0256381), and further in view of Stork et al. (Stork) (US Pat. No. 5,636,326).
Regarding claims 5, 11, and 17, Mehnert in view of Saniee in view of Subbaraya discloses the method of claim 4 (see claim 4 above), wherein the structural pruning is repeated (see claim 1 above); and repeating the weight pruning (see claim 1 above).
Mehnert in view of Saniee in view of Subbaraya is silent about wherein the pruning is repeated until an increase in a test set error value (col. 6, lns. 9-13, pruning is performed until an error value exceeds a threshold); and repeating the weight pruning until an increase in a test set error (col. 6, lns. 9-13, pruning is performed until an error value exceeds a threshold).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Stork into the teachings of Mehnert in view of Saniee in view of Subbaraya for more efficient, accurate, and faster model training.
Claim(s) 12 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mehnert (US 2024/0020512) in view of Saniee et al. (Saniee) (US 2022/0318631), and further in view of Tuli et al. (Tuli) (US 2025/0037028).
Regarding claims 12 and 13, Mehnert in view of Saniee further discloses the method of claim 1 (see claim 1 above).
Mehnert in view of Saniee is silent about wherein the neural network model is a surrogate model; and wherein the surrogate model is used in an active learning process.
Tuli from the same or similar field of endeavor discloses wherein the neural network model is a surrogate model ([0152], a surrogate model is used for active learning); and wherein the surrogate model is used in an active learning process ([0152], a surrogate model is used for active learning).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Tuli into the teachings of Mehnert in view of Saniee for more efficient, accurate, and faster model training.
Allowable Subject Matter
Claim 6 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Lu et al. (Lu) (US 2025/0315675) ([0007] structural and non-structural pruning is performed).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFERY A WILLIAMS whose telephone number is (571)270-7579. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached at 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFERY A WILLIAMS/ Primary Examiner, Art Unit 2488 Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFERY A WILLIAMS whose telephone number is (571)270-7579. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached at 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFERY A WILLIAMS/ Primary Examiner, Art Unit 2488
Prosecution Insights