Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
Claims 1, 11, and 19 have been amended.
Claims 1-3, 5-13, and 15-20 remain pending in the application.
The amendment filed 04/21/2026 is sufficient to overcome the 112(f) claim interpretation of claim 19. The previous claim interpretation has been withdrawn.
The amendment filed 04/21/2026 is sufficient to overcome the 112(a) and 112(b) rejections of claims 1-3, 5-13, and 15-20. The previous rejections have been withdrawn.
Response to Arguments
Argument 1, regarding the 112(a) and 112(b) rejections, applicant argues that the rejections should be withdrawn in view of amendments made to the claims that further clarify the terms “full-sized network” and “sub-network”. Examiner agrees and the rejections have been withdrawn.
Argument 2, regarding the prior art rejections, applicant argues that none of the cited art teaches “wherein each of the sub-networks is a sub-network of the full-sized network”.
Examiner notes that Saniee teaches wherein each of the sub-networks is a sub-network of the full-sized network (each subnetwork exists within the full deep neural network, P0055, P0060-P0061). Examiner also notes that Kuo teaches co-training the full-sized network and a plurality of sub-networks, …, and the co-training of the full-sized network and the plurality of sub-networks comprises: performing a plurality of iterations of supervised co-training of … a respective subset of the plurality of sub-networks (in multiple iterations, sub-networks are trained in the same training process (co-trained), C20:L58-66. The learning may be supervised learning, C13:L17-24).
Examiner also notes that Saniee explicitly teaches a fully-connected deep neural network (see Saniee P0055), and that Kuo recites sub-networks being fully nested within a whole neural network (see Kuo C20:L60-66). Thus, under the broadest reasonable interpretation of the term “full-sized network”, the “whole network” taught by Kuo may be interpreted as a full-sized network in view of Saniee’s teachings of a fully-connected deep neural network.
The full prior art rejections are outlined below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-7, 11-13, 15-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kuo et al (Pub. No.: US 12093817 B2), hereafter Kuo in view of Chen et al (Pub. No.: US 20240283499 A1), hereafter Chen and Saniee et al (Pub. No.: US 20220318631 A1), hereafter Saniee.
Regarding claims 1, 11, and 19, Kuo teaches co-training the full-sized network and a plurality of sub-networks, wherein the co-training of the full-sized network and the plurality of sub-networks comprises: performing a plurality of iterations of supervised co-training of … a respective subset of the plurality of sub-networks (in multiple iterations, sub-networks are trained in the same training process (co-trained), C20:L58-66. The learning may be supervised learning, C13:L17-24)… selecting a sub-network from among co-trained sub-networks based on a hardware constraint (heuristic search algorithm is used to search for an optimal subnetwork from the subnetworks based on a given resource configuration including number of cores and size of memory, C21:L3-8).
Kuo does not appear to explicitly teach “training a full-sized network, wherein the training of the full-sized network comprises performing at least one epoch of training the full-sized network;…performing a plurality of iterations of supervised co-training of the full-sized network”.
Chen teaches training a full-sized network, wherein the training of the full-sized network comprises performing at least one epoch of training the full-sized network;…performing a plurality of iterations of supervised co-training of the full-sized network (precoding matrix determination network consists of four different sub networks, P0104. Subnetworks may be trained independently before the entire precoding matrix determination network is trained, P0118. Training of the precoding matrix determination network may be completed over a number of iterations, P0105).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Kuo and Chen before them, to include Chen’s specific teaching of training an entire network and its sub-networks as a whole in Kuo’s system of artificial neural network configuration. One would have been motivated to make such a combination of training an entire network and its sub-networks as a whole (see Chen P0104-P0105, P0118), and training a bulk of sub-networks of a network to create a fully nested final network (see Kuo C20:L53-66) to improve performance of data transmission (see Chen P0002).
Kuo in view of Chen does not appear to explicitly teach “wherein each of the sub-networks is a sub-network of the full-sized network …excluding a smallest sub-network in size from the respective subset of the plurality of sub-networks in an iteration of the plurality of iterations”.
Saniee teaches wherein each of the sub-networks is a sub-network of the full-sized network (each subnetwork exists within the full deep neural network, P0055, P0060-P0061)… excluding a smallest sub-network in size from the respective subset of the plurality of sub-networks in an iteration of the plurality of iterations (connections of a subnetwork with the smallest magnitude weights may be removed or masked, P0062, P0073. This is an iterative process, P0026, P0083).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Kuo, Chen, and Saniee before them, to include Saniee’s specific teaching of removing or masking connections of a subnetwork with the smallest magnitude weights in Kuo’s system of artificial neural network configuration. One would have been motivated to make such a combination of removing or masking connections of a subnetwork with the smallest magnitude weights (see Saniee P0026, P0062, P0073, P0083), and using a heuristic search algorithm to determine components that should be removed from a neural network (see Kuo C6:L44-49) to improve classification accuracy of a neural network (see Saniee P0002).
Regarding claims 2, 12, and 20, Kuo in view of Chen and Saniee teaches the limitations of claims 1, 11, and 19 as outlined above. Kuo further teaches wherein the co-training of the full-sized network and the respective subset of the sub-networks comprises maximizing the full-sized network only with respect to ground truth labels (each block of the full network is maximized with respect to the ground truth by analyzing errors between the ground truth and prior classifiers. Blocks with larger errors compared to the ground truth are dropped, maximizing the full network, C10:L12-33).
Regarding claims 3 and 13, Kuo in view of Chen and Saniee teaches the limitations of claims 1 and 11 as outlined above. Kuo further teaches wherein the co-training of the full-sized network and the respective subset of the sub-networks comprises maximizing the sub-networks only with respect to output of the full-sized network (Only accurate sub-networks remain after less accurate sub-networks are removed, and this is gauged by the performance of the whole network, C18:L32-46).
Regarding claims 5 and 15, Kuo in view of Chen and Saniee teaches the limitations of claims 1 and 11 as outlined above. Kuo further teaches wherein, for each iteration, each subset of the sub- networks is selected at random (“the training operation includes randomly selecting nested sub-networks during training batches”, C3:L33-35).
Regarding claims 6 and 16, Kuo in view of Chen and Saniee teaches the limitations of claims 1 and 11 as outlined above. Kuo further teaches performing of the at least one epoch of training of the full-sized network is without performing co-training with the sub-networks and is before the performing of the plurality of iterations of supervised co-training (ordered dropout, the training of sub-networks within the full network after removing certain sub-networks is compared to the training of the full network. This comparison cannot be made without a first epoch of training the full network before, C19:L5-14 figure 6).
Regarding claims 7 and 17, Kuo in view of Chen and Saniee teaches the limitations of claims 1 and 11 as outlined above. Kuo further teaches wherein each of the sub-networks has a channel expansion ratio selected from the group consisting of 3, 4, and 6 (channel expansion ratio is interpreted as width in view of P0044 of spec of instant application. Width of the sub-networks may be 3 or 4 bits, C19:L63-67, C20:L1-9).
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kuo in view of Chen and Saniee and further in view of Zhang et al (Pub. No.: CN 112700786 B), hereafter Zhang.
Regarding claims 8 and 18, Kuo in view of Chen and Saniee teaches the limitations of claims 1 and 11 as outlined above. Kuo does not appear to explicitly teach wherein each of the sub-networks has a depth selected from the group consisting of 2, 3, and 4.
Zhang teaches wherein each of the sub-networks has a depth selected from the group consisting of 2, 3, and 4 (depth is interpreted as number of layers in view of P0044 of spec of instant application. Zhang teaches each subnetwork has 3 layers. Page 13, full paragraph 3).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Kuo, Chen, Saniee and Zhang before them, to include Zhang’s specific teaching of each subnetwork having 3 layers in Kuo’s system of Artificial Neural Network Configuration And Deployment. One would have been motivated to make such a combination of each subnetwork having 3 layers and neural networks having at least 2 layers to include different number of building blocks (see Zhang page 13, full paragraph 3), and the sliding windows for the layers are configured to cover the same proportion of building blocks per layer (see Kuo C6:L11-16) to improve model training (see Zhang page 15, paragraph 2).
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Kuo in view Chen and Saniee and further in view of Wan et al (Pub. No: CN 111798469 A), hereafter Wan.
Regarding claim 9, Kuo in view of Chen and Saniee teaches the limitations of claim 1 as outlined above. Kuo does not appear to explicitly teach wherein each of the sub-networks consists of five blocks.
Wan teaches wherein each of the sub-networks consists of five blocks (sub network comprises 5 convolution blocks. Page 3, paragraph 8).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Kuo, Chen, Saniee and Wan before them, to include Wan’s specific teaching of each subnetwork having 5 blocks in Kuo’s system of Artificial Neural Network Configuration And Deployment. One would have been motivated to make such a combination of each subnetwork having 5 blocks (see Wan page 3, paragraph 8) and each subnetwork including blocks for specific functions for respective resource configuration (see Kuo C3:L19-22) to improve efficiency of a neural network by reducing computational resources needed for model training (see Wan page 2, full paragraphs 4-5).
Regarding claim 10, Kuo in view of Chen and Saniee and further in view of Wan teaches the limitations of claim 9 as outlined above. Wan further teaches wherein the five blocks have respective kernel sizes of 3, 5,3, 3, and 5 (In view of P0045-P0046 of the instant application, the kernel sizes of blocks 1, 3, and 4 may be fixed to 3 and blocks 2 and 5 may be fixed to 5. It is not clear in either the specification or in the claim why these kernel sizes in particular are chosen for the specific blocks. Examiner interprets this limitation to mean kernel sizes should be selected from the group consisting of 3 and 5 for each of the 5 blocks. Wan teaches the kernel size of each of the 5 blocks is 3. Page 3, paragraph 8).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHAN MOUNDI whose telephone number is (703)756-1547. The examiner can normally be reached 8:30 A.M. - 5 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.M./Examiner, Art Unit 2141
/DANIEL T PELLETT/Primary Examiner, Art Unit 2121
Prosecution Insights