Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is in response to the amendments filed 20 January 2026. Claims 1, 11-14, and 21-23 are amended. Claims 1-25 are pending and have been examined.
Response to Arguments
Applicant' s arguments, see page 9, filed 20 January 2026, with respect to the rejections of Claims 13 and 22 under 35 U.S.C. 112(b) have been fully considered and are persuasive. The rejections of Claims 13 and 22 under 35 U.S.C. 112(b) have been withdrawn.
APPLICANT'S ARGUMENT: Applicant argues (page X, paragraph X) that "Claim 13 is amended as suggested by the Office. ¶ Claim 22 is amended to clarify that it is the pruned network that has the characteristics recited in claim 22."
EXAMINER'S RESPONSE: The rejections of Claims 13 and 22 under 35 U.S.C. 112(b) have been withdrawn in light of arguments and/or amendments.
Applicant's arguments , see pages 9-12, filed 20 January 2026, with respect to the rejections of Claims 1-25 under 35 U.S.C. 101 have been fully considered but they are not persuasive.
APPLICANT'S ARGUMENT: Applicant argues (page 10, continued paragraph) that "claim 1 does recite elements that improve upon the technological process of inference commonly performed by neural networks .... ¶ ... Through these features [of Claim 1], the accuracy of the neural network may be improved and the computation overhead may be minimized. In other words, the features of claim 1 enable the neural network to perform a more efficient inference operation."
Applicant argues (page 11 paragraph 2) that "Paragraphs 0092, 0093, 0101, and 0111 provide additional details explaining how techniques of some of the disclosed embodiments provide improved neural network efficiency through optimization of the neural network. ¶ Because claim 1 recites features that, viewed as a whole, improve the speed of a neural network inference operation (an important and well-established technical field), and because the improvement is sufficiently documented in the Specification, claim 1 is not directed to an abstract idea."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. Examiner notes that Applicant's argument relies on features, namely, neural network accuracy, overhead, and efficiency, that are not recited in amended Claim 1. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims.
APPLICANT'S ARGUMENT: Applicant argues (page 11, paragraph 5) that "Claim 1 recites processing hardware performing inference with a neural network, determining importances of operation branches, adding channels, and clipping. None of these actions are analogous to observations, evaluations, judgments, or opinions. Nor do the claims amount to 'do it on a computer.'"
EXAMINER'S RESPONSE: Examiner respectfully disagrees. Amended Claim 1 recites several mental process steps relating to the optimization of a target neural network block, for example, determining importance measures of operation branches found therein. However, the claim does not appear to recite additional elements of the mental process step of determining importance measures that integrate the mental process step into a practical application or provide significantly more. Rather, the mental process step appears, at the claimed level of generality, to be performable entirely in the human mind or with pen and paper.
Amended Claim 1 recites claim elements that are additional elements to the mental process steps, for example, "by the processing hardware" and "in the storage hardware." At the claimed level of generality, these steps appear merely to employ computing machinery to perform the recited mental process steps, rather than integrating into a practical application or providing an inventive concept. The claim limitation "perform ... an inference operation" also appears to recite application of computing machinery to perform a mental process step, rather than representing an improvement in a computer or computing technology.
APPLICANT'S ARGUMENT: Applicant argues (page 12, paragraph 2-2) that "The rejection of claims 9 and 11 is traversed because the rejection is based on findings that various elements of claim 1 are 'well-understood, routing and conventional activity', which is unsupported by any factual findings. ¶ ... The rejection is traversed because it lacks any of the three required written citations supporting the rejection's conclusion of various limitations being well-understood, routine, or conventional."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. Per MPEP 2106.05(a)(II):
To show that the involvement of a computer assists in improving the technology, the claims must recite the details regarding how a computer aids the method, the extent to which the computer aids the method, or the significance of a computer to the performance of the method. Merely adding generic computer components to perform the method is not sufficient. Thus, the claim must include more than mere instructions to perform the method on a generic component or machinery to qualify as an improvement to an existing technology.
See also MPEP 2106.05(f), which states:
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. ... [I]mplementing an abstract idea on a generic computer, does not integrate the abstract idea into a practical application in Step 2A Prong Two or add significantly more in Step 2B, similar to how the recitation of the computer in the claim in Alice amounted to mere instructions to apply the abstract idea of intermediated settlement on a generic computer.
As indicated in the 35 U.S.C. 101 rejection below, the addition elements recited by amended Claim 1 do not integrate into a practical application or provide an inventive concept because they appear merely to recite use of computer to perform the recited mental process steps.
APPLICANT'S ARGUMENT: Applicant argues (page 12, paragraph 4) that "the rejection is traversed because it mis-characterizes features of claims 9 and 11. Claim 9 recites storing or marking a channel sequence number as a maximum-contribution channel. Claim 11 recites selecting the target network block. The rejection characterizes these limitations as 'receiving or transmitting data over a network'. This characterization of claims 9 and 11 is in error."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. As interpreted under BRI in light of the specification, the additional element steps of storing values appear to recite activity that amounts to mere data gathering under Step 2A Prong Two of the Alice/May framework and insignificant extra-solution activity under Step 2B. Figures 11A and 11B and paragraph [0145] of the instant specification support the recited elements of Claims 9 and 11 using a bus connecting processing, memory, storage, and network communication components. In the absence of recitation of further limitation by Claims 9 and 11, under BRI, the additional element steps appear to amount to a recitation of the receiving/transmitting data over a network, such as the bus providing communication and data exchange, under Step 2B analysis.
Applicant' s arguments, see pages 12-15, filed 20 January 2026, with respect to the rejections of Claims 1-7, 11-17, and 21-25 under 35 U.S.C. 103 have been fully considered but they are not persuasive.
APPLICANT'S ARGUMENT: Applicant argues (page 13, paragraph 4) that "The rejection is in error because a channel is not an operation branch. This is confirmed by the fact that the rejection cites channels in Yang as comparable to channels in claim 1. The channels recited in claim 1 cannot be properly interpreted as both channels (with respect to adding channels) and operation branches; channels and operation branches have distinctly different meanings in the art of neural networks."
EXAMINER'S RESPONSE: In the rejection of amended Claim 1 under 35 U.S.C. 103 below, Yang is shown to teach the instant target operation branch with a corresponding structure indicated by CE-Block at p. 178420, Table 1, "Multiple convolutional network structures for CIFAR-10 and CIFAR-100 datasets." In the teaching of Yang, the neural network structure of the CE-Block comprises two convolutional layers (for example, Conv2 and Conv3 of Table 1, and p. 178419, IV. Experiments And Results: "As shown in Table 1, the basic model structure is ResNet-32. CE-ResNet and SENet-32 add CE-unit and SE-unit after 'Conv2' and 'Conv3' respectively"). Table 1 also indicates that the Conv2 convolutional layer comprises numerous operation branches, for example, an operation branch corresponding to one of the
3
×
3
convolution operations. The channel expansion operation for a single feature map (
X
and variations) corresponding to a convolution operation of Conv2 is shown on Yang, p. 178417, Figures 1 and 2.
APPLICANT'S ARGUMENT: Applicant argues (page 14, paragraph 1-2) that "Yang does not teach or suggest increasing a number of channels of a target operation branch per set or determining importance measures of respective operation branches per set. As shown above, Yang teaches increasing the number of channels of existing channels. As also shown above, Yang teaches determining importances of channels. ¶ The rejection is further traversed because the rejection interprets the claimed importance measures as having two separate and distinct meanings. Claim 1 recites computing importance measures of respective operation branches. The rejection cites Yang's channel importance vector ('importance of the channels', citing Figure 1)."
EXAMINER'S RESPONSE: Examiner respectfully disagrees that amended Claim 1 recites "increasing a number of channels of a target operation branch per set or determining importance measures of respective operation branches per set." As indicated in the rejection of amended Claim 1 under 35 U.S.C. 103 below, Yang is shown to teach importance measures of the operation branches of the target operation branch, including the target operation branch. Yang's convolution operation is shown to teach a claimed operation branch. The feature map of Yang's Figures 1 and 2, produced by such a convolution operation, is depicted as having the corresponding channels expanded according the cited weight coefficients, which correspond to the instant importance measures. As the cited coefficients are used for channel expansion for the target operation branch, they represent importance measures of the target operation branch and the target network block.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-25 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1
Step 1
Claim 1 recites a method, and thus the claimed process falls within a statutory category of invention.
Step 2A Prong 1
The claim recites performs optimizing for the target network block, which is a mental process. The claim recites generating ... an extended network block of the target network block by increasing ... a number of channels of a target operation branch in the target network block to a determined number of channels, which is a mental process as interpreting a network block to be a neural network architectural component or sub-component and interpreting a channel to represent a unit of input or feature data capacity operated on by a neural network. The claim recites determining ... based on at least the increased channels of the target operation branch, importance measures of the respective operation branches, including the target operation branch with the increased number of channels, in the extended network block, which is a mental process. The claim recites clipping ... a channel of the target operation branch in the extended network block, wherein the clipping is performed according to the importance measures of the respective operation branches including the target operation branch, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element a computing device comprising storage hardware storing a target network block and processing hardware invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element by the processing hardware invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element in the storage hardware invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element the target network block comprises a sub-network of a neural network comprising operation branches that include the target operation branch, and wherein each operation branch comprises at least one respective channel and is a separate computational pathway within the neural network invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element perform, on an input, an inference operation by the neural network comprising the clipped extended network block invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 2
Step 1
Regarding Claim 2, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites generating an output of the target network block by splicing outputs of all the channels of the operation branches included in the target network block, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 3
Step 1
Regarding Claim 3, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites the determined number of channels is determined to be equal to a total number of channels in the target network block, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 4
Step 1
Regarding Claim 4, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites the generating of the extended network block comprises increasing the number of channels of each of the respective operation branches in the target network block to the determined number of channels, which is a mental process when interpreting a network block to be a neural network architectural component or sub-component and interpreting a channel to represent a unit of input or feature data capacity operated on by a neural network.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 5
Step 1
Regarding Claim 5, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites the channel is clipped such that a total number of channels remaining in the clipped extended network block is less than or equal to a total number of channels in the target network block, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 6
Step 1
Regarding Claim 6, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites the importance measure of each respective operation branch in the extended network block is based on an importance value of each respective channel thereof, which is a mental process. The claim recites the clipping of the channel of the operation branch comprises selecting the target channel for clipping based on the target channel having an importance value that is not greater than an importance threshold, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 7
Step 1
Regarding Claim 7, the rejection of Claim 6 is incorporated.
Step 2A Prong 1
The claim recites the clipping of the channel is performed such that, when a total number of remaining channels in the clipped extended network block is less than a total number of channels in the target network block, which is a mental process. The claim recites the importance threshold satisfies a requirement of a ratio of a number of channels of each operation branch in the clipped extended network block to a number of channels of a corresponding operation branch in the target network block to be equal to or greater than 0.2 and less than or equal to 1, which is a mental process. The claim recites the ratio corresponding to each operation branch satisfies a requirement that all the ratios are not 1, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 8
Step 1
Regarding Claim 8, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites determining a weight of each operation branch and a weight of each channel of each operation branch in the extended network block through a first equation, which is a mental process. The claim recites determining an importance of each operation branch in the extended network block, based on the weight of each operation branch and the weight of each channel of each operation branch, which is a mental process. The claim recites generating ... an extended network block of the target network block by increasing ... a number of channels of a target operation branch in the target network block to a determined number of channels (as recited by Claim 1), wherein the extended network block comprises
m
+
1
operation branches and
n
+
1
outputs, wherein the first equation comprises
F
j
=
∑
i
=
1
m
Y
i
×
W
i
j
×
F
i
j
, wherein
F
i
is an output of sequence number
j
j
=
0,1
,
2
,
…
,
n
,
Y
i
is a weight of an operation branch of sequence number
i
i
=
0,1
,
2
,
…
,
m
,
W
i
j
is a weight of a channel of sequence number
i
j
,
F
i
j
is an output of a channel of sequence number
i
j
, and the channel of sequence number
i
j
is a channel of sequence number
j
in the operation branch of sequence number
i
, which is a mathematical concept.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 9
Step 1
Regarding Claim 9, the rejection of Claim 8 is incorporated.
Step 2A Prong 1
The claim recites determining of the importance value of each operation branch in the extended network block, based on the weight of each operation branch and the weight of each channel of each operation branch, which is a mental process. The claim recites when
Y
i
×
W
i
j
, which is a weight product of a channel of sequence number
i
j
, satisfies a second equation comprising
Y
i
×
W
i
j
=
max
{
Y
0
×
W
0
j
,
Y
1
×
W
1
j
,
…
,
Y
m
×
W
m
j
}
, which is a mathematical concept. The claim recites counting the number of maximum-contribution channels in each operation branch as a contribution number, which is a mental process. The claim recites determining an importance of each operation branch according to the contribution number of each operation branch, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2
The additional element storing or marking, in the storage hardware, the channel of sequence number
i
j
as a maximum-contribution channel amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering").
Step 2B
The additional element storing or marking, in the storage hardware, the channel of sequence number
i
j
as a maximum-contribution channel is well-understood, routine, conventional activity (see MPEP 2106.05(d), "receiving or transmitting data over a network").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 10
Step 1
Regarding Claim 10, the rejection of Claim 8 is incorporated.
Step 2A Prong 1
The claim recites determining of the importance value of each operation branch in the extended network block comprises determining an importance measure of each respective operation branch based on a relationship between a weight product of each channel of each operation branch and a weight product threshold, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 11
Step 1
Regarding Claim 11, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites selecting the target network block from the neural network ..., wherein the target network block comprises a sub-network of the neural network, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2
The additional element stored in the storage hardware amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering").
Step 2B
The additional element stored in the storage hardware is well-understood, routine, conventional activity (see MPEP 2106.05(d), "receiving or transmitting data over a network").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 12
Step 1
Regarding Claim 12, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites generating of the target network block comprises: selecting a network block from neural network of which the network block is a subnetwork thereof, which is a mental process.
The claim recites generating the target network block by adding an operation branch to the selected network block, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 13
Step 1
Regarding Claim 13, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites wherein the method further comprises: determining an importance measure of each respective operation branch in the network block, which is a mental process. The claim recites generating a transition network block by clipping at least one operation branch in the network block according to the importance measure of each operation branch, which is a mental process. The claim recites the generating of the target network block comprises increasing a number of channels of at least one operation branch in the transition network block, wherein a total number of channels of the target network block is less than a total number of channels in the network block, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element a computing device comprising storage hardware storing a target network block and processing hardware (as recited by Claim 1), wherein the target network block comprises one network block that is a sub-network of the neural network invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 14
Step 1
Claim 14 recites an apparatus, and thus the claimed machine falls within a statutory category of invention.
Step 2A Prong 1
The claim recites generate an extended network block by increasing a number of channels in a target operation branch in the target network block to a preset number of channels, wherein the target network block comprises a sub-network of a neural network comprising operation branches that include the target operation branch, and wherein each operation branch comprises at least one respective channel and is a separate computational pathway within the neural network, which is a mental process when interpreting a network block to be a neural network architectural component or sub-component and interpreting a channel to represent a unit of input or feature data capacity operated on by a neural network. The claim recites determine based on at least the increased channels of the target operation branch, an importance measure of each respective operation branch in the extended network block, which is a mental process. The claim recites clip a channel of the target operation branch in the extended network block according to the importance measures of the respective operation branches including the target operation branch, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element processing hardware invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element storage hardware storing a target network block invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element storing instructions configured to, when executed by the processing hardware, configure the processing hardware invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element perform, on an input, an inference operation by the neural network comprising the clipped extended network block invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Claims 15-20, dependent on Claim 14, incorporate the rejection of Claim 14. Claims 15-20 incorporate substantively all the limitations of Claims 5-10, respectively, in apparatus form and are rejected under the same rationales.
Regarding Claim 21
Step 1
Claim 21 recites a method, and thus the claimed process falls within a statutory category of invention.
Step 2A Prong 1
The claim recites optimizing ... a neural network stored in the storage hardware, which is a mental process. The claim recites selecting a network block from the neural network, the network block comprising a sub-network of the neural network comprising branches, each branch comprising a respective original number of original channels, wherein each original channel is a separate computational pathway within the neural network comprises a respective channel weight, and wherein the branches include a target branch, which is a mental process.
The claim recites determining numbers of extension channels to add to the branches, respectively, of the network block based at least on the respective numbers of channels of the branches, which is a mental process. The claim recites adding the determined numbers of extension channels to the respective branches of the network block such that the network block comprises the original channels and the extension channels, which is a mental process.
The claim recites pruning a target channel from the network block, the target channel comprising one of the extension channels or one of the original channels, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element performed by a computing device comprising processing hardware and storage hardware by the processing hardware invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element performing, with the neural network having the pruned network block, an inference operation on an input to the neural network invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 22
Step 1
Regarding Claim 22, the rejection of Claim 21 is incorporated.
Step 2A Prong 1
The claim recites optimizing ... a neural network (as recited by Claim 21), wherein at least one branch in the pruned network block comprises a plurality of the original channels and a plurality of the extension channels, and wherein a total number of channels in the pruned network block comprises a total number of the original channels before the adding of the extension channels, which is a mental process, as it can be practically performed in the human mind, or with use a physical aid, such as pen and paper.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 23
Step 1
Regarding Claim 23, the rejection of Claim 21 is incorporated.
Step 2A Prong 1
The claim recites generating importance measures for the respective branches based at least on the channels added to the branches, which is a mental process. The claim recites selecting, from among the branches, a branch for pruning, or for pruning a channel thereof, based on the importance measures, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 24
Step 1
Regarding Claim 24, the rejection of Claim 23 is incorporated.
Step 2A Prong 1
The claim recites wherein the importance measure of a corresponding branch is generated based on the channel weights thereof, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 25
Step 1
Regarding Claim 25, the rejection of Claim 24 is incorporated.
Step 2A Prong 1
The claim recites wherein the target channel is selected from among the extension and original channels of the target branch based on the selection of the target branch, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-7, 11-17, and 21-25 are rejected under 35 U.S.C. 103 as being unpatentable over Yang, et al., "Channel Expansion Convolutional Network for Image Classification" (hereinafter "Yang") in view of Lin, et al. (US 2021/0073644 A1, hereinafter "Lin").
Regarding Claim 1, Yang teaches:
A method (Yang, p. 178414, Abstract: "With the continuous evolution of research on convolutional neural networks, it is an efficient and fashionable method to introduce attention mechanism into the convolutional structure. ... The experimental results show that our proposed new object-level attention convolution model has achieved good image classification results") performed by a computing device comprising storage hardware storing a target network block (Yang, p. 178419, IV. Experiments and Results: "The experiment is implemented on the framework Keras 2.3.1, and its backend is Tensorflow-gpu 1.13.1. ... The experiment used two conventional image classification datasets CIFAR-10 and CIFAR-100 ... ¶ For Cifar-10 and Cifar-100 experiments, ResNet-32, SENet-32, and WideResNet (28, 10) are used as the basic structures, and the proposed 'CE-unit' [channel expansion] modules are embedded after specific convolutional layers," where a storage hardware is inherent in storing datasets, and p. 178420, Table 1, "Multiple convolutional network structures for CIFAR-10 and CIFAR-100 datasets," where Yang's Conv2 layer, embedded CE-block, and Conv3 layer of CE-ResNet corresponds to the instant target network block) and processing hardware (Yang, p. 178419, IV. Experiments and Results: "The experimental environment includes the Intel Z390 work- station motherboard, i7-9700K processor, 32G memory, GeForce RTX 2080TI GPU, Ubuntu 18.0.4 operating system. ... The experiment used two conventional image classification datasets CIFAR-10 and CIFAR-100") that performs optimizing for the target network block (Yang, p. 178415, B. Multi-Scale Convolutional Model: "AMS-CNN [41] introduces channel sorting and grouping convolution, which makes more important feature channels perform more convolution operations to achieve multi-scale expression. Next, we use the improved channel expansion algorithm to construct the object-level attention multi-scale convolution model in AMS-CNN, and implement the object-level attention based on the channel expansion strategy in the intra-layer multi-scale convolution," where Yang's object-level attention implemented by channel expansion corresponds to the instant optimized block), the method comprising:
generating ... an extended network block of the target network block (Yang, p. 178420, Table 1, "Multiple convolutional network structures for CIFAR-10 and CIFAR-100 datasets," where either Conv2 or Conv3 of CE-ResNet-32 corresponds to the the instant target network block, both extended to the corresponding CE-blocks) by increasing ... a number of channels of a target operation branch in the target network block to a determined number of channels (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," depicting feature map
X
~
1
expanded to
X
1
, increasing the number of channels by a factor of
β
1
) ... by the processing hardware ... in the storage hardware ... (Yang, p. 178419, IV. Experiments and Results: "The experiment is implemented on the framework Keras 2.3.1, and its backend is Tensorflow-gpu 1.13.1. ... The experiment used two conventional image classification datasets CIFAR-10 and CIFAR-100 ... ¶ For Cifar-10 and Cifar-100 experiments, ResNet-32, SENet-32, and WideResNet (28, 10) are used as the basic structures, and the proposed 'CE-unit' [channel expansion] modules are embedded after specific convolutional layers"), wherein the target network block comprises a sub-network of a neural network comprising operation branches that include the target operation branch (Yang, p. 178420, Table 1, "Multiple convolutional network structures for CIFAR-10 and CIFAR-100 datasets," where Yang's Conv2 layer, embedded CE-block, and Conv3 layer of CE-ResNet corresponds to the instant target network block, and Conv2 layer corresponds to the instant target operation branch, and where Conv2 and Conv3 layers are sub-networks), and wherein each operation branch comprises at least one respective channel and is a separate computational pathway within the neural network (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," depicting 4C channels as input, and where, for example, the Conv2 layer that correspond to the instant operation branch is separate from the subsequent Conv3 layer, and thus a separate pathway);
determining, by the processing hardware, based on at least the increased channels of the target operation branch, importance measures of the respective operation branches, including the target operation branch with the increased number of channels, in the extended network block (Yang, p. 178416, III. Proposed Approach, A. Channel Expansion Network: "CENet first calculates the weight coefficients of feature channels and arranges the channels in descending order according to the weight coefficients," where Yang's weight coefficient corresponds to the instant importance measure); ...
perform, on an input, an inference operation by the neural network comprising the ... extended network block (Yang, p. 178421, Figure 6, "The 'attention' visualization diagram of the CENets based on ResNet, SENet and WideResNet structures," depicting on the CE-ResNet row of images heatmaps representing inference performed using the processed network, including the CE-block of layers Conv2 and Conv3).
Yang teaches a method of optimizing a target network block of a neural network and performing an inference operation by the network.
Yang does not explicitly teach clipping ... a channel of the target operation branch in the extended network block, wherein the clipping is performed according to the importance measures of the respective operation branches including the target operation branch and perform, on an input, an inference operation by the neural network comprising the clipped extended network block.
However, Lin teaches:
clipping ... a channel ... in the ... network block (Lin, [0005]: "Machine learning model compression systems and related techniques are described herein that can intelligently remove certain parameters of a machine learning model, without introducing a loss in performance of the machine learning model for a given task. ... For example ... certain weights of a channel of a single-branch or multi branch neural network can be pruned (e.g., set to 0)," where Lin's pruning corresponds to the instant clipping), wherein the clipping is performed according to the importance measures of the respective ... branches including the target ... (Lin, [0065]: "the penalty engine 314 can apply a penalty to the duplicate set of filters. ... In some cases, the penalty can be based on one or more scaling factors of the duplicate set of filters. A scaling factor can be denoted by the scale parameter
γ
(e.g., as used in batch normalization layers), which can be used to detect the candidate filters" where [0066]: "The scaling factor
γ
can also be used to represent an importance of a particular channel the scaling factor is associated with");
perform, on an input, an inference operation by the neural network comprising the clipped extended network block (Lin, [0037]: "Once the network 100 is trained, it can be used to provide an output related to a target task during inference (during run - time after the neural network model has been trained). ... The network 100 can be trained to perform other tasks , such as object detection , object classification, natural language processing , font selection, among others" and [0095]: "The results shown in Table 1 are shown as a mean Intersection over Union ( mloU ) metric . The IoU is the ratio between the area of overlap and the area of union between the ground truth and the predicted areas , and can be used as a metric in object detection to evaluate the relevance of the predicted locations. The mIoU is the average between the IoU of the segmented objects over all the images of the test dataset . As shown , even after pruning using the techniques described herein , the mIoU values after the model is com pressed are similar to the full model").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang regarding a method of optimizing a target network block of a neural network and performing an inference operation by the network with those of Lin regarding clipping a channel of the target operation branch in the extended network block, wherein the clipping is performed according to the importance measures of the respective operation branches including the target operation branch and performing, on an input, an inference operation by the neural network comprising the clipped extended network block.
The motivation to do so would be to facilitate reducing the resource usage of a model without reducing the quality of its results (Lin, [0059]: "the penalty is applied by the penalty engine 314 to the complex layers or branches and is not applied to the non-complex layers or branches, which can preserve the parameters of the non-complex layers or branches. Such selective compression provides better resource savings because the complex layers or branches have many more resources (and thus are likely to have more wasted resources that do not add to the quality of the output) as compared to the non-complex layers or branches").
Regarding Claim 14, Yang teaches:
An apparatus comprising: processing hardware; and storage hardware and storing instructions configured to, when executed by the processing hardware, configure the processing hardware (Yang, p. 178419, IV. Experiments and Results: "The experimental environment includes the Intel Z390 work-station motherboard, i7-9700K processor, 32G memory, GeForce RTX 2080TI GPU, Ubuntu 18.0.4 operating system. ... The experiment used two conventional image classification datasets CIFAR-10 and CIFAR-100," where a storage hardware is inherent in storing datasets) to: perform precisely those steps recited by the rejection of Claim 1. Claim 14 is rejected under the same rationale as Claim 1.
Regarding Claim 2, the rejection of Claim 1 is incorporated.
The Yang/Lin combination teaches:
generating an output of the target network block by splicing outputs of all the channels of the operation branches included in the target network block (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," where
X
C
depicts the instant spliced outputs, and p. 178417, III. Proposed Approach, A. Channel Expansion Network: "
X
i
represents the feature map obtained by performing channel expansion convolution for each group.
X
z
represents connecting
X
1
,
X
2
,
X
3
and
X
4
along the direction of the feature channel.
X
z
=
X
1
,
X
2
,
X
3
,
X
4
.
X
c
is the feature map containing 4C channels obtained by performing "
1
×
1
" convolution on
X
z
").
Regarding Claim 3, the rejection of Claim 1 is incorporated.
The Yang/Lin combination teaches:
wherein the determined number of channels is determined to be equal to a total number of channels in the target network block (Yang, p. 178417, III. Proposed approach, A. Channel expansion network: "perform a convolution operation on each small block. This process is similar to grouping convolution operation ...
X
~
m
i
=
C
o
n
v
∑
j
=
1
C
V
k
j
,
X
~
j
i
,
1
≤
m
≤
β
i
C
(9)
¶ ...
X
m
i
are the feature channels corresponding to each group," where Yang's
m
corresponds to the instant number of channels).
Regarding Claim 4, the rejection of Claim 1 is incorporated.
The Yang/Lin combination teaches:
wherein the generating of the extended network block comprises increasing the number of channels of each of the respective operation branches in the target network block to the determined number of channels (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," which depicts increased channels, such as
β
1
C
, and p. 178417, III. Proposed approach, A. Channel expansion network: "
β
is the expansion coefficient,
β
=
β
1
,
β
2
,
β
3
,
β
4
. In the experiment, the value of
β
is
{
2
2
,
2
1
,
2
0
,
2
-
1
}
," where Yang's
β
values correspond to the instant determined number).
Regarding Claim 5, the rejection of Claim 1 is incorporated.
Lin further teaches:
wherein the channel is clipped such that a total number of channels remaining in the clipped extended network block is less than or equal to a total number of channels in the target network block (Lin, [0094]: "pruning of channels in the complex branch is performed using the techniques described herein to generate a compressed model. Table 1 below shows the performance comparison between the full model and the compressed model. As shown, the full model with 10.22 million parameters is compressed by a factor of five (5x) to 2.05 million parameters without affecting the performance of the network").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Yang/Lin combination regarding clipping a channel of the target operation branch in the extended network block with the further teachings of Lin regarding the channel being clipped such that a total number of channels remaining in the clipped extended network block is less than or equal to a total number of channels in the target network block.
The motivation to do so would be to facilitate compression of a model such that the resulting model consumes less computational resources (Lin, [0075]: "the layer or branch B is replaced in the trained neural network 302 with the compressed layer or branch B*. Based on the loss minimization described above , the compressed layer or branch B* will provide a similar output as the original layer or branch B, but will have fewer parameters (e.g., channels , branches , weights , etc. ) and thus require less computational resources").
Claim 15, dependent on Claim 14, incorporates the rejection of Claim 14. Claim 15 incorporates substantively all limitations of Claim 5 in apparatus form and is rejected under the same rationale.
Regarding Claim 6, the rejection of Claim 1 is incorporated.
Lin further teaches:
wherein the importance measure of each respective operation branch in the extended network block is based on an importance value of each respective channel thereof (Lin, [0058]: "once the complex layers or branches of the trained neural network 302 are selected for compression by the layer selection engine 304, the penalty engine 314 applies a penalty to filters of the selected complex layers or branches. In some cases , the penalty can be based on a scaling factor associated with a layer or branch," where Lin's scaling factor may be associated with a channel, as in [0065]: "The candidate filter determination engine 316 can select candidate filters for removal from the duplicate set of filters depending on the penalty applied by the penalty engine 314. In some cases, the penalty can be based on one or more scaling factors of the duplicate set of filters. A scaling factor can be denoted by the scale parameter
γ
" and [0066]: "
γ
is the scaling factor of a channel and corresponding filter"), and
wherein the clipping of the channel of the operation branch comprises selecting the target channel for clipping based on the target channel having an importance value that is not greater than an importance threshold (Lin, [0067]: "The candidate filter determination engine 316 can determine that a filter (and corresponding channel) is a candidate filter (or removable filter) if the scaling factor
γ
of the filter is less than a scaling factor threshold . ... After a period of learning time , the scaling factor
γ
of some of the filters in the duplicate set of filters will become smaller than the scaling factor threshold, and will be identified as candidate filters by the candidate filter determination engine 316").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Yang/Lin combination regarding determining importance measures of the respective operation branches and clipping a channel of the target operation branch with the further teachings of Lin regarding the importance measure of each respective operation branch in the extended network block being based on an importance value of each respective channel thereof, and wherein the clipping of the channel of the operation branch comprises selecting the target channel for clipping based on the target channel having an importance value that is not greater than an importance threshold.
The motivation to do so would be to facilitate producing a compressed version of a model that operates with high accuracy (Lin, [0086]: "the process 600 includes generating a modified duplicate set of filters by removing the one or more candidate filters from the duplicate set of filters. For example ... a compressed version of B* is output . At block 616 , the process 600 includes replacing the set of filters in the neural network with the modified duplicate set of filters.... Re-training the neural network using the training data (and the training loss in some cases) re-tunes the network so that it can complete the target task the network was designed to perform with high accuracy").
Claim 16, dependent on Claim 14, incorporates the rejection of Claim 14. Claim 16 incorporates substantively all limitations of Claim 6 in apparatus form and is rejected under the same rationale.
Regarding Claim 7, the rejection of Claim 6 is incorporated.
Lin further teaches:
wherein the clipping of the channel is performed such that, when a total number of remaining channels in the clipped extended network block is less than a total number of channels in the target network block, the importance threshold (Lin, [0091]: "the penalty engine 314 can apply a penalty to the channels of the layer 702 based on minimization of the loss by the loss minimization engine ... For example, as shown in FIG. 7B, the penalty applied by the penalty engine 314 during minimization of the loss can result in the scaling factor S2 of channel 2 ... being minimized to a value below the scaling factor threshold. The candidate filter determination engine 316 can determine that the channels 2 and 3 include candidate filters, and can remove the candidate filters (and thus the channels) from the neural network," where Lin's importance threshold
γ
is learned during training, and Lin's pruning occurs according to the learned
γ
) satisfies a requirement of a ratio of a number of channels of each operation branch in the clipped extended network block to a number of channels of a corresponding operation branch in the target network block to be equal to or greater than 0.2 and less than or equal to 1 (Lin, [0094]: "pruning of channels in the complex branch is performed using the techniques described herein to generate a compressed model. Table 1 below shows the performance comparison between the full model and the compressed model. As shown, the full model with 10.22 million parameters is compressed by a factor of five (5x) to 2.05 million parameters without affecting the performance of the network," where Lin's factor of five when calculated exactly as 2.05/10.22 corresponds to the recited ratio bounds, and where Lin's without affecting the performance corresponds to the instant satisfied requirement) and the ratio corresponding to each operation branch satisfies a requirement that all the ratios are not 1 (Lin, [0067]: "The candidate filter determination engine 316 can determine that a filter (and corresponding channel) is a candidate filter (or removable filter) if the scaling factor Y of the filter is less than a scaling factor threshold. ... After a period of learning time, the scaling factor
γ
of some of the filters in the duplicate set of filters will become smaller than the scaling factor threshold, and will be identified as candidate filters by the candidate filter determination engine 316," where Lin's some candidate filters corresponds to the instant requirement that some channels be pruned).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Yang/Lin combination regarding the clipping of the channel of the operation branch comprising selecting the target channel for clipping based on the target channel having an importance value that is not greater than an importance threshold with the further teachings of Lin regarding the clipping of the channel is performed such that, when a total number of remaining channels in the clipped extended network block is less than a total number of channels in the target network block, the importance threshold satisfies a requirement of a ratio of a number of channels of each operation branch in the clipped extended network block to a number of channels of a corresponding operation branch in the target network block to be equal to or greater than 0.2 and less than or equal to 1, and the ratio corresponding to each operation branch satisfies a requirement that all the ratios are not 1.
The motivation to do so would be to facilitate avoiding over-parameterization of models thereby improving efficiency of resource usage (Lin, [0093]: "Using the model compression system 300 and the related techniques described herein, parameters of a neural network can be greatly reduced , without affecting the performance of the neural network . In one example, by implementing the techniques described herein, 80 % of the parameters of a trained neural network can be removed without affecting the performance of on a challenging dense prediction task" and [0039]: "Despite the superior performance of deep neural networks, such networks can be over-parameterized in order to achieve good performance. The over-parameterization can result in a large amount of waste in computational resources").
Claim 17, dependent on Claim 16, incorporates the rejection of Claim 16. Claim 17 incorporates substantively all limitations of Claim 7 in apparatus form and is rejected under the same rationale.
Regarding Claim 11, the rejection of Claim 1 is incorporated.
The Yang/Lin combination teaches:
selecting the target network block from the neural network, which is stored in the storage hardware, wherein the target network block comprises a sub-network of the neural network (Yang, p. 178419, IV. Experiments and Results: "For Cifar-10 and Cifar-100 experiments, ResNet-32, SENet-32, and WideResNet (28, 10) are used as the basic structures, and the proposed "CE-unit" modules are embedded after specific convolutional layers. ... ¶ As shown in Table 1, the basic model structure is ResNet-32. CE-ResNet and SENet-32 add CE-unit [Channel Expansion unit] ... after 'Conv2'," where Yang's Conv2 layer, embedded CE-block, and Conv3 layer of CE-ResNet corresponds to the instant target network block, and where a convolution layer corresponds to the instant sub-network).
Regarding Claim 12, the rejection of Claim 1 is incorporated.
The Yang/Lin combination teaches:
wherein the generating of the target network block comprises: selecting a network block from neural network of which the network block is a subnetwork thereof (Yang, p. 178419, IV. Experiments and Results: "As shown in Table 1, the basic model structure is ResNet-32. CE-ResNet and SENet-32 add CE-unit [Channel Expansion unit] ... after 'Conv2'," where Yang's Conv2 corresponds to the instant target network block, and where a convolution layer corresponds to a sub-network under BRI), and generating the target network block by adding an operation branch to the selected network block (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," depicting selected network block
X
~
having operation branches
X
1
~
-
X
4
~
and
X
1
–
X
4
added).
Regarding Claim 13, the rejection of Claim 1 is incorporated.
The Yang/Lin combination teaches:
wherein the target network block comprises one network block that is a sub-network of the neural network (Yang, p. 178419, IV. Experiments and Results: "As shown in Table 1, the basic model structure is ResNet-32. CE-ResNet and SENet-32 add CE-unit [Channel Expansion unit] ... after 'Conv2'," where Yang's Conv2 layer, embedded CE-block, and Conv3 layer of CE-ResNet corresponds to the instant target network block, and where a convolution layer corresponds to the instant sub-network), and wherein the method further comprises:
determining an importance measure of each respective operation branch in the network block (Yang, p. 178416, III. Proposed Approach, A. Channel Expansion Network: "'channel sorting module' is used to obtain channel importance vector ... a one-dimensional vector that records the importance of each feature channel" and "we introduce the 'channel expansion module'. As shown in Figure 1,
X
~
is the output of the channel sorting module and the input of the channel expansion module. It has been sorted in descending order according to the importance of the channels," where Yang's importance of the channels corresponds to the instant importance measures), generating a transition network block ... according to the importance measure of each operation branch (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," where the channel expansion module depicts
X
~
expanded to
X
1
~
-
X
4
~
and
X
1
–
X
4
, corresponding to the instant transition network block), and the generating of the target network block comprises increasing a number of channels of at least one operation branch in the transition network block (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," where the channel expansion module depicts
X
~
expanded to
X
1
~
-
X
4
~
and
X
1
–
X
4
, doubling the original number of channels), wherein a total number of channels of the target network block is less than a total number of channels in the network block (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," which depicts in Channel Expansion a greater total number of channels than either
X
~
or
X
c
).
Lin further teaches:
clipping at least one operation branch in the network block according to the importance measure of each operation branch (Lin, [0091]: "the penalty engine 314 can apply a penalty to the channels of the layer 702 based on minimization of the loss by the loss minimization engine ... For example, as shown in FIG. 7B, the penalty applied by the penalty engine 314 during minimization of the loss can result in the scaling factor S2 of channel 2 ... being minimized to a value below the scaling factor threshold. The candidate filter determination engine 316 can determine that the channels 2 and 3 include candidate filters, and can remove the candidate filters (and thus the channels) from the neural network," where Lin's importance threshold
γ
is learned during training, and Lin's pruning occurs according to the learned
γ
).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Yang/Lin combination regarding determining an importance measure of each respective operation branch in the network block, generating a transition network block by clipping at least one operation branch in the network block according to the importance measure of each operation branch, and generating the target network block by increasing a number of channels of at least one operation branch in the transition network block, wherein a total number of channels of the target network block is less than a total number of channels in the network block with the further teachings of Lin regarding clipping at least one operation branch in the network block according to the importance measure of each operation branch.
The motivation to do so would be to facilitate reducing the resource usage of a model without reducing the quality of its results (Lin, [0059]: "the penalty is applied by the penalty engine 314 to the complex layers or branches and is not applied to the non-complex layers or branches, which can preserve the parameters of the non-complex layers or branches. Such selective compression provides better resource savings because the complex layers or branches have many more resources (and thus are likely to have more wasted resources that do not add to the quality of the output) as compared to the non-complex layers or branches").
Regarding Claim 21, Yang teaches:
A method performed by a computing device comprising processing hardware and storage hardware (Yang, p. 178419, IV. Experiments and Results: "The experimental environment includes the Intel Z390 work- station motherboard, i7-9700K processor, 32G memory, GeForce RTX 2080TI GPU, Ubuntu 18.0.4 operating system. The experiment is implemented on the framework Keras 2.3.1, and its backend is Tensorflow-gpu 1.13.1. ... The experiment used two conventional image classification datasets CIFAR-10 and CIFAR-100 ... ¶ For Cifar-10 and Cifar-100 experiments, ResNet-32, SENet-32, and WideResNet (28, 10) are used as the basic structures, and the proposed 'CE-unit' [channel expansion] modules are embedded after specific convolutional layers," where a storage hardware is inherent in storing datasets"), the method comprising:
optimizing, by the processing hardware, a neural network stored in the storage hardware (Yang, p. 178415, B. Multi-Scale Convolutional Model: "AMS-CNN [41] introduces channel sorting and grouping convolution, which makes more important feature channels perform more convolution operations to achieve multi-scale expression. Next, we use the improved channel expansion algorithm to construct the object-level attention multi-scale convolution model in AMS-CNN, and implement the object-level attention based on the channel expansion strategy in the intra-layer multi-scale convolution," where Yang's object-level attention implemented by channel expansion corresponds to the instant optimized block), the optimizing comprising: selecting a network block from the neural network (Yang, p. 178419, IV. Experiments and Results: "For Cifar-10 and Cifar-100 experiments, ResNet-32, SENet-32, and WideResNet (28, 10) are used as the basic structures, and the proposed 'CE-unit' [channel expansion] modules are embedded after specific convolutional layers" and p. 178420, Table 1, "Multiple convolutional network structures for CIFAR-10 and CIFAR-100 datasets," where Yang's Conv2 layer, embedded CE-block, and Conv3 layer of CE-ResNet corresponds to the instant target network block), the network block comprising a sub-network of the neural network comprising branches (Yang, p. 178420, Table 1, "Multiple convolutional network structures for CIFAR-10 and CIFAR-100 datasets," where Yang's Conv2 layer corresponds to the instant sub-network, and where the Conv2 convolution operations such as
3
×
3
correspond to the instant branches) each branch comprising a respective original number of original channels (Yang, p. 178417, Figure 2, "The structure schematic of channel sorting module, which adds a sorting process on the basis of the squeeze-and-excitation module," depicting feature map
X
of a convolution operation comprising an original
C
channels), wherein each original channel is a separate computational pathway within the neural network and comprises a respective channel weight (Yang, p. 178416, III. Proposed Approach, A. Channel Expansion Network: "we first calculate the importance of each channel based on the weight of the channel features"), and wherein the branches include a target branch (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," where the convolution of Yang's
X
~
feature map corresponds to the instant target branch);
determining numbers of extension channels to add to the branches, respectively, of the network block based at least on the respective numbers of channels of the branches (Yang, p. 178417, III. Proposed approach, A. Channel expansion network: "In Figure 1, the depth of color indicates the strength of importance. Then, perform a convolution operation on each small block. This process is similar to grouping convolution operation.
X
~
i
=
S
p
l
i
t
X
~
,
n
,
1
≤
i
≤
n
(8)
X
~
m
i
=
C
o
n
v
∑
j
=
1
C
V
k
j
,
X
~
j
i
,
1
≤
m
≤
β
i
C
(9)
¶ The function Split(*) divides
X
~
into
n
groups along the channel direction (as shown in Figure 1, n takes the value 4)," where Yang's
m
corresponds to the instant number of channels), and
adding the determined numbers of extension channels to the respective branches of the network block such that the network block comprises the original channels and the extension channels (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," where
X
~
corresponds both to the instant target network block and target operation branch);
performing, with the neural network having the ... network block, an inference operation on an input to the neural network (Yang, p. 178421, Figure 6, "The 'attention' visualization diagram of the CENets based on ResNet, SENet and WideResNet structures," depicting on the CE-ResNet row of images heatmaps representing inference performed using the processed network, including the CE-block of layers Conv2 and Conv3).
Yang teaches a method of optimizing a target network block of a neural network and performing an inference operation by the network.
Yang does not explicitly teach pruning a target channel from the network block, the target channel comprising one of the extension channels or one of the original channels and performing, with the neural network having the pruned network block, an inference operation on an input to the neural network.
pruning a target channel from the network block, the target channel comprising one of the extension channels or one of the original channels (Lin, [0005]: "Machine learning model compression systems and related techniques are described herein that can intelligently remove certain parameters of a machine learning model, without introducing a loss in performance of the machine learning model for a given task. ... For example ... certain weights of a channel of a single-branch or multi branch neural network can be pruned (e.g., set to 0)");
performing, with the neural network having the pruned network block, an inference operation on an input to the neural network (Lin, [0037]: "Once the network 100 is trained, it can be used to provide an output related to a target task during inference (during run - time after the neural network model has been trained). ... The network 100 can be trained to perform other tasks , such as object detection , object classification, natural language processing , font selection, among others" and [0095]: "The results shown in Table 1 are shown as a mean Intersection over Union ( mloU ) metric . The IoU is the ratio between the area of overlap and the area of union between the ground truth and the predicted areas , and can be used as a metric in object detection to evaluate the relevance of the predicted locations. The mIoU is the average between the IoU of the segmented objects over all the images of the test dataset . As shown , even after pruning using the techniques described herein , the mIoU values after the model is com pressed are similar to the full model").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang regarding a method for performing optimization of a neural network by generating an extended network block with an increased number of channels and determining the importance of branches of the extended block with those of Lin regarding pruning a target channel from the network block, the target channel comprising one of the extension channels or one of the original channels.
The motivation to do so would be to facilitate reducing the resource usage of a model without reducing the quality of its results (Lin, [0059]: "the penalty is applied by the penalty engine 314 to the complex layers or branches and is not applied to the non-complex layers or branches, which can preserve the parameters of the non-complex layers or branches. Such selective compression provides better resource savings because the complex layers or branches have many more resources (and thus are likely to have more wasted resources that do not add to the quality of the output) as compared to the non-complex layers or branches").
Regarding Claim 22, the rejection of Claim 21 is incorporated.
The Yang/Lin combination teaches:
wherein at least one branch in the pruned network block comprises a plurality of the original channels and a plurality of the extension channels (Yang, p. 178417, Figure 1, "The structure schematic of channel expansion unit, which includes the channel sorting module and the channel expansion module," depicting original channels of
X
~
1
-
X
~
4
and extension channels of
X
1
-
X
4
), and wherein a total number of channels in the pruned network block comprises a total number of the original channels before the adding of the extension channels (Yang, p. 178418, Figure 4, "The improved channel sorting strategy for the intra-layer multi-scale convolution model requires only one channel expansion coefficient," depicting
X
1
–
X
4
having the same number of channels as
X
, i.e., 4C).
Regarding Claim 23, the rejection of Claim 21 is incorporated.
The Yang/Lin combination teaches:
generating importance measures for the respective branches based at least on the channels added to the branches (Yang, p. 178417, III. Proposed approach, A. Channel expansion network: "In Figure 1, the depth of color indicates the strength of importance. Then, perform a convolution operation on each small block").
Yang teaches a method for performing optimization of a network block by generating an extended network block with an increased number of channels and determining the importance of branches of the extended block.
Yang does not explicitly teach selecting a branch for pruning, or for pruning a channel thereof, based on the importance measures.
However, Lin teaches:
selecting , from among the branches, a branch for pruning, or for pruning a channel thereof (Lin, [0005]: "Machine learning model compression systems and related techniques are described herein that can intelligently remove certain parameters of a machine learning model, without introducing a loss in performance of the machine learning model for a given task. ... For example ... certain weights of a channel of a single-branch or multi branch neural network can be pruned (e.g., set to 0)"), based on the importance measures (Lin, [0065]: "the penalty engine 314 can apply a penalty to the duplicate set of filters. ... In some cases, the penalty can be based on one or more scaling factors of the duplicate set of filters. A scaling factor can be denoted by the scale parameter
γ
(e.g., as used in batch normalization layers), which can be used to detect the candidate filters" where [0066]: "The scaling factor
γ
can also be used to represent an importance of a particular channel the scaling factor is associated with").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang regarding a method for performing optimization of a network block by generating an extended network block with an increased number of channels and determining the importance of branches of the extended block with those of Lin regarding selecting a branch for pruning, or for pruning a channel thereof, based on the importance measures.
The motivation to do so would be to facilitate reducing the resource usage of a model without reducing the quality of its results (Lin, [0059]: "the penalty is applied by the penalty engine 314 to the complex layers or branches and is not applied to the non-complex layers or branches, which can preserve the parameters of the non-complex layers or branches. Such selective compression provides better resource savings because the complex layers or branches have many more resources (and thus are likely to have more wasted resources that do not add to the quality of the output) as compared to the non-complex layers or branches").
Regarding Claim 24, the rejection of Claim 23 is incorporated.
The Yang/Lin combination teaches:
wherein the importance measure of a corresponding branch is generated based on the channel weights thereof (Yang, p. 178416, III. Proposed Approach, A. Channel Expansion Network: "we first calculate the importance of each channel based on the weight of the channel features").
Regarding Claim 25, the rejection of Claim 24 is incorporated.
Lin further teaches:
wherein the target channel is selected from among the extension and original channels of the target branch based on the selection of the target branch (Lin, Fig. 7A, depicting the pruning result of channels 3 and 4 from their branch, which is selected for consideration by inherency, where [0020]: "FIG. 7A is a diagram illustrating an example of a branch of a multi-branch neural network").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Yang/Lin combination regarding pruning a target channel from the network block with those of Lin regarding wherein the target channel is selected from among the extension and original channels of the target branch based on the selection of the target branch.
The motivation to do so would be to take advantage of a smaller model without sacrificing performance (Lin, [0005]: "Machine learning model compression systems and related techniques are described herein that can intelligently remove certain parameters of a machine learning model, without introducing a loss in performance of the machine learning model for a given task. ... For example , one or more channels of a single-branch or multi branch neural network can be removed").
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Claims 8-10 and 18-20 are rejected as reciting subject-matter that is patent ineligible. After a thorough search of the art, no reference was found to teach or fairly suggest:
8. The method of claim 1,
wherein the determining of the importance value of each operation branch in the extended network block comprises:
determining a weight of each operation branch and a weight of each channel of each operation branch in the extended network block through a first equation, and
determining an importance of each operation branch in the extended network block, based on the weight of each operation branch and the weight of each channel of each operation branch, and
wherein the extended network block comprises
m
+
1
operation branches and
n
+
1
outputs, wherein the first equation comprises
F
j
=
∑
i
=
1
m
Y
i
×
W
i
j
×
F
i
j
, wherein
F
i
is an output of sequence number
j
j
=
0,1
,
2
,
…
,
n
,
Y
i
is a weight of an operation branch of sequence number
i
i
=
0,1
,
2
,
…
,
m
,
W
i
j
is a weight of a channel of sequence number
i
j
,
F
i
j
is an output of a channel of sequence number
i
j
, and the channel of sequence number
i
j
is a channel of sequence number
j
in the operation branch of sequence number
i
.
9. The method of claim 8, wherein the determining of the importance value of each operation branch in the extended network block, based on the weight of each operation branch and the weight of each channel of each operation branch comprises:
when
Y
i
×
W
i
j
, which is a weight product of a channel of sequence number
i
j
, satisfies a second equation comprising
Y
i
×
W
i
j
=
max
{
Y
0
×
W
0
j
,
Y
1
×
W
1
j
,
…
,
Y
m
×
W
m
j
}
, storing or marking, in the storage hardware, the channel of sequence number
i
j
as a maximum-contribution channel, counting the number of maximum-contribution channels in each operation branch as a contribution number, and determining an importance of each operation branch according to the contribution number of each operation branch.
10. The method of claim 8, wherein the determining of the importance value of each operation branch in the extended network block comprises determining an importance measure of each respective operation branch based on a relationship between a weight product of each channel of each operation branch and a weight product threshold.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT N DAY whose telephone number is (703)756-1519. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.N.D./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights