DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment/Status of Claims
Claims 1, 10, and 19 were amended.
Claims 1-22 are pending and examined herein.
Claims 1-22 are rejected under 35 U.S.C. 101.
Claims 1-22 are rejected under 35 U.S.C. 103.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/18/2026 has been entered.
Response to Arguments
Applicant's arguments filed 3/18/2026 regarding the 35 U.S.C. 101 rejection of claims 1-22 have been fully considered but they are not persuasive. Applicant argues, see pages 7-14, that the claims do not recite abstract ideas. In particular, applicant argues that the claims do not recite a mathematical concept or a mental process. Examiner respectfully disagrees.
MPEP 2106.04(a)(2) states “A claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation. There is no particular word or set of words that indicates a claim recites a mathematical calculation. That is, a claim does not have to recite the word "calculating" in order to be considered a mathematical calculation. For example, a step of "determining" a variable or number using mathematical methods or "performing" a mathematical operation may also be considered mathematical calculations when the broadest reasonable interpretation of the claim in light of the specification encompasses a mathematical calculation.”
For example, claim 1 recites “perform first batch normalization on input data”. This is a mathematical calculation because batch normalization is a mathematical operation and the claim recites performing batch normalization. Therefore, the claims do recite mathematical concepts. See 35 U.S.C. 101 rejection below for further explanation.
MPEP 2106.04(a)(2) states “The courts consider a mental process (thinking) that "can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 (2012) ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same).
“Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions. A discussion of concepts performed in the human mind, as well as concepts that cannot practically be performed in the human mind and thus are not "mental processes", is provided below with respect to point A.”
For example, claim 1 recites “quantize the first batch normalized input data”. This is a mental process, as it can be performed in the human mind. Quantization is mapping a range of numbers to certain discrete numbers. This process can be practically performed in the human mind. Therefore, it is a mental process. See 35 U.S.C. 101 rejection below for further explanation.
Applicant further argues that the claims represent an improvement to neural network processing and neural network quantization. Examiner respectfully disagrees.
MPEP 2106.05(a) states "It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. See the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981)) in subsection II, below. In addition, the improvement can be provided by the additional element(s) in combination with the recited judicial exception. See MPEP § 2106.04(d) (discussing Finjan, Inc. v. Blue Coat Sys., Inc., 879 F.3d 1299, 1303-04, 125 USPQ2d 1282, 1285-87 (Fed. Cir. 2018))."
Applicant states "For example, while the independent claims may not explicitly recite the improvement "reduce a dimension" described in the specification, one of ordinary skill understands these improvements to be recited in the claims via the claimed operations such as batch normalization and quantization." Therefore, the improvement is claimed to be provided by the steps "perform first batch normalization on input data; quantize the first batch normalized input data." However, both of these limitations are abstract ideas, which cannot provide the improvement.
Thus, the claims do not represent an improvement to technology and does not integrate the abstract idea into a practical application nor amounts to significantly more than the abstract idea.
Applicant’s arguments, see pages 14-18, filed 3/18/2026, with respect to the rejection(s) of claim(s) 1-22 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Bethge (“MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy?”, 2020), Zhang (“FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations”, February 2021), and Jun (“ECG arrhythmia classification using a 2-D convolutional neural network”, 2018).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
MPEP § 2109(III) sets out steps for evaluating whether a claim is drawn to patent-eligible subject
matter. The analysis of claims 1-22, in accordance with these steps, follows.
Step 1 Analysis:
Step 1 is to determine whether the claim is directed to a statutory category (process, machine,
manufacture, or composition of matter. Claims 1-9 are directed to a machine, claims 10-17 are directed to a process, claim 18 is directed to an article of manufacture, and claims 19-22 are directed to a process. All claims are directed to statutory categories and analysis proceeds.
Step 2A Prong One, Step 2A Prong Two, and Step 2B Analysis:
Step 2A Prong One asks if the claim recites a judicial exception (abstract idea, law of nature, or natural phenomenon). If the claim recites a judicial exception, analysis proceeds to Step 2A Prong Two, which asks if the claim recites additional elements that integrate the abstract idea into a practical application. If the claim does not integrate the judicial exception, analysis proceeds to Step 2B, which asks if the claim amounts to significantly more than the judicial exception. If the claim does not amount to significantly more than the judicial exception, the claim is not eligible subject matter under 35 U.S.C. 101.
Regarding claim 1, the following claim elements are abstract ideas:
perform first batch normalization on input data; (Performing batch normalization is mathematical calculations, which is a mathematical concept.)
quantize the first batch normalized input data; (One could practically in the human mind map the data into -1 or 1, which is quantization. This is a mental process.)
perform a convolution operation based on the quantized input data; (Convolution is a mathematical calculation, which is a mathematical concept.)
apply an activation function to a result of the convolution operation, wherein a result of the applying is output data directly output from an activation layer; and (An activation function is a mathematical operation. As the mathematical operation is applied, this is a mathematical calculation, which is a mathematical concept.)
perform the first operation by performing second batch normalization on the output data directly output from an activation layer. (Performing batch normalization is mathematical calculations, which is a mathematical concept.)
The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A device, the device comprising: one or more processors configured to perform a first operation for executing one or more basic blocks of a neural network model and a second operation for executing one or more transition blocks of the neural network model to execute the neural network model, wherein, for the performing of the first operation, the one or more processors are configured to: (This limitation recites generic computer components with generic computer functions. This is mere instructions to apply an exception. See MPEP § 2106.05(f).)
Regarding claim 2, the rejection of claim 1 is incorporated herein. The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the one or more basic blocks comprise: (Basic blocks are comprised of generic machine learning components. This is mere instructions to apply an exception.)
a first batch normalization layer; (A batch normalization layer is a generic machine learning component. This is mere instructions to apply an exception.)
a quantization layer; (A quantization layer is a generic machine learning component. This is mere instructions to apply an exception.)
a convolution layer; (A convolution layer is a generic machine learning component. This is mere instructions to apply an exception.)
the activation layer; and (An activation function layer is a generic machine learning component. This is mere instructions to apply an exception.)
a second batch normalization layer. (A batch normalization layer is a generic machine learning component. This is mere instructions to apply an exception.)
Regarding claim 3, the rejection of claim 1 is incorporated herein. The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the one or more transition blocks comprise: (Transition blocks are comprised of generic machine learning components. This is mere instructions to apply an exception.)
a pooling layer; (A pooling layer is a generic machine learning component. This is mere instructions to apply an exception.)
a channel upscaling layer; and (A channel scaling layer is a generic machine learning component. This is mere instructions to apply an exception.)
a third batch normalization layer. (A batch normalization layer is a generic machine learning component. This is mere instructions to apply an exception.)
Regarding claim 4, the rejection of claim 1 is incorporated herein. The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the activation function comprises: a rectified linear unit (ReLU) function. (ReLU is a generic machine learning component. This is mere instructions to apply an exception.)
Regarding claim 5, the rejection of claim 1 is incorporated herein. The following is an abstract idea:
binarize the input data by applying a sign function on the first batch normalized input data. (A sign function is a mathematical operation. As the sign function is applied, this is a mathematical calculation, which is a mathematical concept.)
Claim 5 does not recite any additional elements.
Regarding claim 6, the rejection of claim 1 is incorporated herein. The following is an abstract idea:
apply a step function on the first batch normalized input data (A step function is a mathematical operation. As the step function is applied, this is a mathematical calculation, which is a mathematical concept.)
Claim 6 does not recite any additional elements.
Regarding claim 7, the rejection of claim 1 is incorporated herein. The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the one or more basic blocks comprise: a residual connection that connects the input data to the second batch normalized output data. (Residual connections are generic machine learning components. This is mere instructions to apply an exception.)
Regarding claim 8, the rejection of claim 1 is incorporated herein. The following are abstract ideas:
perform pooling on output data of the one or more basic blocks; (Pooling is a mathematical calculation, which is a mathematical concept.)
The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
duplicate a channel of the neural network model; and (Channel scaling is a generic machine learning process. This is mere instructions to apply an exception.)
perform the second operation by performing third batch normalization on input data of the neural network model in which the channel is duplicated. (Batch normalization is a generic machine learning process. This is mere instructions to apply an exception.)
Regarding claim 9, the rejection of claim 8 is incorporated herein. The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
perform average pooling on the output data of the one or more basic blocks. (Average pooling is a mathematical calculation, which is a mathematical concept.)
Claim 9 does not recite any additional elements.
Regarding claim 10, the following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A processor-implemented method, comprising: (A processor is a generic computer component. This is mere instructions to apply an exception.)
The remainder of claim 10 recites substantially similar subject matter to claim 1 and is rejected with the same rationale, mutatis mutandis.
Claims 11-17 recite substantially similar subject matter to claims 2-8 respectively and are rejected with the same rationale, mutatis mutandis.
Regarding claim 18, the rejection of claim 10 is incorporated herein, the following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 10. (This limitation recites generic computer components and generic computer functions. This is mere instructions to apply an exception.)
Regarding claim 19, the following are abstract ideas:
performing first batch normalization on input data; (Performing batch normalization is mathematical calculations, which is a mathematical concept.)
quantizing the first batch normalized input data; (One could practically in the human mind map the data into -1 or 1, which is quantization. This is a mental process.)
performing a convolution operation based on the quantized input data; (Convolution is a mathematical calculation, which is a mathematical concept.)
applying an activation function to a result of the convolution operation, wherein a result of the applying is output data directly output from an activation layer; and (An activation function is a mathematical operation. As the mathematical operation is applied, this is a mathematical calculation, which is a mathematical concept.)
performing the first operation by performing second batch normalization on the output data directly output from an activation layer. (Performing batch normalization is mathematical calculations, which is a mathematical concept.)
[performing] pooling [and] third batch normalization (Pooling and batch normalization are mathematical calculations, which are mathematical concepts.)
The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A method, the method comprising: performing a first neural network operation by: (Performing a neural network operation is generic training. This is mere instructions to apply an exception.)
performing a second neural network operation based on the second batch normalized output data by performing any one or any combination of ... a channel upscaling, ... . (A neural network operation is generic training. Channel scaling is a generic machine learning process. This is mere instructions to apply an exception.)
Regarding claim 20, the rejection of claim 19 is incorporated herein. The following are abstract ideas:
wherein the pooling comprises performing pooling on the second batch normalized output data, (Pooling is a mathematical calculation, which is a mathematical concept.)
the third batch normalization comprises performing third batch normalization on the pooled output data in which the channel is duplicated. (Batch normalization is mathematical calculations, which are mathematical concepts.)
The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
the channel upscaling comprises duplicating a channel of the pooled output data, and (Channel scaling is a generic machine learning process, which is mere instructions to apply an exception.)
Regarding claim 21, the rejection of claim 20 is incorporated herein. The following are abstract ideas:
wherein the pooling comprises performing downsampling on a width and height of the second batch normalized output data. (Downsampling can practically be performed in the human mind, for example, selecting a width and height to reduce to and reducing it. This is a mental process.)
Claim 21 does not recite any additional elements.
Regarding claim 22, the rejection of claim 19 is incorporated herein. The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
performing the first neural network operation a plurality of times, (Repeating a calculation is an insignificant extra-solution activity. See MPEP § 2106.05(d).
wherein the second neural network operation is performed based on a result of the performing of the first neural network a plurality of times. (As above, performing the second neural network operation is mere instructions to apply an exception.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bethge (“MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy?”, 2020), Zhang (“FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations”, February 2021), and Jun (“ECG arrhythmia classification using a 2-D convolutional neural network”, 2018).
Regarding claim 1, Bethge teaches
A device, the device comprising: (One of ordinary skill in the art would realize that a neural network, such as the one taught by Bethge, is implemented using a computer, interpreted as the device.)
one or more processors configured to perform a first operation for executing one or more basic blocks of a neural network model and (One of ordinary skill in the art would realize that a processor in the computer is necessary for running a neural network such as the one taught by Bethge. Page 8 shows the results of the neural network, meaning that the processor has executed the neural network. The neural network includes a basic block, as seen on page 4, Fig. 1a. The improvement block is interpreted as the basic block.)
a second operation for executing one or more transition blocks of the neural network model to drive the neural network model, wherein, for the performing of the first operation, the one or more processors are configured to: (Page 8 shows the results of the neural network, meaning that the processor has executed the neural network. The neural network includes a transition block, as seen on page 4, Fig 1a.)
perform first batch normalization on input data; (The BatchNorm layer in the improvement block in Figure 1a performs batch normalization on the input data.)
quantize the first batch normalized input data; (The sign layer in the improvement block in Fig. 1a performs quantization on the batch normalized input data. Page 7 states "The weights and activations are binarized by using the sign function".)
perform a convolution operation based on the quantized input data; (The BinaryConv layer in the improvement block in Fig. 1a performs a convolution operation on the quantized input data.)
Bethge does not appear to explicitly teach
apply an activation function to a result of the convolution operation, wherein a result of the applying is output data of an activation layer; and
perform the first operation by performing second batch normalization on the output data.
However, Zhang—directed to analogous art—teaches
apply an activation function to a result of the convolution operation, wherein a result of the applying is output data ….; and (Page 173, Fig. 5, “Normal Block” shows BPRelu applied to the result of the batch normalized convolution operation result. Page 4 states "After passing through the convolution layer, the information of the main branch is also corrected by the BN layer and the BPReLU layer. BPReLU helps align the output feature map’s feature of binary convolution with that of the input." Therefore, there is an output data from BPReLU, interpreted as the activation layer.)
perform the first operation by performing second batch normalization on the output data... (The final BatchNorm layer of the “Normal Block” on page 173, Fig. 5 shows that the result of the output data is batch normalized.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang because, as Zhang states on page 173, "Different from ReActNet, we move the BPReLUs before the shortcut connections, given that previous works have pointed out that activation functions residing before the shortcuts tend to perform better [6, 33]. We also add a BatchNorm layer [22] after each shortcut connection. The affine transformation of the BatchNorm layer serves as learning a new distribution for both branches in the next block such that the number of positive and negative values in the activations are more balanced." Zhang also states "Finally, we introduce the fractional convolutional layer to further improve model accuracy. FracBNN preserves the key hardware benefits of conventional BNNs. Meanwhile, it achieves a top-1 accuracy of 71.8% on ImageNet, which rivals that of 8-bit MobileNetV2-level with a slightly larger model size."
The combination of Bethge and Zhang does not appear to explicitly teach
[apply an activation function to a result of the convolution operation, wherein a result of the applying is output data] directly output from the activation layer
[performing batch normalization on the output data] directly output from the activation layer
However, Jun—directed to analogous art—teaches
[apply an activation function to a result of the convolution operation, wherein a result of the applying is output data] directly output from the activation layer (Page 11, Fig. 5 shows the architecture of the CNN model, where the activation function ELU is applied to the result of the convolutional operation Conv1.)
[performing batch normalization on the output data] directly output from the activation layer (Page 10 states "However, from our experience, in some cases, it is better to place the batch normalization layer after the activation function, and the ECG arrhythmia classification is the case in this case. Therefore, we applied a batch normalization layer immediately after every activation function in the model, including the convolutional block and the fully-connected block.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang with the teachings of Jun because, as stated by Jun on page 10, "However, from our experience, in some cases, it is better to place the batch normalization layer after the activation function, and the ECG arrhythmia classification is the case in this case."
Regarding claim 2, the rejection of claim 1 is incorporated herein. Bethge teaches
wherein the one or more basic blocks comprise: (The neural network includes a basic block, as seen on page 4, Fig. 1a. The improvement block is interpreted as the basic block.)
a first batch normalization layer; (The BatchNorm layer in the dense block in Figure 1a performs batch normalization on the input data.)
a quantization layer; (The sign layer in the dense block in Fig. 1a performs quantization on the batch normalized input data. Page 7 states "The weights and activations are binarized by using the sign function".)
a convolution layer; (The BinaryConv layer in the dense block in Fig. 1a performs a convolution operation on the quantized input data.)
Bethge does not appear to explicitly teach
the activation layer; and
a second batch normalization layer.
However, Zhang—directed to analogous art—teaches
the activation layer; and (Page 173, Fig. 5, “Normal Block” shows BPRelu applied to the result of the batch normalized convolution operation result.)
a second batch normalization layer. (The final BatchNorm layer of the “Normal Block” on page 173, Fig. 5 shows that the result of the output data is batch normalized.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang for the reasons given above in regards to claim 1.
Regarding claim 3, the rejection of claim 1 is incorporated herein. Bethge teaches
wherein the one or more transition blocks comprise: (The neural network includes a transition block, as seen on page 4, Fig 1a.)
a pooling layer; (The MaxPooling layer on page 4, Fig. 1b is a pooling layer in the transition block.)
a third batch normalization layer. (The BatchNorm layer on page 4, Fig. 1b is a BatchNorm layer in the transition block.)
Bethge does not appear to explicitly teach
a channel upscaling layer; and
However, Zhang—directed to analogous art—teaches
a channel upscaling layer; and (Page 173, Fig. 5 “Downsample Block” is interpreted as the transition block. The Duplicate layer upscales the channels.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang for the reasons given above in regards to claim 1. Additionally, on page 173, Zhang states "In the downsample layer, the average pooling layer and the channel duplication ensure the shortcut matches the spatial and channel dimensions of the residual."
Regarding claim 4, the rejection of claim 1 is incorporated herein. Bethge does not appear to explicitly teach
wherein the activation function comprises: a rectified linear unit (ReLU) function.
However, Zhang—directed to analogous art—teaches
wherein the activation function comprises: a rectified linear unit (ReLU) function. (BPRelu on page 173, fig. 5 “Normal Block” is a variation of a ReLU function.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang for the reasons given above in regards to claim 1.
Regarding claim 5, the rejection of claim 1 is incorporated herein. Bethge teaches
wherein, for the quantizing, the one or more processors are configured to: binarize the input data by applying a sign function on the first batch normalized input data. (As can be seen in Fig. 1a, the improvement block, interpreted as the basic block that is executed by the first operation, applies a sign function. Page 7 states “The weights and activations are binarized by using the
s
i
g
n
function:
s
i
g
n
x
=
+
1
i
f
x
≥
0
-
1
o
t
h
e
r
w
i
s
e
.
” Binarization is quantization.)
Regarding claim 6, the rejection of claim 1 is incorporated herein. Bethge teaches
wherein, for the quantizing, the one or more processors are configured to: apply a step function on the first batch normalized input data. (As can be seen in Fig. 1a, the improvement block, interpreted as the basic block that is executed by the first operation, applies a sign function. Page 7 states “The weights and activations are binarized by using the
s
i
g
n
function:
s
i
g
n
x
=
+
1
i
f
x
≥
0
-
1
o
t
h
e
r
w
i
s
e
.
” One of ordinary skill in the art would realize that the sign function is a step function. Binarization is quantization.)
Regarding claim 7, the rejection of claim 1 is incorporated herein. Bethge teaches
wherein the one or more basic blocks comprise: a residual connection that connects the input data to the [output]. ("The novel Improvement Block increases the quality of these newly concatenated channels. It uses a binary convolution to compute 64 channels again based on the input feature map of 320 channels. These 64 output channels are added to the previously computed 64 channels through a residual connection, without changing the first 256 channels of the feature map (see Figure 1a)." The previously computed 64 channels is the input to the improvement block.)
Bethge does not appear to explicitly teach
[residual connection to the] second batch normalized output data
However, Zhang—directed to analogous art—teaches
[residual connection to the] second batch normalized output data (Page 173 states "Different from ReActNet, we move the BPReLUs before the shortcut connections, given that previous works have pointed out that activation functions residing before the shortcuts tend to perform better [6, 33]. We also add a BatchNorm layer [22] after each shortcut connection." A shortcut connection is another name for a residual connection. The connection can be seen in Fig. 5, “Normal Block”.)
Regarding claim 8, the rejection of claim 1 is incorporated herein. Bethge teaches
wherein, for the performing of the second operation, the one or more processors are configured to: (One of ordinary skill in the art would realize that a processor in the computer is necessary for running a neural network such as the one taught by Bethge. Page 8 shows the results of the neural network, meaning that the processor has executed the neural network, including executing the second block.)
perform pooling on output data of the one or more basic blocks; (As can be seen in Fig. 1b, the MaxPooling layer performs pooling on the output of the improvement block.)
Bethge does not appear to explicitly teach
duplicate a channel of the neural network model; and
perform the second operation by performing third batch normalization on input data of the neural network model in which the channel is duplicated.
However, Zhang—directed to analogous art—teaches
duplicate a channel of the neural network model; and (Page 173 states "In the downsample layer, the average pooling layer and the channel duplication ensure the shortcut matches the spatial and channel dimensions of the residual." As can be seen in Fig. 5 “Downsample Block”, there is a block that duplicates the channel.)
perform the second operation by performing third batch normalization on input data of the neural network model in which the channel is duplicated. (As can be seen in Fig. 5 “Downsample Block”, the output of the block that duplicates the channel is batch normalized after the shortcut.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang for the reasons given above in regards to claim 1.
Regarding claim 9, the rejection of claim 8 is incorporated herein. Bethge teaches
perform average pooling on the output data of the one or more basic blocks. (Fig 4b shows the “final layers” of which there is an AvgPooling 7x7 layer that performs average pooling on the output of the improvement block, interpreted as the basic block.)
Regarding claim 10, Bethge teaches
A processor-implemented method, comprising: (One of ordinary skill in the art would realize that a neural network method, such as the one taught by Bethge, is intended to be implemented on a computer using a processor.)
The remainder of claim 10 recites substantially similar subject matter to claim 1 and is rejected with the same rationale, mutatis mutandis.
Claims 11-17 recite substantially similar subject matter to claims 2-8 respectively and are rejected with the same rationale, mutatis mutandis.
Regarding claim 18, the rejection of claim 10 is incorporated herein. Bethge teaches
A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 10. (One of ordinary skill in the art would realize that a neural network method, such as the one taught by Bethge, is intended to be implemented on a computer using a processor, which must take its instructions from a non-transitory computer-readable storage medium.)
Regarding claim 19, Bethge teaches
A method, the method comprising: (One of ordinary skill in the art would realize that a processor in the computer is necessary for running a neural network such as the one taught by Bethge. Page 8 shows the results of the neural network, meaning that the processor has executed the neural network. The execution of the neural network is interpreted as the method.)
performing a first neural network operation by: (The execution of the improvement block is interpreted as the first neural network operation.)
performing first batch normalization on input data; (The BatchNorm layer in the improvement block in Figure 1a performs batch normalization on the input data.)
quantizing the first batch normalized input data; (The sign layer in the improvement block in Fig. 1a performs quantization on the batch normalized input data. Page 7 states "The weights and activations are binarized by using the sign function".)
performing a convolution operation based on the quantized input data; (The BinaryConv layer in the improvement block in Fig. 1a performs a convolution operation on the quantized input data.)
performing a second neural network operation based on the second batch normalized output data by performing any one or any combination of a pooling, … and a third batch normalization. (The execution of the transition block is interpreted as the second neural network operation. The transition block, according to Fig. 1b, includes a third batch normalization and a max pooling.)
Bethge does not appear to explicitly teach
applying an activation function to a result of the convolution operation, wherein a result of the applying is output data of an activation layer; and
performing second batch normalization on the output data; and
a channel upscaling,
However, Zhang—directed to analogous art—teaches
applying an activation function to a result of the convolution operation, wherein a result of the applying is output data of an activation layer… ; and (Page 173, Fig. 5, “Normal Block” shows BPRelu applied to the result of the batch normalized convolution operation result. Page 4 states "After passing through the convolution layer, the information of the main branch is also corrected by the BN layer and the BPReLU layer. BPReLU helps align the output feature map’s feature of binary convolution with that of the input." Therefore, there is an output data from BPReLU, interpreted as the activation layer.)
performing second batch normalization on the output data…; and (The final BatchNorm layer of the “Normal Block” on page 173, Fig. 5 shows that the result of the output data is batch normalized.)
a channel upscaling, (Page 173, Fig. 5 “Downsample Block” is interpreted as the transition block. The Duplicate layer upscales the channels.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang because, as Zhang states on page 173, "Different from ReActNet, we move the BPReLUs before the shortcut connections, given that previous works have pointed out that activation functions residing before the shortcuts tend to perform better [6, 33]. We also add a BatchNorm layer [22] after each shortcut connection. The affine transformation of the BatchNorm layer serves as learning a new distribution for both branches in the next block such that the number of positive and negative values in the activations are more balanced." Zhang also states "Finally, we introduce the fractional convolutional layer to further improve model accuracy. FracBNN preserves the key hardware benefits of conventional BNNs. Meanwhile, it achieves a top-1 accuracy of 71.8% on ImageNet, which rivals that of 8-bit MobileNetV2-level with a slightly larger model size." Additionally, on page 173, Zhang states "In the downsample layer, the average pooling layer and the channel duplication ensure the shortcut matches the spatial and channel dimensions of the residual."
However, Jun—directed to analogous art—teaches
[apply an activation function to a result of the convolution operation, wherein a result of the applying is output data] directly output from the activation layer (Page 11, Fig. 5 shows the architecture of the CNN model, where the activation function ELU is applied to the result of the convolutional operation Conv1.)
[performing batch normalization on the output data] directly output from the activation layer (Page 10 states "However, from our experience, in some cases, it is better to place the batch normalization layer after the activation function, and the ECG arrhythmia classification is the case in this case. Therefore, we applied a batch normalization layer immediately after every activation function in the model, including the convolutional block and the fully-connected block.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang with the teachings of Jun because, as stated by Jun on page 10, "However, from our experience, in some cases, it is better to place the batch normalization layer after the activation function, and the ECG arrhythmia classification is the case in this case."
Regarding claim 20, the rejection of claim 19 is incorporated herein. Bethge teaches
[building the model using basic blocks followed by transition blocks] (As can be seen in Fig. 1b, the improvement blocks, interpreted as basic blocks, are immediately followed by transition blocks in every case except the last.)
Bethge does not appear to explicitly teach
wherein the pooling comprises performing pooling on the second batch normalized output data,
the channel upscaling comprises duplicating a channel of the pooled output data, and
the third batch normalization comprises performing third batch normalization on the pooled output data in which the channel is duplicated.
However, Zhang—directed to analogous art—teaches
wherein the pooling comprises performing pooling on the second batch normalized output data, (As taught by Bethge, the transition block (downsample block) comes after the basic block (normal block). Therefore, the last BatchNorm of the normal block in Fig. 5 “Normal Block”, is interpreted as the second batch normalized output as it normalizes the output of the activation function. The next layer in the downsample block, as can be seen in Fig. 5 “Downsample Block”, taking the shortcut, is the AvgPool 2x2 layer, which performs pooling.)
the channel upscaling comprises duplicating a channel of the pooled output data, and (As can be seen in Fig. 5 “Downsample Block”, the result of the AvgPool 2x2 layer is batch normalized then the channel, taking the shortcut, is duplicated. Page 173 states "In the downsample layer, the average pooling layer and the channel duplication ensure the shortcut matches the spatial and channel dimensions of the residual.")
the third batch normalization comprises performing third batch normalization on the pooled output data in which the channel is duplicated. (As can be seen in Fig. 5 “Downsample Block”, the duplicated channel is batch normalized again.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang for the reasons given above in regards to claim 19.
Regarding claim 21, the rejection of claim 20 is incorporated herein. Bethge teaches
wherein the pooling comprises performing downsampling on a width and height of the [output of the basic block] (Page 5 states "Our network progresses through four stages, with transition layers in between, which halve the height and width of the feature map with a MaxPool layer." As the transition layers come after the basic blocks, the output data width and height from the basic block are halved.
Bethge does not appear to explicitly teach
second batch normalized output data.
However, Zhang—directed to analogous art—teaches
second batch normalized output data. (The final BatchNorm layer of the “Normal Block” on page 173, Fig. 5 shows that the result of the output data is batch normalized.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Bethge and Zhang for the reasons given above in regards to claim 19.
Regarding claim 22, the rejection of claim 19 is incorporated herein. Bethge teaches
performing the first neural network operation a plurality of times, wherein the second neural network operation is performed based on a result of the performing of the first neural network a plurality of times. (As can be seen in Fig. 1b, the improvement blocks, interpreted as basic blocks, are immediately followed by transition blocks. Therefore, the execution of these layers will flow from the basic blocks (first operations) to the transition blocks (second operations). As one of ordinary skill in the art would understand, the output of a layer is used in the next layer of a neural network. Therefore, the second neural network operation (execution of the transition block) is performed based on a result of the performing of the first neural network operation (execution of the basic block) as it follows the first neural network operation.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA THUY PHAM whose telephone number is (571)272-2605. The examiner can normally be reached Monday - Friday, 9 A.M. - 5:00 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.P./Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights