METHODS AND APPARATUS TO DYNAMICALLY NORMALIZE DATA IN NEURAL NETWORKS

Application No. 18/005,804

• Filed: Jan 17, 2023
• Priority: Aug 26, 2020 — nonprovisional of PCTCN2020111272
• Examiner: GORMLEY, AARON PATRICK
• Art Unit: 2148
• Tech Center: 2100 — Computer Architecture & Software
• Assignee: Intel Corporation
• OA Round: 2 (Final)

Office Action

DETAILED ACTION
This action is in response to the amendments filed 05/18/2026. Claims 1-3, 5-8, 10-13, 15-21, and 26-32 are pending and have been examined.

Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .

Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.

The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.

Claim 31 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.

Claim 31 recites “wherein the feature vector has a number of elements equal to a number of channels of the input data”. While the instant specification discloses “the size of the input sample (e.g., the input data 202) is defined as the number of channels times the height of the sample (e.g., input image, intermediate feature map, etc.) times the width of the sample (e.g., C x H x W).” ([0030]), it fails to disclose a feature vector with a number of elements equal to a number of input data channels. Thus, claim 31 contains new matter not described in the instant specification, and fails to comply with the written description requirement.

Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.

The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.

Claims 1-2, 5-8, 10-13, 15-18, 27, 30-32 are rejected under 35 U.S.C. 103 as being unpatentable over Luo et al. (Switchable Normalization for Learning-to-Normalize Deep Representation, published July 22nd, 2019, arXiv:1907.10473v1), hereafter referred to as SN Luo, in view of Wang (Semi- Supervised Domain Adaptation for Weakly Labeled Semantic Video Object Segmentation, published 6/7/2016, arXiv:1606.02280v1), and further in view of Nemlekar (FUSED CONVOLUTION AND BATCH NORMALIZATION FOR NEURAL NETWORKS, published 6/18/2020, US 20200192631 A1).

Regarding claim 1, SN Luo discloses [a]n apparatus for use with a machine learning model, the apparatus comprising … instructions to at least:
generate a plurality of alternate normalized outputs associated with input data for the machine learning model, different ones of the alternate normalized outputs based on different normalization techniques:
“We take CNN (machine learning model) as an illustrative example. Let h be the input data of an arbitrary normalization layer represented by a 4D tensor (N, C, H, W), indicating number of samples, number of channels, height and width of a channel respectively.” (SN Luo, page 3, left column, paragraph 6)
“Let

h

n
c
i
j

(input data / feature vector) … be a pixel before … normalization” (SN Luo, page 3, right column, paragraph 1)
“we compare SN with five popular normalization methods, i.e. BN, IN, LN, GN and WN” (SN Luo, page 2, right column, paragraph 4)
“In general, we have
PNG
media_image1.png
179
610
media_image1.png
Greyscale
where
k
∈
{
i
n
,
l
n
,
b
n
}

is used to distinguish different methods.

I

k

is a set pixels [sic] and
|

I

k

|

denotes the number of pixels. Specifically,

I

i
n

,

I

l
n

, and

I

b
n

are the sets of pixels used to compute statistics in different approaches” (SN Luo, page 3, right column, paragraph 3). These functions together comprise a mapping between the feature vectors (input pixels) and k sets of statistics (

μ

k

and

σ

k

(plurality of alternate normalized outputs) across all k normalization techniques).
map the feature vector to a weight vector having a number of elements equal to a number of the different normalization techniques, each element of the weight vector corresponding to a respective one of the different normalization techniques; and generate a final normalized output as a weighted combination of the plurality of alternate normalized outputs using the weight vector:
“Furthermore,

w

k

(weight vector) and

w

k

'

(weight vector) in Eqn.(3) are importance ratios used to weighted average the means and variances respectively. Each

w

k

or

w

k

'

is a scalar variable, which is shared across all channels. There are 3 x 2 = 6 importance weights in SN. We have

∑

k
∈
Ω

w

k

=
1

,

∑

k
∈
Ω

w

k

'

=
1

and
∀

w

k

,

w

k

'

∈

0,1

, and define
PNG
media_image2.png
81
579
media_image2.png
Greyscale
Here each

w

k

is computed by using a softmax function with

λ

i
n

,

λ

l
n

, and

λ

b
n

as the control parameters ” (SN Luo, page 4, left column, paragraph 2). Two weight vectors are formed, each containing k statistic values, one for each normalization method.
“Let …

h

^

n
c
i
j

be a pixel … after normalization” (SN Luo, page 3, right column, paragraph 1)
“SN has an intuitive expression
PNG
media_image3.png
96
613
media_image3.png
Greyscale
where
Ω

is a set of statistics estimated in different ways. In this work, we define
Ω
=
{
i
n
,
l
n
,
b
n
}

the same as above where

μ

k

and

σ

k

(plurality of alternate normalized outputs) can be calculated by following Eqn. (2)” (SN Luo, page 4, left column, paragraph 1).

∑

k
∈
Ω

w

k

μ

k

and

∑

k
∈
Ω

w

k

'

σ

k

2

are weighted combinations of the alternate normalized outputs.
While SN Luo fails to disclose the further limitations of the claim, Wang discloses instructions to aggregate the input data into a feature vector:
“Spatial Average Pooling
After the initial discovery, a large number of region proposals are positively detected with regard to a class label, which include overlapping regions on the same objects and spurious detections. We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal

r

i

, we rescore it by multiplying its score and classification confidence, which is denoted by

s

~

r

i

=

s

r

i

∙

c

r

i

. We then generate score map

S

r

i

of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score

s

~

r

i

. We perform an average pooling over the score maps of all the proposals to compute a confidence map” (Wang, page 6, paragraph 1). As noted by paragraph [0030] of the instant specification, spatial average pooling is a viable implementation of the aggregation operation.

PNG
media_image4.png
200
400
media_image4.png
Greyscale
”An illustration of the weighted spatial average pooling strategy” (Wang, page 6, Fig. 2)
Wang relates to adaptive convolutional neural networks and is analogous to the claimed invention. The existing combination teaches an apparatus that has unique composite normalization for data at each layer. Wang teaches an apparatus that performs spatial average pooling on CNN data. It would have been obvious to one of ordinary skill in the art to combine the existing combination and Wang by pooling SN Luo’s input data before further processing. This would achieve the predictable result of emphasizing the most important aspects of the images while ensuring their compatibility with further processing in the CNN, with SN Luo’s normalization and Wang’s pooling performing the same together as they did separately. (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
While Wang fails to disclose the further limitations of the claim, Nemlekar discloses an apparatus comprising: memory: machine readable instructions: and at least one processor circuit to at least one of instantiate or execute the machine readable instructions: “In some embodiments, certain aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.” (Nemlekar, [0029])
Nemlekar relates to normalization in convolutional neural networks and is analogous to the claimed invention. The existing combination teaches an apparatus for performing normalization in convolutional neural networks. The claimed invention improves upon this method by storing it in the form of instructions on computer hardware. Nemlekar teaches computer hardware for normalization in CNNs, applicable to the existing combination. A person of ordinary skill in the art would have recognized that storing the existing combination’s method as computer instructions on Nemlekar’s hardware would lead to the predictable result of the method being executable by a computing system, and would improve the known device by allowing it to be performed with real data (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).

Regarding claim 2, the rejection of claim 1 is incorporated. SN Luo further discloses instructions wherein the final normalized output is a sum of a product of values of the weight vector and respective ones of the plurality of alternate normalized outputs: “SN has an intuitive expression
PNG
media_image3.png
96
613
media_image3.png
Greyscale
where
Ω

is a set of statistics estimated in different ways. In this work, we define
Ω
=
{
i
n
,
l
n
,
b
n
}

the same as above where

μ

k

and

σ

k

(plurality of alternate normalized outputs) can be calculated by following Eqn. (2).” (SN Luo, page 4, left column, paragraph 1).

∑

k
∈
Ω

w

k

μ

k

and

∑

k
∈
Ω

w

k

'

σ

k

2

are each a sum of a product of values of the weight vector and alternate normalized outputs.

Regarding claim 5, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein one or more of the at least one processor circuit is to scale the values in the weight vector: “
γ

and
β

are a scale and a shift parameter respectively“ (SN Luo, page 3, right column, paragraph 1); “SN has an intuitive expression
PNG
media_image3.png
96
613
media_image3.png
Greyscale
where
Ω

is a set of statistics estimated in different ways. In this work, we define
Ω
=
{
i
n
,
l
n
,
b
n
}

the same as above where

μ

k

and

σ

k

can be calculated by following Eqn. (2).” (SN Luo, page 4, left column, paragraph 1)

Regarding claim 6, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein the machine learning model is a neural network with multiple layers: “By enabling each normalization layer in a deep network (neural network) to have its own operation, SN helps ease the usage of normalizers, pushes the frontier of normalization in deep learning” (SN Luo, page 2, left column, paragraph 2); “There are 53 SN layers.” (SN Luo, page 7, Fig. 5).

Regarding claim 7, the rejection of claim 6 is incorporated. SN Luo further discloses an apparatus, wherein the input data is first input data for a first layer in the neural network and weight vector is a first weight vector, one or more of the at least one processor circuit is to generate a second weight vector based on second input data for a second layer in the neural network, the second input data based on the final normalized output:
Examiner’s note: As discussed regarding parent claim 1, weights are generated based on input data.

PNG
media_image5.png
627
869
media_image5.png
Greyscale
”There are 53 SN layers. (a, b) show the importance weights for
μ

and
σ

of (8, 32), while (c, d) show those of (8, 2). The y-axis represents the importance weights that sum to 1, while the x-axis shows different residual blocks of ResNet50. The SN layers in different places are highlighted differently. For example, the SN layers follow the
3
×
3

conv layers are outlined by shaded color, those in the shortcuts are marked with ‘■’, while those follow the
1
×
1

conv layers are in flat color. The first SN layer follows a
7
×
7

conv layer. We see that SN learns distinct importance weights for different normalization methods as well as
μ

and
σ

, adapting to different batch sizes, places, and depths of a deep network” (SN Luo, page 7, Fig. 5). Each layer of the network has a unique input. As shown in figure 5, this results in each layer having a unique weight vector. Any two layers can be labeled as having a first and second weight vector. The input of one layer is based on the normalized outputs of the previous layers.

Regarding claim 8, the rejection of claim 7 is incorporated. SN Luo further discloses an apparatus, wherein the plurality of alternate normalized outputs is a first plurality of alternate normalized outputs associated with the first layer in the neural network and the final normalized output is a first final normalized output associated with the first layer in the neural network, one or more of the at least one processor circuit is to generate a second plurality of alternate normalized outputs associated with second input data, and generate a second final normalized output as a weighted combination of the second plurality of alternate normalized outputs using the second weight vector:

PNG
media_image5.png
627
869
media_image5.png
Greyscale
”There are 53 SN layers” (SN Luo, page 7, Fig. 5). In this example, SN Luo’s method (switchable normalization) is performed 53 times, for each of 53 layers. That necessarily includes calculating 53 sets of final normalized outputs accordingly, including a first and second calculation.
“SN has an intuitive expression
PNG
media_image3.png
96
613
media_image3.png
Greyscale
where
Ω

is a set of statistics estimated in different ways. In this work, we define
Ω
=
{
i
n
,
l
n
,
b
n
}

the same as above where

μ

k

and

σ

k

(plurality of alternate normalized outputs) can be calculated by following Eqn. (2).” (SN Luo, page 4, left column, paragraph 1).

∑

k
∈
Ω

w

k

μ

k

and

∑

k
∈
Ω

w

k

'

σ

k

2

are weighted combinations of the alternate normalized outputs.. As discussed regarding parent claim 7, each layer has a unique plurality of weights resulting from its unique input.
“In general, we have
PNG
media_image1.png
179
610
media_image1.png
Greyscale
” (SN Luo, page 3, right column, paragraph 3). The value of the alternate normalized outputs is dependent on the input of the layer being normalized (h).

Regarding claim 10, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein the values of the weight vector correspond to soft weights with values that may differ along a range from 0 to 1:

PNG
media_image6.png
87
563
media_image6.png
Greyscale
”Here each

w

k

is computed by using a softmax function” (SN Luo, page 4, left column, paragraph 2).
Examiner’s note: Continuous weight values, as shown here, fall within the definition given by paragraph [0028] of the instant specification: “As used herein, the term ‘soft,’ used in the context of ‘soft weights,’ means that the weights are given a value on a continuous scale rather than being defined as one of different discrete values”.
Examiner’s note: As known to one of ordinary skill in the art, softmax scales values between 0 and 1.

Regarding claim 11, SN Luo discloses instructions to:
generate a plurality of alternate normalized outputs associated with input data for the machine learning model, different ones of the alternate normalized outputs based on different normalization techniques:
“We take CNN (machine learning model) as an illustrative example. Let h be the input data of an arbitrary normalization layer represented by a 4D tensor (N, C, H, W), indicating number of samples, number of channels, height and width of a channel respectively.” (SN Luo, page 3, left column, paragraph 6)
“Let

h

n
c
i
j

(input data) … be a pixel before … normalization” (SN Luo, page 3, right column, paragraph 1)
“we compare SN with five popular normalization methods, i.e. BN, IN, LN, GN and WN” (SN Luo, page 2, right column, paragraph 4)
“In general, we have
PNG
media_image1.png
179
610
media_image1.png
Greyscale
where
k
∈
{
i
n
,
l
n
,
b
n
}

is used to distinguish different methods.

I

k

is a set pixels [sic] and
|

I

k

|

denotes the number of pixels. Specifically,

I

i
n

,

I

l
n

, and

I

b
n

are the sets of pixels used to compute statistics in different approaches” (SN Luo, page 3, right column, paragraph 3).

μ

k

and

σ

k

are a plurality of alternate normalized outputs.
map the feature vector to a weight vector having a number of elements equal to a number of the different normalization techniques each element of the weight vector corresponding to a respective one of the different normalization techniques; and generate a final normalized output as a weighted combination of the plurality of alternate normalized outputs using the weight vector:
“Furthermore,

w

k

(weight vector) and

w

k

'

(weight vector) in Eqn.(3) are importance ratios used to weighted average the means and variances respectively. Each

w

k

or

w

k

'

is a scalar variable, which is shared across all channels. There are 3 x 2 = 6 importance weights in SN. We have

∑

k
∈
Ω

w

k

=
1

,

∑

k
∈
Ω

w

k

'

=
1

and
∀

w

k

,

w

k

'

∈

0,1

, and define
PNG
media_image2.png
81
579
media_image2.png
Greyscale
Here each

w

k

is computed by using a softmax function with

λ

i
n

,

λ

l
n

, and

λ

b
n

as the control parameters ” (SN Luo, page 4, left column, paragraph 2). Two weight vectors are formed, each containing k statistic values, one for each normalization method.
“Let …

h

^

n
c
i
j

be a pixel … after normalization” (SN Luo, page 3, right column, paragraph 1)
“SN has an intuitive expression
PNG
media_image3.png
96
613
media_image3.png
Greyscale
where
Ω

is a set of statistics estimated in different ways. In this work, we define
Ω
=
{
i
n
,
l
n
,
b
n
}

the same as above where

μ

k

and

σ

k

(plurality of alternate normalized outputs) can be calculated by following Eqn. (2)” (SN Luo, page 4, left column, paragraph 1).

∑

k
∈
Ω

w

k

μ

k

and

∑

k
∈
Ω

w

k

'

σ

k

2

are weighted combinations of the alternate normalized outputs.
While SN Luo fails to disclose the further limitations of the claim, Wang discloses instructions to aggregate the input data into a feature vector:
“Spatial Average Pooling
After the initial discovery, a large number of region proposals are positively detected with regard to a class label, which include overlapping regions on the same objects and spurious detections. We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal

r

i

, we rescore it by multiplying its score and classification confidence, which is denoted by

s

~

r

i

=

s

r

i

∙

c

r

i

. We then generate score map

S

r

i

of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score

s

~

r

i

. We perform an average pooling over the score maps of all the proposals to compute a confidence map” (Wang, page 6, paragraph 1). As noted by paragraph [0030] of the instant specification, spatial average pooling is a viable implementation of the aggregation operation.

PNG
media_image4.png
200
400
media_image4.png
Greyscale
”An illustration of the weighted spatial average pooling strategy” (Wang, page 6, Fig. 2)
Wang relates to adaptive convolutional neural networks and is analogous to the claimed invention. The existing combination teaches an apparatus that has unique composite normalization for data at each layer. Wang teaches an apparatus that performs spatial average pooling on CNN data. It would have been obvious to one of ordinary skill in the art to combine the existing combination and Wang by pooling SN Luo’s input data before further processing. This would achieve the predictable result of emphasizing the most important aspects of the images while ensuring their compatibility with further processing in the CNN, with SN Luo’s normalization and Wang’s pooling performing the same together as they did separately. (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
While Wang fails to disclose the further limitations of the claim, Nemlekar discloses [a]t least one non-transitory computer readable medium comprising instructions that, when executed, cause at least one processor to at least: “In some embodiments, certain aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.” (Nemlekar, [0029])
Nemlekar relates to normalization in convolutional neural networks and is analogous to the claimed invention. The existing combination teaches an apparatus for performing normalization in convolutional neural networks. The claimed invention improves upon this method by storing it in the form of instructions on computer hardware. Nemlekar teaches computer hardware for normalization in CNNs, applicable to The existing combination. A person of ordinary skill in the art would have recognized that storing The existing combination’s method as computer instructions on Nemlekar’s hardware would lead to the predictable result of the method being executable by a computing system, and would improve the known device by allowing it to be performed with real data (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).

The analysis of claims 12-13 & 16-18 mirrors that of claims 2-3 & 6-8, with the exception that claims 12-13 & 16-18 are directed to generic computer hardware which executes the methods of claims 2-3 & 6-8. This generic hardware is taught by Nemlekar, as discussed regarding claim 11. Thus, claims 12-13 & 16-18 are rejected under the same rationales used for claims 2-3 & 6-8, respectively.
The analysis of claim 15 mirrors that of claim 5, with the exception that claim 15 is directed to generic computer hardware which executes the methods of claim 5. This generic hardware is taught by Nemlekar, as discussed regarding claim 11. Thus, claim 15 is rejected under the same rationales used for claim 5.

Regarding claim 27, the rejection of claim 1 is incorporated. Wang further discloses an apparatus, wherein one or more of the at least one processor circuit is to aggregate the input data into the feature vector using spatial average pooling over spatial dimensions of the input data: “Spatial Average Pooling After the initial discovery, a large number of region proposals are positively detected with regard to a class label, which include overlapping regions on the same objects and spurious detections. We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal

r

i

, we rescore it by multiplying its score and classification confidence, which is denoted by

s

~

r

i

=

s

r

i

∙

c

r

i

. We then generate score map

S

r

i

of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score

s

~

r

i

. We perform an average pooling over the score maps of all the proposals to compute a confidence map” (Wang, page 6, paragraph 1).
Wang relates to adaptive convolutional neural networks and is analogous to the claimed invention. The existing combination teaches an apparatus that has unique composite normalization for data at each layer. Wang teaches an apparatus that performs spatial average pooling on CNN data. It would have been obvious to one of ordinary skill in the art to combine the existing combination and Wang by pooling the existing combination’s input data before further processing. This would achieve the predictable result of emphasizing the most important aspects of the images while ensuring their compatibility with further processing in the CNN, with the existing combination’s normalization and Wang’s pooling performing the same together as they did separately. (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).

Regarding claim 30, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein one or more of the at least one processor circuit is to generate the weight vector independently of statistics computed by the different normalization techniques:
“In general, we have
PNG
media_image1.png
179
610
media_image1.png
Greyscale
” (SN Luo, page 3, right column, paragraph 3). The formula for calculating statistics of the different normalization techniques.

PNG
media_image6.png
87
563
media_image6.png
Greyscale
(SN Luo, page 4, left column, paragraph 2). The formula for generating the weight vectors.
Examiner’s note: The formula for weight generation doesn’t include any normalization statistics, nor do the formulas for the normalization statistics include weights. Thus, these operations are independent.

Regarding claim 31, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein the feature vector has a number of elements equal to a number of channels of the input data: “In IN, we have

μ

i
n

,

σ

i
n

2

∈

R

N
×
C

and

I

l
n

=
{
(
c
,
i
,
j
)
|
c
∈

1
,
C

,

i
∈

1
,
H

,

j
∈
[
1
,
W
]

, meaning that IN has 2NC elements of statistics, where each mean and variance value is computed along (H, W) for each channel of each sample” (SN Luo, page 3, right column, paragraph 3).

Regarding claim 32, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein to scale the values in the weight vector, one or more of the at least one processor circuit is to apply a softmax function to the values such that the scaled values sum to one: “each

w

k

(weight vector) is computed by using a softmax function with

λ

i
n

,

λ

l
n

, and

λ

b
n

as the control parameters, which can be learned by back-propagation (BP).

w

k

'

(weight vector) are defined similarly by using another three control parameters

λ

i
n

'

,

λ

l
n

'

, and

λ

b
n

'

” (SN Luo, page 4, left column, paragraph 2). As one of ordinary skill in the art would know, a set of values calculated with softmax will sum to one.

Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Luo et al. (Switchable Normalization for Learning-to-Normalize Deep Representation, published July 22nd, 2019, arXiv:1907.10473v1), hereafter referred to as SN Luo, in view of Wang (Semi- Supervised Domain Adaptation for Weakly Labeled Semantic Video Object Segmentation, published 6/7/2016, arXiv:1606.02280v1), and further in view of Nemlekar (FUSED CONVOLUTION AND BATCH NORMALIZATION FOR NEURAL NETWORKS, published 6/18/2020, US 20200192631 A1), and Shih (HYBRID AUTONOMY SYSTEM FOR AUTONOMOUS AND AUTOMATED DELIVERY VEHICLE, filed 6/30/2020, US 2021/0403024 A1).

Regarding claim 3, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein the input data is first input data, the weight vector is a first weight vector, and one or more of the at least one processor circuit is to aggregate second input data for a second layer in the machine learning model into a second feature vector and map the second feature vector to a second weight vector, the second input data based on the final normalized output:
Examiner’s note: As discussed regarding parent claim 1, weights are generated based on input data.

PNG
media_image5.png
627
869
media_image5.png
Greyscale
”There are 53 SN layers. (a, b) show the importance weights for
μ

and
σ

of (8, 32), while (c, d) show those of (8, 2). The y-axis represents the importance weights that sum to 1, while the x-axis shows different residual blocks of ResNet50. The SN layers in different places are highlighted differently. For example, the SN layers follow the
3
×
3

conv layers are outlined by shaded color, those in the shortcuts are marked with ‘■’, while those follow the
1
×
1

conv layers are in flat color. The first SN layer follows a
7
×
7

conv layer. We see that SN learns distinct importance weights for different normalization methods as well as
μ

and
σ

, adapting to different batch sizes, places, and depths of a deep network” (SN Luo, page 7, Fig. 5). Each layer of the network has a unique input. As shown in figure 5, this results in each layer having a unique weight vector. Any two layers can be labeled as having a first and second weight vector. The input of one layer is based on the outputs of the previous layers, generated using final normalized outputs, as discussed regarding claim 1.
While SN Luo fails to disclose the further limitations of the claim, Shih discloses instructions to aggregate second input data for a second layer in the machine learning model into a second feature vector: “In an example embodiment, the neural network may comprise a series of convolution layers and pooling layers. In some embodiments, one or more convolution layers are followed by a pooling layer and the output of each layer is input into the subsequent layer. In some embodiments, each pooling layer is followed by a series of convolution layers which take the downsampled output of the pooling layer as input.” (Shih, [0114])
Shih relates to adaptive convolutional neural networks and is analogous to the claimed invention. The existing combination teaches an apparatus that has unique composite normalization for data at each layer. Wang teaches an apparatus that performs spatial average pooling on CNN data. It would have been obvious to one of ordinary skill in the art to combine the existing combination and Shih by pooling the existing combination’s input data at each stage of pixel processing. This would achieve the predictable result of emphasizing the most important aspects of the images at each stage of pixel processing while ensuring compatibility with further layers in the CNN, with the existing combination’s normalization and Shih’s pooling performing the same together as they did separately. (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).

Claims 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Luo et al. (Switchable Normalization for Learning-to-Normalize Deep Representation, published July 22nd, 2019, arXiv:1907.10473v1), hereafter referred to as SN Luo, in view of Wang (Semi- Supervised Domain Adaptation for Weakly Labeled Semantic Video Object Segmentation, published 6/7/2016, arXiv:1606.02280v1).

Regarding claim 19, SN Luo discloses [a] method for using a machine learning model, the method comprising:
generating a plurality of alternate normalized outputs associated with input data for the machine learning model, different ones of the alternate normalized outputs based on different normalization techniques:
“We take CNN (machine learning model) as an illustrative example. Let h be the input data of an arbitrary normalization layer represented by a 4D tensor (N, C, H, W), indicating number of samples, number of channels, height and width of a channel respectively.” (SN Luo, page 3, left column, paragraph 6)
“Let

h

n
c
i
j

(input data) … be a pixel before … normalization” (SN Luo, page 3, right column, paragraph 1)
“we compare SN with five popular normalization methods, i.e. BN, IN, LN, GN and WN” (SN Luo, page 2, right column, paragraph 4)
“In general, we have
PNG
media_image1.png
179
610
media_image1.png
Greyscale
where
k
∈
{
i
n
,
l
n
,
b
n
}

is used to distinguish different methods.

I

k

is a set pixels [sic] and
|

I

k

|

denotes the number of pixels. Specifically,

I

i
n

,

I

l
n

, and

I

b
n

are the sets of pixels used to compute statistics in different approaches” (SN Luo, page 3, right column, paragraph 3).

μ

k

and

σ

k

encompass a plurality of alternate normalized outputs.
map the feature vector to a weight vector having a number of elements equal to a number of the different normalization techniques each element of the weight vector corresponding to a respective one of the different normalization techniques, the weight vector generated independently of the plurality of alternate normalized outputs; and generate a final normalized output as a weighted combination of the plurality of alternate normalized outputs using the weight vector:
“Furthermore,

w

k

(weight vector) and

w

k

'

(weight vector) in Eqn.(3) are importance ratios used to weighted average the means and variances respectively. Each

w

k

or

w

k

'

is a scalar variable, which is shared across all channels. There are 3 x 2 = 6 importance weights in SN. We have

∑

k
∈
Ω

w

k

=
1

,

∑

k
∈
Ω

w

k

'

=
1

and
∀

w

k

,

w

k

'

∈

0,1

, and define
PNG
media_image2.png
81
579
media_image2.png
Greyscale
Here each

w

k

is computed by using a softmax function with

λ

i
n

,

λ

l
n

, and

λ

b
n

as the control parameters ” (SN Luo, page 4, left column, paragraph 2). Two weight vectors are formed, each containing k statistic values, one for each normalization method.
Examiner’s note: The formula for weight generation doesn’t include any alternate normalized outputs, nor do the formulas for the alternate normalized outputs include weights. Thus, these operations are independent.
“Let …

h

^

n
c
i
j

be a pixel … after normalization” (SN Luo, page 3, right column, paragraph 1)
“SN has an intuitive expression
PNG
media_image3.png
96
613
media_image3.png
Greyscale
where
Ω

is a set of statistics estimated in different ways. In this work, we define
Ω
=
{
i
n
,
l
n
,
b
n
}

the same as above where

μ

k

and

σ

k

(plurality of alternate normalized outputs) can be calculated by following Eqn. (2)” (SN Luo, page 4, left column, paragraph 1).

∑

k
∈
Ω

w

k

μ

k

and

∑

k
∈
Ω

w

k

'

σ

k

2

are weighted combinations of the alternate normalized outputs.
While SN Luo fails to disclose the further limitations of the claim, Wang discloses instructions to aggregate the input data into a feature vector:
“Spatial Average Pooling
After the initial discovery, a large number of region proposals are positively detected with regard to a class label, which include overlapping regions on the same objects and spurious detections. We adopt a simple weighted spatial average pooling strategy to aggregate the region-wise score, confidence as well as their spatial extent. For each proposal

r

i

, we rescore it by multiplying its score and classification confidence, which is denoted by

s

~

r

i

=

s

r

i

∙

c

r

i

. We then generate score map

S

r

i

of the size of image frame, which is composited as the binary map of current region proposal multiplied by its score

s

~

r

i

. We perform an average pooling over the score maps of all the proposals to compute a confidence map” (Wang, page 6, paragraph 1). As noted by paragraph [0030] of the instant specification, spatial average pooling is a viable implementation of the aggregation operation.

PNG
media_image4.png
200
400
media_image4.png
Greyscale
”An illustration of the weighted spatial average pooling strategy” (Wang, page 6, Fig. 2)
Wang relates to adaptive convolutional neural networks and is analogous to the claimed invention. The existing combination teaches an apparatus that has unique composite normalization for data at each layer. Wang teaches an apparatus that performs spatial average pooling on CNN data. It would have been obvious to one of ordinary skill in the art to combine the existing combination and Wang by pooling SN Luo’s input data before further processing. This would achieve the predictable result of emphasizing the most important aspects of the images while ensuring their compatibility with further processing in the CNN, with SN Luo’s normalization and Wang’s pooling performing the same together as they did separately. (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).

The methods of claims 20-21 mirror the apparatus operations of claims 2-3, respectively. Thus, claims 20-21 are rejected under the same rationales used for claims 2-3, respectively.

Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Luo et al. (Switchable Normalization for Learning-to-Normalize Deep Representation, published July 22nd, 2019, arXiv:1907.10473v1), hereafter referred to as SN Luo, in view of Wang (Semi- Supervised Domain Adaptation for Weakly Labeled Semantic Video Object Segmentation, published 6/7/2016, arXiv:1606.02280v1), and further in view of Nemlekar (FUSED CONVOLUTION AND BATCH NORMALIZATION FOR NEURAL NETWORKS, published 6/18/2020, US 20200192631 A1), and Liu (DISENTANGLED REPRESENTATION LEARNING GENERATIVE ADVERSARIAL NETWORK FOR POSE-INVARIANT FACE RECOGNITION, published 8/20/2020, US 2020/0265219 A1).

Regarding claim 26, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein one or more of the at least one processor circuit is to generate the weight vector for each input sample processed by the machine learning model during a training phase:
“Let h be the input data of an arbitrary normalization layer represented by a 4D tensor (N, C, H, W), indicating number of samples, number of channels, height and width of a channel respectively” (SN Luo, page 3, left column, paragraph 6)
“SN has an intuitive expression
PNG
media_image3.png
96
613
media_image3.png
Greyscale
where
Ω

is a set of statistics estimated in different ways. In this work, we define
Ω
=
{
i
n
,
l
n
,
b
n
}

the same as above where

μ

k

and

σ

k

can be calculated by following Eqn. (2)” (SN Luo, page 4, left column, paragraph 1).
“Furthermore,

w

k

(weight vector) and

w

k

'

(weight vector) in Eqn.(3) are importance ratios used to weighted average the means and variances respectively. Each

w

k

or

w

k

'

is a scalar variable, which is shared across all channels. There are 3 x 2 = 6 importance weights in SN. We have

∑

k
∈
Ω

w

k

=
1

,

∑

k
∈
Ω

w

k

'

=
1

and
∀

w

k

,

w

k

'

∈

0,1

, and define
PNG
media_image2.png
81
579
media_image2.png
Greyscale
Here each

w

k

is computed by using a softmax function with

λ

i
n

,

λ

l
n

, and

λ

b
n

as the control parameters, which can be learned by back-propagation (BP) (training).

w

k

'

are defined similarly” (SN Luo, page 4, left column, paragraph 2).
While SN Luo fails to disclose the further limitations of the claim, Liu discloses an apparatus, wherein one or more of the at least one processor circuit is to generate the weight vector for each input sample processed by the machine learning model during a training phase and during an inference phase of the machine learning model, the weight vector varying between different input samples during the inference phase:
“Given a real face image x or a synthetic face image (input sample)

x

^

=G(x,c,z), D estimates the identity and pose of

x

^

, or classifies

x

^

as fake … To this end, D may be trained to optimize this objective function
PNG
media_image7.png
114
875
media_image7.png
Greyscale
” (Liu, [0056])
“D may be trained to optimize Eqn. 4, by defining the softmax loss for Nd+ 1 and NP classes for identity and pose, respectively” (Liu, [0059])
“in the testing stage (inference phase), the output (vector) of the softmax layer could be used as the probability of each testing image (input sample) belonging to the right identity class” (Liu, [0109])
Liu relates to processing, pooling, and softmax categorization and is analogous to the claimed invention. The existing combination teaches a method of determining normalization weights via softmax during training. The claimed invention improves upon this method by varying normalization weights at inference time. Liu teaches a method of calculating softmax values on test data during inference time, applicable to the existing combination. A person of ordinary skill in the art would have recognized that calculating weights via softmax during inference time would lead to the predictable result of weights being tailored for inference data unseen during training, and would improve the known device by making normalization weights more accurate for test data distribution shift (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).

Claim 28 is rejected under 35 U.S.C. 103 as being unpatentable over Luo et al. (Switchable Normalization for Learning-to-Normalize Deep Representation, published July 22nd, 2019, arXiv:1907.10473v1), hereafter referred to as SN Luo, in view of Wang (Semi- Supervised Domain Adaptation for Weakly Labeled Semantic Video Object Segmentation, published 6/7/2016, arXiv:1606.02280v1), and further in view of Nemlekar (FUSED CONVOLUTION AND BATCH NORMALIZATION FOR NEURAL NETWORKS, published 6/18/2020, US 20200192631 A1), and Tang (METHOD AND SYSTEM FOR FACE IMAGE RECOGNITION, published 12/27/2016, US 9,530,047 B1).

Regarding claim 28, the rejection of claim 1 is incorporated. Tang, in combination with SN Luo, discloses an apparatus, wherein one or more of the at least one processor circuit is to map the feature vector to the weight vector using a fully connected layer having a number of output units equal to the number of the different normalization techniques:
“Each convolutional neural network takes a pair of aligned face regions as input. Its convolutional layers extract the identity relational features hierarchically. the extracted relational features pass a fully connected layer and are fully connected to an output layer, such as the softmax layer, which indicates whether the two regions belong to the same identity, as shown in FIG. 4” (Tang, column 4, paragraph 1)

PNG
media_image8.png
233
715
media_image8.png
Greyscale
(Tang, Figure 4). The softmax layer has two output units, one for each softmax category.
Examiner’s note: SN Luo calculates a normalization statistic for each normalization technique utilizing softmax. As one of ordinary skill in the art would know, this would result in a softmax output unit for each normalization technique.
Tang relates to data processing, pooling, and categorization using softmax and is analogous to the claimed invention. The existing combination teaches a neural network that produces normalization technique weights using softmax. Tang teaches a method of implementing softmax by connecting a fully-connected layer to a softmax layer with a number of output units equal to softmax categories. It would have been obvious to one of ordinary skill in the art to combine the existing combination and Tang by structuring the existing combination’s network’s final layer accordingly. This would achieve the predictable result of having an output node for each distinct softmax category, with the existing combination’s softmax calculations and Tang’s network structure performing the same together as they did separately. (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).

Claim 29 is rejected under 35 U.S.C. 103 as being unpatentable over Luo et al. (Switchable Normalization for Learning-to-Normalize Deep Representation, published July 22nd, 2019, arXiv:1907.10473v1), hereafter referred to as SN Luo, in view of Wang (Semi- Supervised Domain Adaptation for Weakly Labeled Semantic Video Object Segmentation, published 6/7/2016, arXiv:1606.02280v1), and further in view of Nemlekar (FUSED CONVOLUTION AND BATCH NORMALIZATION FOR NEURAL NETWORKS, published 6/18/2020, US 20200192631 A1), and Luo et al. (Differentiable Dynamic Normalization for Learning Deep Representation, published 2019, Proceedings of the 36th International Conference on Machine Learning, PMLR 97:4203-4211), hereafter referred to as DN Luo.

Regarding claim 29, the rejection of claim 1 is incorporated. SN Luo further discloses an apparatus, wherein the different normalization techniques include batch normalization, instance normalization, layer normalization, and group normalization: “In recent years, many normalization methods such as Batch Normalization (BN) [1], Instance Normalization (IN) [2], and Layer Normalization (LN) [3] have been developed” (SN Luo, page 1, left column, paragraph 1); “To address the above issues, we propose Switchable Normalization (SN), which combines three types of statistics estimated channel-wise, layer-wise, and minibatch-wise by using IN, LN, and BN respectively” (SN Luo, page 1, left column, paragraph 2)
While SN Luo fails to disclose the further limitations of the claim, DN Luo discloses an apparatus, wherein the different normalization techniques include batch normalization, instance normalization, layer normalization, and group normalization:
“Normalization approaches are indispensable components in recent deep neural networks (DNNs), such as batch normalization (BN) (Ioffe & Szegedy, 2015), layer normalization (LN) (Ba et al., 2016), instance normalization (IN) (Ulyanov et al., 2016), and group normalization (GN) (Wu & He, 2018)” (DN Luo, page 1, left column, paragraph 1)
“SN (Luo et al., 2019a; Shao et al., 2019) linearly combines the statistics of existing methods. The definition of SN is
PNG
media_image9.png
208
650
media_image9.png
Greyscale
”(DN Luo, page 3, left column, paragraph 1)
DN Luo relates to dynamic weighted normalization and is analogous to the claimed invention. the existing combination teaches an apparatus that adaptively combines batch normalization, instance normalization, and layer normalization results together. The claimed invention improves upon this method by including group normalization in the process. DN Luo teaches an apparatus that adaptively combines batch, instance, layer, and group normalization results together, applicable to the existing combination. A person of ordinary skill in the art would have recognized that adding group normalization to the normalization methods would lead to the predictable result of more degrees of flexibility in the weighted combination of normalized features, and would improve the known device by adding a new normalization technique which may be more effective on certain data inputs (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).

Response to Arguments
The following responses address arguments and remarks made in the instant remarks dated 05/18/2026.

Objections
Previous objections to the specification have been withdrawn in light of the instant amendments.

Claim Interpretation
In light of the instant amendments, the claims are no longer interpreted under 35 U.S.C. 112(f).

112 Rejections
In light of the instant amendments, previous rejections under 35 U.S.C. 112(a) and 112(b) are withdrawn. However, new rejections under 35 U.S.C. 112(a) have been made in light of the instant amendments.

101 Rejections
On pages 10-11 of the instant remarks, the Applicant argues that the claimed invention does not recite mental processes:
“Amended claim 1 sets forth at least one processor circuit to aggregate input data into a
feature vector, map that feature vector to a weight vector having a number of elements equal
to the number of different normalization techniques, and generate a final normalized output
as a weighted combination of the alternate normalized outputs using the weight vector. These
operations require processing input data through independent computational pipelines whose
results are combined to produce the final normalized output. Operations of this nature,
applied to input data in a machine learning model, cannot practically be performed in the
human mind. Step 2A Prong One is not satisfied, and the claim is eligible without further
analysis”
In regards to the Applicant’s arguments above, the Examiner respectfully disagrees that claim 1, as amended, recites no mental processes. As stated in MPEP 2106.04(a)(2)(III), The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation. See, e.g., Benson, 409 U.S. at 67, 65, 175 USPQ at 674-75, 674 … Nor do the courts distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer. As the Federal Circuit has explained, "[c]ourts have examined claims that required the use of a computer and still found that the underlying, patent-ineligible invention could be performed via pen and paper or in a person’s mind." Versata Dev. Group v. SAP Am., Inc., 793 F.3d 1306, 1335, 115 USPQ2d 1681, 1702 (Fed. Cir. 2015). See also Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1318, 120 USPQ2d 1353, 1360 (Fed. Cir. 2016) (‘‘[W]ith the exception of generic computer-implemented steps, there is nothing in the claims themselves that foreclose them from being performed by a human, mentally or with pen and paper.’’); Mortgage Grader, Inc. v. First Choice Loan Servs. Inc., 811 F.3d 1314, 1324, 117 USPQ2d 1693, 1699 (Fed. Cir. 2016) (holding that computer- implemented method for "anonymous loan shopping" was an abstract idea because it could be "performed by humans without a computer").
Claim 1 recites limitations amounting to mental processes performed on generic computer hardware, insufficient to render a mentally performable task non-abstract. For example, claim 1 recites the limitation “generate a final normalized output as a weighted combination of the plurality of alternate normalized outputs using the weight vector”, reciting a mental process performed by a generic computer comprising generic memory, machine-readable instructions, and processor circuitry.
The Examiner asserts that claim 1, as amended, recites mental processes.

On pages 11-12 of the instant remarks, the Applicant argues that recited judicial exceptions are integrated into a practical application through an improvement to technology:
“The reasoning of Desjardins also applies here. The specification identifies a specific
technical problem: existing normalization techniques rely on fixed parameters that cannot
adapt to different input samples. (See Specification ¶¶[0016]-[0018]). Amended claim 1 sets
forth a specific technical solution, namely a processor circuit that dynamically generates a
weight vector from the input data through an aggregation and mapping pipeline,
independently of the normalization outputs, to produce an input-conditioned weighted
combination of alternate normalized outputs. The specification demonstrates measurable
improvements in model accuracy over existing normalization approaches. (See Specification ¶¶[0038]-[0039], Tables 1-2). Under Desjardins and Enfish, these specific technological
improvements to the functioning of a machine learning model integrate any recited judicial
exception into a practical application at Step 2A Prong Two”
The Applicant’s arguments above have been fully considered and are persuasive. Previous rejections under 35 U.S.C. 101 have been withdrawn.

102 / 103 Rejections
On pages 12-13 of the instant remarks, the Applicant argues that SN Luo fails to disclose the independent claims, as amended:
“Luo describes a Switchable Normalization (SN) method in which importance weights
are computed according to:
PNG
media_image10.png
72
555
media_image10.png
Greyscale
(Luo, p. 4, left column, paragraph 2), where "each Wk is computed by using a softmax
function with

λ

i
n

,

λ

l
n

, and

λ

b
n

as the control parameters, which can be learned by backpropagation."
(Luo, p. 4, left column, paragraph 2). This weight generation mechanism of
Luo does not teach or suggest "aggregate the input data into a feature vector" or "map the
feature vector to a weight vector having a number of elements equal to a number of the
different normalization techniques, each element of the weight vector corresponding to a
respective one of the different normalization techniques," as set forth in claim 1. In Luo, the
importance weights Wk are derived solely from the learned scalar control parameters A. The
input data tensor of Luo is not aggregated into any feature vector, and no mapping of any
such feature vector to a weight vector is performed. As a result, Luo fails to teach or suggest
the apparatus of claim 1.
…
Independent claim 11 sets forth a computer readable medium including instructions
that cause a processor to "aggregate the input data into a feature vector" and "map the feature
vector to a weight vector having a number of elements equal to a number of the different
normalization techniques, each element of the weight vector corresponding to a respective
one of the different normalization techniques." Luo fails to teach or suggest such a computer
readable medium. Thus, the §102 rejection of claim 11 and all claims dependent thereon
must be withdrawn.
…
Independent claim 19 sets forth a method including aggregating input data into a
feature vector and mapping the feature vector to a weight vector having a number of elements
equal to a number of the different normalization techniques, each element of the weight
vector corresponding to a respective one of the different normalization techniques, the weight
vector generated independently of the plurality of alternate normalized outputs. Luo does not
teach or suggest such a method. Thus, the § 102 rejections of claim 19 and all claims
dependent thereon must be withdrawn.”
Regarding the Applicant’s arguments above, the Examiner agrees that claim 1, as amended, is not disclosed in its entirety by Luo (now named SN Luo in the current office action). However, these deficiencies are remedied by Wang and Nemlekar.
Regarding “at least one processor circuit to at least one of instantiate or execute the machine readable instructions”, Nemlekar discloses an apparatus that uses processors to execute machine readable instructions for CNN normalization methods (Nemlekar, [0029]).
Regarding “aggregate the input data into a feature vector”, Wang discloses a method of using spatial average pooling to aggregate input data images into pooled feature vectors (Wang, page 6, paragraph 1). Paragraph [0030] of the instant specification makes clear that spatial average pooling is a viable implementation of the claimed aggregation operation.
Regarding “map the feature vector to a weight vector having a number of elements equal to a number of the different normalization techniques, each element of the weight vector corresponding to a respective one of the different normalization techniques”, SN Luo discloses a method of generating a pair of weight vectors, each containing a statistic value for each normalization method (SN Luo, page 4, left column, paragraph 2). While SN Luo doesn’t disclose performing this operation using aggregated input data, this deficiency is remedied by Wang, as noted above.
All limitations of amended claim 1 are disclosed by SN Luo in view of Wang and Nemlekar, as noted in more detail in the 103 rejections section. Substantially similar independent claims 11 and 19 are obvious over the prior art under similar rationales. The Examiner further notes for amended claim 19 that the formulas for calculating weight vectors and for calculating normalized outputs are independent from each other. Thus, no rejections are withdrawn on these grounds.

Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
SN Luo et al. (NORMALIZATION METHOD AND APPARATUS FOR DEEP NEURAL NETWORK, AND STORAGE MEDIA, filed 4/29/2020, US 2020/0257979 A1) details a proposed patent for a system highly similar to SN Luo’s “Switchable Normalization” method.
Jia et al. (Instance-Level Meta Normalization, published 4/6/2019, arXiv:1904.03516v1) discloses a method of normalizing data in a CNN with dynamic, separately calculated normalization parameters that can be combined with existing normalization methods
Li et al. (Attentive Normalization, published 11/23/2019, arXiv:1908.01259v2) discloses normalization through a weighted sum of different normalization parameters based on attention

Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.

Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aaron P Gormley whose telephone number is [phone]. The examiner can normally be reached Monday - Friday 12:00 PM - 8:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle T Bechtold can be reached at [phone]. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.

/AG/Examiner, Art Unit 2148
/MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148

Prosecution Timeline

• Jan 17, 2023: Application Filed
• Feb 18, 2026: Non-Final Rejection mailed — §101, §103, §112
• May 18, 2026: Response Filed
• Jul 30, 2026: Final Rejection mailed — §101, §103, §112 (current)