Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-13 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Below is a claim-by-claim analysis.
Claim 1
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites:
performing a sequence of convolutional operations of a quantized convolutional neural network (quantized CNN) on an input video frame using quantized weights to generate feature maps
Can be broadly interpreted as doing matrix math on video data and is considered a mental process that can be done in one’s head (with aid of paper and pencil, MPEP 2106.04(a)(2), III)
applying respective batch normalizations to the feature maps to obtain normalized feature maps, wherein applying a batch normalization to a feature map of the feature maps comprises applying a linear function to the feature map, wherein the linear function includes multiplying each feature of the feature map by a learned scaling factor
the linear scaling of values is a mental process that can be done in one’s head (with aid of paper and pencil, MPEP 2106.04(a)(2), III)
determine a probability that the input video frame is of low quality
a determination is a mental process and is thus considered an abstract idea
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining
limitations of the claim are directed to generally linking the judicial exception to a technological environment (“processing the normalized feature maps through additional layers of the quantized CNN”)
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 2
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on. It also recites:
selecting encoding parameters for a video segment that includes the input video frame based on the probability that the input video frame is of low quality
selecting values based on a determined probability is an abstract idea that can be done in one’s head
Step 2A Prong 2: The judicial exception is not integrated into a practical application. There are no remaining limitations in the claim.
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 3
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on. It also recites:
selecting the encoding parameters for the video segment based on an average of respective probabilities of input video frames that include the input video frame
selecting values based on a determined probability is an abstract idea that can be done in one’s head (with aid of paper and pencil, MPEP 2106.04(a)(2), III)
Step 2A Prong 2: The judicial exception is not integrated into a practical application. There are no remaining limitations in the claim.
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 4
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on. It also recites:
quantized weights are obtained by simulating quantization during a training phase that uses floating-point weights
simulating quantization, as understood by the specification, is simply rounding values prior to computation during training, which is an abstract component that can be done in one’s head (with aid of paper and pencil, MPEP 2106.04(a)(2), III)
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining limitations are directed towards generally linking
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 5
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining limitations are directed to additional components of the judicial exception that aren’t significantly enough (“… batch normalization is fused with a respective convolutional operation”)
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 6
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining limitations are directed to additional components of the judicial exception that aren’t significantly enough (“… batch normalization is performed using fixed-point arithmetic”)
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 7
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining limitations are directed to additional components of the judicial exception that aren’t significantly enough (“… applying a respective quantized activation function after each convolutional operation of at least some of the convolutional operations”)
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 8
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining limitations are directed to additional components of the judicial exception that aren’t significantly enough (“… batch normalization is performed using fixed-point arithmetic”)
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 9
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining
limitations of the claim are directed to insignificant extra-solution activity (“wherein the input video frame consists of a luminance plane”).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 10
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining
limitations of the claim are directed to insignificant extra-solution activity that is only tangentially related to the inventive concept (“wherein the quantized CNN is trained using a loss function that is based on a correlation between determined probabilities and human subjective quality ratings of a batch of videos of size n”).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 11
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining
limitations of the claim are directed to insignificant extra-solution activity that is only tangentially related to the inventive concept (“wherein the loss function is based on a negative squared Pearson linear correlation coefficient”).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 12
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The remaining
limitations of the claim are further components of the abstract idea (“…convolutional layer is configured to apply a plurality of 1×N kernels to analyze horizontal dimensions of an input”, “the second convolutional layer is configured to apply a plurality of N×1 kernels to analyze vertical dimensions of an input”)
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claim 13
Step 1: Recites a method. Therefore, it is directed to a statutory category of invention.
Step 2A Prong 1: The claim recites the abstract idea it inherits from the claim it depends on. It also recites:
determining the probability that the input video frame is of low quality by comparing an output of a layer of the quantized CNN to a predefined threshold and not calculating a sigmoid function for the determination
mere comparison of values is a mental process that can be done in one’s head
Step 2A Prong 2: The judicial exception is not integrated into a practical application. There are no remaining limitations in the claim.
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors
the analysis of step 2A prong 2.
Claims 14-20 are rejected under 35 U.S.C. 101 because the claimed invention is not directed to a statutory category of invention. All cite software per se with no structural elements and thus fail Step 1 of the eligibility analysis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 10, 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Partovi et al. (“US 20210089925”), in view of Tu et al. (“Regression or classification? New methods to evaluate no-reference picture and video quality models”, 2021).
Regarding claim 1, Partovi teaches:
performing a sequence of convolutional operations (Figure 2) of a quantized convolutional neural network (quantized CNN) (Paragraph 11, “Recently, different NN1 compression techniques have been proposed to reduce NN model size and computational cost. One such NN compression technique is neural network quantization.”) … using quantized weights to generate feature maps (Abstract, “quantizing a set of real-valued weights of a filter to generate a corresponding set of quantized weights; generating an output feature tensor2”);
applying respective batch normalizations to the feature maps (Paragraph 53-543), to obtain normalized feature maps, wherein applying a batch normalization to a feature map of the feature maps comprises applying a linear function to the feature map, wherein the linear function includes multiplying each feature of the feature map by a learned scaling factor (Paragraph 9-104);
after applying the respective batch normalizations, processing the normalized feature maps through additional layers of the quantized CNN (Figure 85)
Partovi fails to teach the input as a video frame, and determining a probability that the input video frame is of low quality. However, Tu teaches the input of a video frame (Page 2086, Column 1, Paragraph 3, “Consider a set of training samples… x is either in the picture or video space”, Page 2087, Column 2, Paragraph 2, “When evaluated on videos, the BIQA models were computed at one frame per second”), used to determine a probability that the input video frame is of low quality (Page 2086, Column 1, Figure 1).
Tu and Partovi are considered analogous to the invention because all are directed to optimized statistical models that can take images as input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Tu, and apply a quantized convolutional network for video image quality assessment. Doing so allows more efficient computation.
Regarding claim 2, Partovi fails to teach the further limitations of the claim. However, Tu teaches selecting encoding parameters for a video segment that includes the input video frame based on the probability that the input video frame is of low quality (Page 2088, Column 1, Paragraph 1, “involves encoding videos uploaded to YouTube with parameters optimized based on its input quality category: {low, medium, high}6”).
Tu and Partovi are considered analogous to the invention because all are directed to optimized statistical models that can take images as input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Tu, and select encoding parameters based on the probability of quality found. Calculating probability of classification over regression streamlines the process for transcoding on platforms that use user-generated content (see Page 2088, Column 1, Paragraph 1 of Tu).
Regarding claim 3, Partovi fails to teach the further limitations of the claim. However, Tu teaches selecting the encoding parameters for the video segment based on an average of respective probabilities of input video frames that include the input video frame (Page 2088, Column 2, Paragraph 2, “When evaluated on videos, the features average pooled across sampled frames to obtain video-level features to be used for training.”7)
Tu and Partovi are considered analogous to the invention because all are directed to optimized statistical models that can take images as input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Tu, and select encoding parameters based on the average probability of quality found over an entire video. Doing so results in a more accurate prediction and handles outlier events better.
Regarding claim 10, Partovi teaches the use of a quantized CNN (see claim 1 analysis) but fails to teach a loss function that is based on a correlation between determined probabilities and human subjective quality ratings of a batch of videos of size n. However, Tu teaches… a loss function that is based on a correlation between determined probabilities and human subjective quality ratings of a batch of videos of size n (Page 2085, Column 2, Paragraph 1, “The success of these models is evaluated by comparing their quality predictions to subjective mean opinion scores (MOSs)”, Page 2086, Column 2, Paragraph 1, “The standard performance metrics for UGC-QA regression are the Spearman rank-order correlation coefficient (SRCC) calculated between the ground truth MOSs and the predicted scores”).
Tu and Partovi are considered analogous to the invention because all are directed to optimized statistical models that can take images as input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Tu, and applied a loss function that compares determined probabilities with human subjective ratings. Doing is much more computationally inexpensive task than traditional loss functions used in CNN models.
Regarding claim 13, Partovi teaches processing the normalized feature maps through the additional layers of the quantized CNN (see claim 1 analysis). Partovi fails to teach determining the probability that the input video frame is of low quality by comparing an output to a predefined threshold and not calculating a sigmoid function for the determination. However, Tu teaches determining the probability that the input video frame is of low quality by comparing an output to a predefined threshold and not calculating a sigmoid function for the determination8 (Page 2086, “Column 2, Paragraph 4, “…i.e., to predict whether an input UGC belongs to the High or Low quality category. The binarizing threshold T is automatically determined by the GMM clustering described above”).
Tu and Partovi are considered analogous to the invention because all are directed to optimized statistical models that can take images as input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Tu, and determined probability by using a threshold. Doing so is much less computationally expensive than using a sigmoid function.
Claim 14 is a device claim corresponding to method claim 1 and is rejected for the same reasons as given in the rejection of that claim.
Claim 15 is a device claim corresponding to method claim 2 and is rejected for the same reasons as given in the rejection of that claim.
Claims 4-7, 16-17, are rejected under 35 U.S.C. 103 as being unpatentable over Partovi et al. (“US 20210089925”), in view of Tu et al. (“Regression or classification? New methods to evaluate no-reference picture and video quality models”, 2021) and in further view of Hanumante et al. (“US 20230214639”).
Regarding claim 4, Partovi fails to teach the further limitations of the claim. However, Hanumante teaches the quantized weights are obtained by simulating quantization during a training phase that uses floating-point weights (Paragraph 118, “simulated quantization… may be used… to facilitate training for the quantized models… some operations, such as batch normalization (BN), are conducted with full-precision9 to stabilize training”).
Tu and Partovi are considered analogous to the invention because all are directed to optimized statistical models that can take images as input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Tu to incorporate the teachings of Partovi, and applied a quantized convolutional network for video image quality assessment that obtains quantized weights by simulating quantization. Doing so can provide greater stability during the training process (see Paragraph 118 of Hanumante).
Regarding claim 5, Partovi fails to teach the further limitations of the claim. However, Hanumante teaches the batch normalization is fused with a respective convolutional operation (Paragraph 149, “the method quantizes on the effective weight that fuses the weight of convolution layers with the weight and running variance from BN10”).
Hanumante and Partovi are considered analogous to the invention because all are directed to quantized CNNs. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Hanumante, and applied a fusion of the batch normalization with a convolutional operation. Doing so can provide non-negligible improvements to the speed of the model (see Paragraph 24 of Hanumante).
Regarding claim 6, Partovi fails to teach the further limitations of the claim. However, Hanumante teaches the batch normalization is performed using fixed-point arithmetic (Paragraph 149, “To quantize the whole model with only 8-bit fixed-point multiplication involved, the scaling factor from the batch normalization (BN) layer is tackled”).
Hanumante and Partovi are considered analogous to the invention because all are directed to quantized CNNs. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Hanumante, and applied fixed-point arithmetic when doing batch normalization. Doing so can provide non-negligible improvements to the speed of the model (see Paragraph 24 of Hanumante).
Regarding claim 7, Partovi fails to teach the further limitations of the claim. However, Hanumante teaches a respective quantized activation function after each convolutional operation of at least some of the convolutional operations (Abstract, “A fixed-point activation between adjacent computational layers is related using PACT quantization”).
Partovi and Hanumante are considered analogous to the invention because all are directed to optimized statistical models that can take images as input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Hanumante, and used quantized activation functions. Doing so can provide non-negligible improvements to the speed of the model (see Paragraph 24 of Hanumante).
Claim 16 is a device claim corresponding to method claim 5 and is rejected for the same reasons as given in the rejection of that claim.
Claim 17 is a device claim corresponding to method claim 6 and is rejected for the same reasons as given in the rejection of that claim.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Partovi et al. (“US 20210089925”), in view of Tu et al. (“Regression or classification? New methods to evaluate no-reference picture and video quality models”, 2021) and in further view of Vaze et al. (“Segmentation of Fetal Adipose Tissue Using Efficient CNNs for Portable Ultrasound”, 2018).
Regarding claim 8, Partovi teaches the use of a quantized CNN (see claim 1 analysis). Partovi fails to teach the additional layers of the including at least one depth-wise separable convolutional layer. However, Vaze teaches at least one of the additional layers including a depth-wise separable convolutional layer (Page 59, Figure 3).
Partovi and Vaze are considered analogous to the invention because all are directed to Convolutional Neural Networks. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Vaze, and use a CNN with incorporated depth-wise convolution. Doing so can better streamline computation as the number of inputs and parameters rise (see page 58, paragraph 2 of Vaze).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Partovi et al. (“US 20210089925”), in view of Tu et al. (“Regression or classification? New methods to evaluate no-reference picture and video quality models”, 2021), and in further view of Huynh-Thu et al. (“The accuracy of PSNR in predicting video quality for different video scenes and frame rates”, 2010).
Regarding claim 9, Partovi fails to teach the further limitations of the claim. However, Huynh-Thu teaches wherein the input video frame consists of a luminance plane (Page 38, Column 1, Paragraph 1, “…after applying a Sobel filter on the luminance plane of each video frame”).
Partovi and Huynh-Thu are considered analogous to the invention because all are directed to optimized statistical models that can take images as input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Huynh-Thu and incorporate the luminance plane as an input to the model. Doing so provides a quicker and simpler way to compute image quality metrics (“The popularity of PSNR as a quality metric comes mainly from the fact that it is fast to compute and mathematically”, Page 35-36 of Huynh-Thu).
Claims 11, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Partovi et al. (“US 20210089925”), in view of Tu et al. (“Regression or classification? New methods to evaluate no-reference picture and video quality models”, 2021) and in further view of Abbasi-Sureshjani et al. (“Risk of Training Diagnostic Algorithms on Data with Demographic Bias”, 2020).
Regarding claim 11, Partovi fails to teach the further limitations of the claim. However, Abbasi-Sureshjani teaches wherein the loss function is based on a negative squared Pearson linear correlation coefficient (Page 6, Paragraph 2, “While for optimizing the bias predictor head, a bias prediction loss… is defined as the negative-squared Pearson correlation coefficient”).
Partovi and Abbasi-Sureshjani are considered analogous to the invention because all are directed to the usage of statistical models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Abbasi-Sureshjani and use a negative squared Pearson linear correlation coefficient as the loss function. Doing so opens analysis to regression, allowing for more granular metrics.
Claim 18 is a device claim corresponding to method claim 11 and is rejected for the same reasons as given in the rejection of that claim.
Claims 12, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Partovi et al. (“US 20210089925”), in view of Tu et al. (“Regression or classification? New methods to evaluate no-reference picture and video quality models”, 2021), and in further view of Schmitt et al. (“US 12205340”, 202211) and Zhou et al. (“CN 113793341”).
Regarding claim 12, Partovi fails to teach the further limitations of the claim. However, Schmitt teaches wherein the convolutional operations comprise a first convolutional layer and a second convolutional layer (Abstract, “The neural network comprises a first convolutional layer… and a second convolutional layer”), wherein the first convolutional layer is configured to apply a plurality of… kernels to analyze… an input to the first convolutional layer, and wherein the second convolutional layer is configured to apply a plurality of… kernels to analyze… an input to the second convolutional layer (Figure 2, Figure 412).
Zhou teaches the use of a kernel of size 1 x N to analyze a horizontal dimension of an input for a first convolutional layer, and the usage of a kernel of size N x 1 to analyze a vertical dimension of an input for a second convolutional layer (Page 2, Paragraph 1, “the first layer convolution kernel size is 1 * 7…, the second layer is 7 * 1…, extracting horizontal and vertical spatial structure respectively”)
Partovi, Schmitt and Zhou are considered analogous to the invention because all are directed to convolutional neural networks. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Partovi to incorporate the teachings of Schmitt and Zhou, and incorporate a two-layer convolutional layer structure to analyze the horizontal and vertical dimensions of an input image. Doing so can provide a much less computationally expensive model that retains high accuracy (see Column 1, Lines 41-45 of Schmitt).
Claim 19 is a device claim corresponding to method claim 12 and is rejected for the same reasons as given in the rejection of that claim.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Aono et al. (“US 12552188”, 202213), in view of Teichner et al. (“US 20210004641”), and in further view of Kshirsagar et al. (“US 20210342455”), Tasli et al. (“US 20190354772”), Partovi et al. (“US 20210089925”) Hanumante et al. (“US 20230214639”), and Bade et al. (“FPGA-Based Stochastic Neural Networks -Implementation”, 1994).
Regarding claim 20, Aono teaches a… convolutional neural network (Column 4, Lines 56-59) …, comprising:
layers consisting of:
a third convolutional layer subsequent to and coupled to the second convolutional layer (Figure 8);
a fourth max pooling layer subsequent to and coupled to the third convolutional layer (Figure 8);
a fifth convolutional layer subsequent to and coupled to the fourth max pooling layer (Figure 8);
Aono fails to teach:
a first convolutional layer;
a second convolutional layer subsequent to and coupled to the first convolutional layer;
a sixth global average pooling layer subsequent to and coupled to the fifth convolutional layer; and a seventh dense layer subsequent to and coupled to the sixth global average pooling layer, wherein the seventh dense layer simulates a sigmoid function by comparing an output of a fully connected layer to a constant learned during a training phase,
wherein at least one of the first convolutional layer, the second convolutional layer, the third convolutional layer, or the fifth convolutional layer is configured to apply a batch normalization to an input feature map by applying a linear function to the input feature map,
wherein the linear function includes multiplying each feature of the input feature map by a learned scaling factor,
wherein none of the layers are configured to perform floating point operations, and wherein weights of the quantized CNN are fixed point weights learned during a training process that uses simulated and heterogeneous quantization.
However,
Teichner teaches a sixth global average pooling layer subsequent to and coupled to the fifth convolutional layer (Figure 2B);
Kshirsagar teaches a seventh dense layer subsequent to and coupled to the sixth global average pooling layer (Paragraph 58, “The sixth layer is a flattening layer that transforms the three-dimensional array of data from the sixth layer into a vector. In this case, the vector output may be 640×640 when the input is a three-dimensional array. The seventh layer may be a dense layer.”).
Bade teaches the simulation of a sigmoid function (Page 191, Column 2, Paragraph 2, “For this architecture, the activation function, is a continuous sigmoid-like14 function that is achieved by simply outputting a “1” if the sum of the synapses is above a threshold value”),
Tasli teaches a last layer that compares the outputs of a fully-connected layer to a learned constant (Paragraph 49, “…the commonly used softmax layer in the final output is not used. Instead, the raw outputs of the leaky rectified linear units of the last fully-connected perceptron layer are used with a threshold learned”),
Partovi teaches:
a quantized convolutional neural network (see claim 1 analysis);
wherein at least one of the first convolutional layer, the second convolutional layer, the third convolutional layer, or the fifth convolutional layer is configured to apply a batch normalization to an input feature map (Figure 115) by applying a linear function to the input feature map (see claim 1 analysis);
wherein the linear function includes multiplying each feature of the input feature map by a learned scaling factor (see claim 1 analysis);
wherein none of the layers are configured to perform floating point operations (Paragraph 7, “NN layers that use bitwise operations can be configured as low-bit layers in which operations are performed using elements that are represented as 1 or 2 bit values.”), and
Hanumante teaches wherein weights of the quantized CNN are fixed point weights learned during a training process that uses simulated (see claim 4 analysis) and heterogeneous quantization (Paragraph 122, “Another direction studies mixed-precision that determines bit-width for each layer of the neural network through searching algorithms, aiming at better accuracy-efficiency trade-off.”)
Aono, Teichner, Kshirsagar, Talsi, Bade, Partovi and Hanumante considered analogous to the invention because all are directed to convolutional neural networks. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Aono and incorporated a final dense layer in the CNN structure that is additionally also quantized heterogeneously. Doing so creates a more structured output and provides better accuracy-efficiency trade-off (see paragraph 122 of Hanumante).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEWOS MESFIN whose telephone number is (571)270-0782. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEWOS MESFIN/ Examiner, Art Unit 2145
/CESAR B PAULA/ Supervisory Patent Examiner, Art Unit 2145
1 This technique is applied to NNs in general, but specific citations to its application on CNNs are littered through the reference (e.g. Paragraph 75 “low-bit quantized CNNs”)
2 Feature tensor includes multiple feature maps
3 Equation between the two paragraphs showcase the input of feature maps to batch normalization
4 Equation between the two paragraphs showcase the use of a trainable scale that is multiplied to the input (a feature map, as shown in above reference). The use of the mini-batch mean and variance imply a normalization of said feature map
5 Implies multiple layers of a CNN, wherein activation occurs after batch normalization
6 The probability limitation taught in analysis of claim 1
7 The probability and encoding parameters limitations are taught in analysis of claim 1 and claim 2 respectively
8 No mention of use of sigmoid in Tu
9 Full-precision (floating points) as opposed to quantized or fixed-point
10 BN = Batch normalization
11 US Patent published in 2025 with filing date in 2022
12 Figure 2, c2 showcases the use of a plurality of kernels, where i is an input. Figure 4 shows that this applies to both the first and second convolutional layers.
13 US Patent published in 2026 with filing date in 2022
14 “sigmoid-like” implies simulation
15 Shows it applied to first convolutional layer
Prosecution Insights