DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Response to Amendment
The amendment filed on 29 December 2025 has been entered.
The amendment of claim 1 and cancellation of claim 5 have been acknowledged.
Terminal Disclaimer
The terminal disclaimer filed on 29 December 2025 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of U.S. patent application S/N: 18/270,649 has been reviewed and is accepted. The terminal disclaimer has been recorded and the Double Patenting rejections have been withdrawn.
Response to Arguments
Applicant's arguments filed on 29 December 2025, with respect to the pending claims, have been fully considered but they are not persuasive.
Applicant’s Representative submits that the prior art (Lin, in view of Gadelrab) does not teach the limitations of original claim 5 (now incorporated into claim 1) because the term “probability” refers to the probability distribution of input signal values within the neural network layers in Lin whereas the amended claim 1 recites “calculating probabilities that the input image data will correspond to a plurality of classes related to object recognition of the deep learning network.” Applicant’s Representative submits that this is different from Lin because the probability of amended claim 1 refers to the semantic inference confidence (e.g., Softmax output scores).
The examiner respectfully disagrees that the prior art does not teach amended claim 1. Lin teaches calculating output scores of the deep convolutional network that determines how closely the output matches the target (Lin ¶¶0044) and using probabilistic deep belief networks to learn a probability distribution in the absence of information about the class to which each input should be categorized (Lin ¶¶0048).
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., semantic inference confidence, softmax output scores) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Even if the claims were to explicitly recite semantic inference confidence and softmax scores, the prior art teaches these limitations (see Gadelrab ¶¶0041: “Feature maps in a DCN may be convolved to generate one or more feature vectors. Each feature of the feature vector may correspond to a possible feature of an image, and a softmax function generate a probability for each feature. As such, the output of the DCN may thus be a probability that the input image includes one or more features”).
In view of this reasonable interpretation of the claims and the prior art, the examiner respectfully submits that the rejections set forth below remain proper.
Claim Rejections - 35 USC § 103
Claim(s) 1-4 and 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 2016/0328646 A1), in view of Gadelrab et al. (US 2021/0279635 A1), hereinafter referred to as Lin and Gadelrab, respectively.
Regarding claim 1, Lin teaches an image recognition method comprising:
generating a plurality of quantization models corresponding to a plurality of different bit numbers by performing quantization corresponding to the plurality of bit numbers on a deep learning network which perform object recognition on any image (Lin Abstract: “A method of quantizing a floating point machine learning network to obtain a fixed point machine learning network using a quantizer may include selecting at least one moment of an input distribution of the floating point machine learning network”; Lin ¶¶0035: “detect and recognize gestures”; Lin ¶¶0043: “For instance, a network 300 designed to recognize visual features from a car-mounted camera may develop high layer neurons with different properties depending on their association with the lower versus the upper portion of the image. Neurons associated with the lower portion of the image may learn to recognize lane markings, for example, while neurons associated with the upper portion of the image may learn to recognize traffic lights, traffic signs, and the like”; Lin ¶¶0059-¶¶0060: “a sign bit, an 8-bit exponent, and a 23-bit fraction component … where m is a number of bits for an integer part and n is a number of bits for a fraction part … may use an m+n+1 bit signed integer container with n fractional bits … may use sixteen bits”; Lin ¶¶0078: “where n is the number of fractional bits that may be specified to represent the quantizer input and 2−n may be specified as the step size”);
receiving image data as an input to the deep learning network (Lin Fig. 3A);
determining uncertainty of the received image data (Lin Fig. 4 & ¶¶0048: “Deep belief networks (DBNs) are probabilistic models comprising multiple layers of hidden nodes. DBNs may be used to extract a hierarchical representation of training data sets … a type of artificial neural network that can learn a probability distribution over a set of inputs. Because RBMs can learn a probability distribution in the absence of information about the class to which each input should be categorized, RBMs are often used in unsupervised learning”);
selecting any one of the plurality of quantization models (Lin ¶¶0063: “Quantization efficiency in artificial neural networks, according to aspects of the present disclosure, may be better understood by a review of quantization according to the probability distribution function 400 shown in FIG. 4. For example, an input to the quantizer may be uniformly distributed over [Xmin, Xmax], where Xmin and Xmax define the range of a fixed point representation … a signal to quantization noise ratio (SQNR), assuming M is the number of integer bits”; Lin ¶¶0064: “Application of quantization to the weights, biases, and activation values in artificial neural networks includes the determination of a step size. For example, the step sizes of a symmetric uniform quantizer for Gaussian, Laplacian, and Gamma distributions may be calculated with a deterministic function of the standard deviation of the input distribution, if it is assumed that the distributions have zero mean and unit variance”);
performing object recognition on the image data through the selected quantization model and outputting a label corresponding to the image data as an object recognition result (Lin Abstract: “The method may also include determining quantizer parameters for quantizing values of the floating point machine learning network based at least in part on the at least one selected moment of the input distribution of the floating point machine learning network to obtain corresponding values of the fixed point machine learning network”; Lin ¶¶0035 & ¶¶0043 discussed above);
wherein the determining of the uncertainty comprises:
calculating probabilities that the input image data will correspond to a plurality of classes related to object recognition of the deep learning network (Lin Fig. 4 & ¶¶0048, ¶¶0063 discussed above; also see Lin ¶¶0044: “The output 322 may be a vector of values corresponding to features such as “sign,” “60,” and “100.” The network designer may want the DCN to output a high score for some of the neurons in the output feature vector, for example the ones corresponding to “sign” and “60” as shown in the output 322 for a network 300 that has been trained. Before training, the output produced by the DCN is likely to be incorrect, and so an error may be calculated between the actual output and the target output. The weights of the DCN may then be adjusted so that the output scores of the DCN are more closely aligned with the target”); and
calculating an uncertainty score based on the probabilities calculated according to the plurality of classes (Lin Fig. 4 & ¶¶0044, ¶¶0048, ¶¶0063 discussed above; Lin eqs. (2)-(3), (13)-(16)).
However, Lin does not appear to explicitly teach that the quantization model is selected based on the determined uncertainty.
Pertaining to the same field of endeavor, Gadelrab teaches selecting the quantization model based on the determined uncertainty (Gadelrab ¶¶0048: “To improve the efficiency of inference performance using these computational hardware blocks, model parameters, such as model weights, may be quantized and reduced in size from a number n bits to a smaller number of bits and/or from floating point to integer representations”; Gadelrab ¶¶0123: “after an inference is performed using the high efficiency model, a difference between the current input (e.g., the data specified in an inference request received at block 502) and the input used in the most recent execution of an inference in the high accuracy mode is compared. This scenario may exist, for example, when an input data set changes … in which there is a sufficient difference between different inputs such that previously quantized weights and activation parameters are no longer valid. If the difference between the current input and the input used in the most recent execution of an inference in the high accuracy mode exceeds a threshold value, the system can determine that the quantized weights and parameters may no longer be applicable to the data on which inferences are to be performed. Thus, if the difference between the current input and the input used in the most recent execution of an inference in the high accuracy mode exceeds a threshold value, the system can proceed to block 616, where the system exits the high efficiency mode and executes subsequent inferences in the high accuracy mode”).
In addition, Gadelrab also teaches calculating probabilities that the input image data will correspond to a plurality of classes related to object recognition of the deep learning network (Gadelrab ¶¶0041: “Feature maps in a DCN may be convolved to generate one or more feature vectors. Each feature of the feature vector may correspond to a possible feature of an image, and a softmax function generate a probability for each feature. As such, the output of the DCN may thus be a probability that the input image includes one or more features. Before training, the output produced by the DCN is likely to be incorrect. Thus, an error may be calculated between the output produced by the DCN and a target output. The target output is the ground truth of the image. The weights of the DCN may then be adjusted so the output of the DCN is more closely aligned with the target output”).
Lin and Gadelrab are considered to be analogous art because they are directed to image processing using machine learning. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of quantizing a floating point machine learning network (as taught by Lin) to select the quantization model based on the determined uncertainty (as taught by Gadelrab) because the combination improves the efficiency of inference performance by using adaptive quantization (Gadelrab ¶¶0048).
Regarding claim 2, Lin, in view of Gadelrab, teaches the image recognition method of claim 1, wherein the generating of the plurality of quantization model comprises:
generating a first quantization model corresponding to 8 bits (Lin ¶¶0059-¶¶0060 discussed above);
generating a second quantization model corresponding to 4 bits (Lin Table 1: bit number ranges from 1-8); and
generating a third quantization model corresponding to 2 bits (Lin Table 1).
Regarding claim 3, Lin, in view of Gadelrab, teaches the image recognition method of claim 2, wherein the selecting of any one of the plurality of quantization models comprises, when the determined uncertainty is a preset first reference value or more, selecting the first quantization model (Gadelrab ¶¶0123 discussed above teaches using different models based on the accuracy value satisfying a threshold value).
Regarding claim 4, Lin, in view of Gadelrab, teaches the image recognition method of claim 3, wherein the selecting any one of the plurality of quantization models comprises, when the determined uncertain is a preset second reference value or less, selecting the third quantization model (Gadelrab ¶¶0123 discussed above teaches using different models based on the accuracy value satisfying a threshold value).
Regarding claim 6, Lin, in view of Gadelrab, teaches the image recognition method of claim 1, wherein the determining of the uncertainty is performed by an uncertainty determination network which is separate from the deep learning network, and the uncertainty determination network includes a smaller number of layers than the deep learning network (Lin ¶¶0048: “Deep belief networks (DBNs) are probabilistic models comprising multiple layers of hidden nodes. DBNs may be used to extract a hierarchical representation of training data sets”; Lin ¶¶0049: “Deep convolutional networks (DCNs) are networks of convolutional networks, configured with additional pooling and normalization layers”; Lin ¶¶0053: “The deep convolutional network 350 may include multiple different types of layers based on connectivity and weight sharing”; Lin ¶¶0056: “a machine learning model, such as a neural model, is configured for quantizing a floating point neural network to obtain a fixed point neural network. The model includes a reducing means and/or balancing means”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOO J SHIN whose telephone number is (571)272-9753. The examiner can normally be reached M-F; 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Soo Shin/Primary Examiner, Art Unit 2667 571-272-9753
soo.shin@uspto.gov