DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on05/04/2023 and 04/23/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 16-34 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 16 recites the limitation "the method for processing unit…" in line 1, instead it should recite “A method for processing unit…”. There is insufficient antecedent basis for this limitation in the claim.
Dependent claims 17-34 are rejected for being dependency of claim 16.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 16-35 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea and does not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Regarding claim 16
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“…the method comprising: estimating quality drop corresponding to the Al model according to predetermined bandwidth for each layer among the multiple layers; determining a layer for quantization among the multiple layers; quantizing the determined layer; determining a processing unit of the NPU based on the quantization; and obtaining output data corresponding to the input data based on the determined …”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “The method for processing input data using Artificial Intelligence (AI) model including multiple layers in Neural Processing Unit (NPU)… processing unit.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 34
Claim 34 recites analogous limitations to independent claim 16 and therefore is rejected on the same ground as independent claim 16.
Regarding claim 17
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
“wherein the quality drop includes Peak Signal-to-Noise Ratio (PSNR) drop.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “computer-implemented,”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 18
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
“wherein the determining the layer for quantization comprises: determining the layer through a Dynamic Range Estimation (DRE) selection module.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Thus, the judicial exception is not integrated into a practical application [see MPEP 2106.05(d) I.], failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 19
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the quantizing the determined layer comprising: quantizing the determined layer through a Runtime Quantization Unit (RQU).”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 20
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “the AI model includes optimized super-resolution Deep Neural Network (DNN), of a Machine Learning (ML) model, on a processing unit to perform super-resolution.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 21
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“ obtaining at least one low resolution image as the input data; dividing the low resolution image into fixed-size patches to be up scaled; … concatenating the up scaled patches to form a super-resolution image; and outputting the super-resolution image as the output data”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein each layer of the optimized ML model has a quantized activations tensor that is either pre- defined or determined using Dynamic Range Estimation (DRE) at runtime;”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 22
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“…scheduling execution of partitions of the DNN that have layers with pre-defined quantized activations tensors without supervision; and scheduling execution of partitions of the DNN that have layers with quantized activations tensors determined at runtime, wherein the scheduling is monitored to quantize the activations tensors at runtime.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the processing each fixed- size patch using the optimized ML model comprising: partitioning the DNN into groups of consecutive layers based on an associated word length of each layer and whether the quantized activations tensors are pre-defined or determined at runtime;”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 23
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“wherein the quantizing the activations tensors at runtime comprising: extracting minimum and maximum values from an input tensor of each layer; and using the extracted minimum and maximum values to compute a quantization for each layer.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Thus, the judicial exception is not integrated into a practical application [see MPEP 2106.05(d) I.], failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 24
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“… determining, for each layer, whether to keep the uniform word length for the values of the activations tensor of the layer or to switch to a new word length that is supported by the processing unit…”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “further comprising: quantizing, using scale factors, a word length for all values of an activations tensor of each layer of the DNN to a uniform word length;…and quantizing a word length for all values of the activations tensor of each layer based on the determining, and generating a hybrid-precision DNN optimized for implementation on the processing unit”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 25
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein quantizing a word length for all values of an activations tensor of each layer comprising: deriving, for each layer, a scale factor based on an estimated dynamic range of the activations tensor for the layer.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 26
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“ obtaining a user-defined minimum quality threshold value for the super-resolution, and using the minimum quality threshold value to determine whether to keep the uniform word length or to switch to a new word length for the values of the activations tensor of each layer.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Thus, the judicial exception is not integrated into a practical application [see MPEP 2106.05(d) I.], failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 27
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“ determining a computational cost in terms of a number of bit operations, BOPs, associated with each layer; …”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein determining whether to keep the uniform word length comprises prioritizing quantization of layers of the DNN that have a high computational cost”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 28
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“wherein the determining whether to keep the uniform word length comprising: keeping the uniform word length or switching to a new word length by identifying, for each layer, which word length supported by the processing unit minimizes the computational cost of an operation performed by the layer on the …while maintaining the minimum quality threshold value”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “processing unit”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 29
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“wherein the identifying comprising: ordering each quantized layer based on the number of bit operations, BOPs, associated with the layer; temporarily adjusting the word length of the activations tensor of a 1-th layer to a lower- precision word length; determining whether a minimum quality threshold value is satisfied; and setting the word length of the 1-th layer to the lower-precision word length when the minimum quality threshold value is determined to be satisfied.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Thus, the judicial exception is not integrated into a practical application [see MPEP 2106.05(d) I.], failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 30
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“further comprising: repeating the adjusting, determining and ordering steps for each layer of the DNN”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Thus, the judicial exception is not integrated into a practical application [see MPEP 2106.05(d) I.], failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 31
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“further comprising: identifying one or more quantized layers of the DNN to be further quantized at runtime based on a dynamically derived scale factor applied to the activations tensor of the identified quantized layers.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Thus, the judicial exception is not integrated into a practical application [see MPEP 2106.05(d) I.], failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 32
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“wherein the identifying one or more quantized layers of the DNN to be further quantized at runtime comprising: determining a resilience of each quantized layer of the … to low precision.”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “DNN”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 33
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“wherein the determining a resilience of each quantized layer comprising: calculating a degradation in a peak signal-to-noise ratio value caused by each quantized layer; ordering each quantized layer in a list sorted by a decreasing order of degradation; calculating an energy concentration of a subset of quantized layers up to a 1-th layer in the list; selecting one or more quantized layers up to the 1-th layer that satisfy an energy concentration threshold;”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “and specifying that the selected quantized layers will be further quantized by having their scale factors dynamically derived at runtime”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 34
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“further comprising: repeating the calculating, selecting and specifying steps for each quantized layer in the list”
This limitation is directed to the abstract idea of a mental process (concepts performed in the human mind, including observation and evaluation [see MPEP 2106.04(a)(2) III. C.]).
Thus, the judicial exception is not integrated into a practical application [see MPEP 2106.05(d) I.], failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 16-17 and 35 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mustafa et al. (“Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices”).
Regarding claim 16
Mustafa teaches the method for processing input data using Artificial Intelligence (AI) model including multiple layers in Neural Processing Unit (NPU), (abstract “In this paper, we propose a hard ware (Synaptics Dolphin NPU) limitation aware, extremely lightweight quantization robust real-time super resolution network (XLSR). The proposed model’s building block is inspired from root modules introduced in [15] for Image classification. We successfully applied root modules to SISR problem, further more to make the model uint8 quantization robust we used Clipped ReLU at the last layer of the network and achieved great balance between reconstruction quality and runtime.”)
the method comprising: estimating quality drop corresponding to the Al model according to predetermined bandwidth for each layer among the multiple layers; (pg. 4 right col “To make a model quantization friendly, we should first focus on the reasons behind why quantization adversely af fects the accuracy. Linear output activation function is very common among super resolution models and it helps with the model optimization. Although, not strictly enforced when the model converges we are pretty sure that the out put will be bounded in between 0-255 (or 0-1.0). However, if such a model is quantized, the quantized output tend to be dull and accuracy can drop about 5-7dB compared to float16/32 model. The reason we believe is the following; during the earlier steps of training, the output is not guaran teed to be in between 0-1.0 and intermediate activations can also be unbounded or might contain outliers.”)
determining a layer for quantization among the multiple layers; (pg. 5 left col “Furthermore, although placing a single Clipped ReLU was enough to regularize the intermediate activations of our proposed network, when a model gets deeper, regularization effect might vanish for deeper layers, to overcome this issue we suggest to change a few of the ReLU’s with Clipped ReLU’s.”)
quantizing the determined layer; (pg. 2 left col “Building an extremely lightweight network with low number of parameters was not enough by itself since the model for the challenge needs to be fully uint8 quantized. To make the model quantization robust and keep it still run ning fast on the deployment platform, instead of “Linear””)
determining a processing unit of the NPU based on the quantization; and obtaining output data corresponding to the input data based on the determined processing unit. (Pg. 4 right col “To make a model quantization friendly, we should first focus on the reasons behind why quantization adversely affects the accuracy. Linear output activation function is very common among super resolution models and it helps with the model optimization. Although, not strictly enforced when the model converges we are pretty sure that the output will be bounded in between 0-255 (or 0-1.0).”)
Regarding claim 17
Mustafa teaches claim 16.
Mustafa further teaches wherein the quality drop includes Peak Signal-to-Noise Ratio (PSNR) drop. (Pg. 4 right col “These allowed intermediate unbounded activations create outliers (very large a few numbers). The outliers in intermediate layer activations, when uint8 quantized leads to some important information carrying, comparably low amplitude values to be zeroed out. Hence effective signal energy reaching to the last layer drops, which results in dull colors and drastic PSNR drops.”)
Regarding claim 19
Mustafa teaches claim 16.
Mustafa further teaches wherein the quantizing the determined layer comprising: quantizing the determined layer through a Runtime Quantization Unit (RQU). (Section 5 “conclusion, we proposed a real-time single image super resolution method driven by the hardware constraints of the Mobile AI 2021 challenge, although it is driven by the target hardware, the resulting model is very efficient in terms of runtime and model parameters.” Also see the layers at pg. 4 left col “Furthermore, although placing a single Clipped ReLU was enough to regularize the intermediate activations of our proposed network, when amodel gets deeper, regularization effect might vanish for deeper layers, to overcome this issue we suggest to change a few of the ReLU’s with Clipped ReLU’s.”)
Regarding claim 20
Mustafa teaches claim 16.
Mustafa further teaches the AI model includes optimized super-resolution Deep Neural Network (DNN), of a Machine Learning (ML) model, on a processing unit to perform super-resolution. (Abstract “In this paper, we propose a hard ware (Synaptics Dolphin NPU) limitation aware, extremely lightweight quantization robust real-time super resolution network (XLSR). The proposed model’s building block is inspired from root modules introduced in [15] for Image classification. We successfully applied root modules to SISR problem, further more to make the model uint8 quantization robust we used Clipped ReLU at the last layer of the network and achieved great balance between reconstruction quality and runtime.”)
Regarding claim 21
Mustafa teaches claim 20.
Mustafa further teaches the method further comprising: obtaining at least one low resolution image as the input data; dividing the low resolution image into fixed-size patches to be up scaled; (abstract “In this paper, we propose a hard ware (Synaptics Dolphin NPU) limitation aware, extremely lightweight quantization robust real-time super resolution network (XLSR). The proposed model’s building block is inspired from root modules introduced in [15] for Image classification. We successfully applied root modules to SISR problem, further more to make the model uint8 quantization robust we used Clipped ReLU at the last layer of the network and achieved great balance between reconstruction quality and runtime.”)
upscaling a resolution of each fixed-size patch using the optimized ML model, wherein each layer of the optimized ML model has a quantized activations tensor that is either pre-defined (Table 1: Example set of number of parameters from high performing deep learning networks. (*) For a fair comparison of parameters model is assumed to accept RGB input and scaling is x3)
or determined using Dynamic Range Estimation (DRE) at runtime;
concatenating the up scaled patches to form a super-resolution image; and outputting the super-resolution image as the output data. (Pg. 4 left col “On the other hand, when channel shuffling operator is relaxed to 1x1 convolution (to still allow interchannel com munication) or skip connection in ResNext block is re moved, we arrive at the building block used in our network (See Figure 3)…. Thus, we avoided all addition and scaling operations and used concatenation operation when necessary and input data scaling and normalization were not used.”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mustafa et al. (“Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices”) in view of Chalmers et al. (“Reconstructing Reflection Maps Using a Stacked-CNN for Mixed Reality Rendering”).
Regarding claim 18
Mustafa teaches claim 16.
Mustafa does not teach wherein the determining the layer for quantization comprises: determining the layer through a Dynamic Range Estimation (DRE) selection module.
Chalmers teaches wherein the determining the layer for quantization comprises: determining the layer through a Dynamic Range Estimation (DRE) selection module. (Abstract “To achieve this, we propose a stacked convolutional neural network (SCNN) that predicts high dynamic range (HDR) 360 RMswith varying roughness from a limited field of view, low dynamic range photograph. The SCNN is progressively trained from high to low roughness to predict RMs at varying roughness levels, where each roughness level corresponds to a virtual object’s roughness (from diffuse to glossy) for rendering”)
Mustafa and Chalmers are analogous art because they are both directed to machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combined super resolution machine learning for mobile devices of Mustafa with reconstructing reflection maps using stacked CNN of Chalmers.
One of ordinary skill in the art would have been motivated to make this modification in order to “predicts high dynamic range (HDR) 360 RMswith varying roughness from a limited field of view, low dynamic range photograph” as disclosed by (Chalmers abstract “we propose a stacked convolutional neural network (SCNN) that predicts high dynamic range (HDR) 360 RMswith varying roughness from a limited field of view, low dynamic range photograph. The SCNN is progressively trained from high to low roughness to predict RMs at varying roughness levels, where each roughness level corresponds to a virtual object’s roughness (from diffuse to glossy) for rendering. The predicted RM provides high-fidelity rendering of virtual objects to match with the background photograph. We illustrate the use of our method with indoor and outdoor scenes trained on separate indoor/outdoor SCNNs showing plausible rendering and composition of virtual objects in AR/MR. We show that our method has improved quality over previous methods with a comparative user study and error metrics”).
Allowable Subject Matter
Claims 22-34 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN C MANG whose telephone number is (571)270-7598. The examiner can normally be reached Mon - Fri 8:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at 5712707519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN C MANG/Primary Examiner, Art Unit 2126
Prosecution Insights