Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-20 are presented for examination in this application, 18/544,169 filed 12/18/2023, having an effective filing date of 10/12/2023.
The Examiner cites particular sections in the references as applied to the claims below for the convenience of the applicant(s). Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant(s) fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Drawings
The drawings submitted on 12/18/2023 have been considered and accepted.
Information Disclosure Statement
Acknowledgement is made of the information disclosure statements filed 01/21/2025. All patents and non-patent literature have been considered.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
The following appears to be the closest portions of the specification corresponding to the 35 U.S.C 112(f) invocations:
means for determining a practical domain for a following function in a following layer of an artificial neural network
[0076] Aspects of the present disclosure maximize quantization resolution or, equivalently, minimize quantization error by determining activation quantization ranges based on a practical domain of the following function, therefore considering inter-layer dependency. By looking at the next layer, an output of the previous layer may be limited, resulting in a narrower quantization range. Stated another way, interlayer dependencies are considered in order to improve quantization.
means for setting a first quantization range of an output activation of the leading function based on the practical domain
[0087] In some aspects, the process 1300 may include setting a first quantization range of an output activation of the leading function based on the practical domain (block 1304). For example, the process 1300 sets the first quantization range based on a maximum value of a predetermined quantization range and a minimum value of the predetermined quantization range.
means for calculating the practical domain based on an inverse of the following function and a plurality of tolerances
[0086] For example, the process 1300 may include calculating the practical domain based on an inverse of the following function and multiple tolerances for lower bounds and upper bounds. The following function may be an activation function having a many-to-one mapping.
means for setting the first quantization range based on a maximum value of a predetermined quantization range and a minimum value of the predetermined quantization range
[0087] In some aspects, the process 1300 may include setting a first quantization range of an output activation of the leading function based on the practical domain (block 1304). For example, the process 1300 sets the first quantization range based on a maximum value of a predetermined quantization range and a minimum value of the predetermined quantization range.
means for setting a practical range of the leading function in the leading layer based on the first quantization range
[0076] Aspects of the present disclosure maximize quantization resolution or, equivalently, minimize quantization error by determining activation quantization ranges based on a practical domain of the following function, therefore considering inter-layer dependency. By looking at the next layer, an output of the previous layer may be limited, resulting in a narrower quantization range. Stated another way, interlayer dependencies are considered in order to improve quantization.
means for determining a practical domain of the leading function of the leading layer based on the practical range of the leading function
[0078] Assuming these definitions, the practical domain
PNG
media_image1.png
27
90
media_image1.png
Greyscale
can be determined as
PNG
media_image2.png
28
332
media_image2.png
Greyscale
, where ϵl, ϵu, ՠl, and ՠu represent tolerances (or margins) for lower/upper bounds for practical ranges and domains.
means for setting a second quantization range for a prior function of a prior layer, which is immediately before the leading layer, based on the practical domain of the leading function
[0092] Aspect 5: The apparatus of any of the preceding Aspects, in which the at least one processor is further configured to: set a practical range of the leading function in the leading layer based on the first quantization range; determine a practical domain of the leading function of the leading layer based on the practical range of the leading function; and set a second quantization range for a prior function of a prior layer, which is immediately before the leading layer, based on the practical domain of the leading function. Also, [0087] In some aspects, the process 1300 may include setting a first quantization range of an output activation of the leading function based on the practical domain (block 1304). For example, the process 1300 sets the first quantization range based on a maximum value of a predetermined quantization range and a minimum value of the predetermined quantization range.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C 112(b) for being indefinite to particularly point out and distinctly claim the subject matter which the inventor regards as the invention.
Independent claims 1, 8, and 15 recite “determining a practical domain for a following function in a following layer of an artificial neural network”. The closest portions of the specification that aid in understanding the above limitation can be found at para [0078] “Assuming these definitions, the practical domain
PNG
media_image1.png
27
90
media_image1.png
Greyscale
can be determined as
PNG
media_image2.png
28
332
media_image2.png
Greyscale
, where ϵl, ϵu, ՠl, and ՠu represent tolerances (or margins) for lower/upper bounds for practical ranges and domains” and “[0086] For example, the process 1300 may include calculating the practical domain based on an inverse of the following function and multiple tolerances for lower bounds and upper bounds. The following function may be an activation function having a many-to-one mapping.” While the practical domain can and may be derived in such a manner, the metes and bounds of the term “practical” are indefinite and unclear.
As claims 2-7, 9-14, and 16-20 depend from claim 1, they too contain the same deficiencies that render the 35 U.S.C 112(b) rejection to be made and are thus rejected for the same reasons.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C 101 as being unpatentable because the claimed invention in these claims is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg 50-57 (January 7, 2019) (“2019 PEG”).
Regarding claim 1:
Step1 -— Is the claim directed to a process, machine, manufacture, or a composition of matter?
Yes, the claim is directed to a machine.
Step 2A — Prong 1 — Does the claim recite an abstract idea, law of nature, or a natural phenomenon?
Yes, the claim recites an abstract idea:
determine a practical range for a following function in a following layer of an artificial neural network, the artificial neural network including a leading function in a leading layer and the following function in the following layer, which is a subsequent consecutive layer of the artificial neural network — this limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Step 2A — Prong 2 — Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, the claim recites additional elements that do not integrate the judicial exception into a practical application:
An apparatus, comprising at least one memory; and at least one processor coupled to the at least one memory configured to — the use of a computer or machinery in its ordinary capacity amounts to invoking computer components merely as a tool to perform an existing process (see MPEP 2106.05(f)(2)).
set a first quantization range of an output activation of the leading function based on the practical domain — the process of classifying and organizing data amounts to mere instructions to apply an exception, as the use of a computer or machinery in its ordinary capacity amounts to invoking computer components merely as a tool to perform an existing process (see MPEP 2106.05(f)(2)).
Step 2B — Does the claim recite additional elements that amount to significantly more than the abstract idea itself?
No, there are no additional elements that amount to significantly more than the judicial exception.
Regarding claim 8:
Step1 -— Is the claim directed to a process, machine, manufacture, or a composition of matter?
Yes, the claim is directed to a method.
Step 2A — Prong 1 — Does the claim recite an abstract idea, law of nature, or a natural phenomenon?
Yes, the claim recites an abstract idea:
determine a practical range for a following function in a following layer of an artificial neural network, the artificial neural network including a leading function in a leading layer and the following function in the following layer, which is a subsequent consecutive layer of the artificial neural network — this limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Step 2A — Prong 2 — Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, the claim recites additional elements that do not integrate the judicial exception into a practical application:
An apparatus, comprising at least one memory; and at least one processor coupled to the at least one memory configured to — the use of a computer or machinery in its ordinary capacity amounts to invoking computer components merely as a tool to perform an existing process (see MPEP 2106.05(f)(2)).
set a first quantization range of an output activation of the leading function based on the practical domain — the process of classifying and organizing data amounts to mere instructions to apply an exception, as the use of a computer or machinery in its ordinary capacity amounts to invoking computer components merely as a tool to perform an existing process (see MPEP 2106.05(f)(2)).
Step 2B — Does the claim recite additional elements that amount to significantly more than the abstract idea itself?
No, there are no additional elements that amount to significantly more than the judicial exception.
Regarding claim 15:
Step1 -— Is the claim directed to a process, machine, manufacture, or a composition of matter?
Yes, the claim is directed to a machine.
Step 2A — Prong 1 — Does the claim recite an abstract idea, law of nature, or a natural phenomenon?
Yes, the claim recites an abstract idea:
determine a practical range for a following function in a following layer of an artificial neural network, the artificial neural network including a leading function in a leading layer and the following function in the following layer, which is a subsequent consecutive layer of the artificial neural network — this limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Step 2A — Prong 2 — Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, the claim recites additional elements that do not integrate the judicial exception into a practical application:
An apparatus, comprising at least one memory; and at least one processor coupled to the at least one memory configured to — the use of a computer or machinery in its ordinary capacity amounts to invoking computer components merely as a tool to perform an existing process (see MPEP 2106.05(f)(2)).
set a first quantization range of an output activation of the leading function based on the practical domain — the process of classifying and organizing data amounts to mere instructions to apply an exception, as the use of a computer or machinery in its ordinary capacity amounts to invoking computer components merely as a tool to perform an existing process (see MPEP 2106.05(f)(2)).
Step 2B — Does the claim recite additional elements that amount to significantly more than the abstract idea itself?
No, there are no additional elements that amount to significantly more than the judicial exception.
Dependent claim(s) 2-7, 9-14, and 16-20 when analyzed as a whole are held to be patent ineligible under 35 U.S.C 101 because the additional limitation(s) fail(s) to establish that the claim(s) is/are not directed to an abstract idea. The claims are reciting further embellishment of the judicial exception.
Claim 2: recites, inter alia, calculating practical domain based on inverse of a function and tolerances which is equivalent to the abstract idea of a mathematical calculation (see MPEP 2106.04(a)(2) I. C.). Furthermore, in which at least one processor is configured to calculate the practical domain which is equivalent to the use of a computer or machinery in its ordinary capacity amounts to invoking computer components merely as a tool to perform an existing process (see MPEP 2106.05(f)(2)).
Claims 9 and 16 recite analogous limitations to claim 2 and therefore are found rejected on the same grounds.
Claim 3: recites, inter alia, different tolerances for a function, which is equivalent to organizing information and manipulating information through mathematical relationships (see MPEP 2106.04(a)(2) I. A.).
Claims 10 and 17 recite analogous limitations to claim 3 and therefore are found rejected on the same grounds.
Claim 4: recites, inter alia, setting a range based on a max value and a minimum value which is equivalent to which is equivalent to organizing information and manipulating information through mathematical relationships (see MPEP 2106.04(a)(2) I. A.).
Claims 11 and 18 recite analogous limitations to claim 4 and therefore are found rejected on the same grounds.
Claim 5: recites, inter alia, determining a domain of a function of a layer which is equivalent to concepts performed in the human mind (including at least one of observation, evaluation, judgement, opinion) (see MPEP 2106.04(a)(2).
Claims 12 and 19 recite analogous limitations to claim 5 and therefore are found rejected on the same grounds.
Claim 6: recites, inter alia, that the activation function has a many-to-one mapping which is equivalent to which is equivalent to organizing information and manipulating information through mathematical relationships (see MPEP 2106.04(a)(2) I. A.).
Claims 13 and 20 recite analogous limitations to claim 6 and therefore are found rejected on the same grounds.
Claim 7: recites, inter alia, that activation function is a non-linear activation function which is equivalent to which is equivalent to organizing information and manipulating information through mathematical relationships (see MPEP 2106.04(a)(2) I. A.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 4-7, 8-9, 11-16, and 18-20 are rejected under 35 U.S.C 103 as being unpatentable over Mori (US20220237455A1 hereinafter, Mori) in view of Kim (“Low-overhead inverted LUT design for bounded DNN activation functions on floating-point vector ALUs” hereinafter, Kim).
Regarding claim 1:
Mori teaches an apparatus, comprising: at least one memory; and at least one processor coupled to the at least one memory, the at least one processor configured to (see para [0036]: “As illustrated in FIG. 1, the quantization apparatus 2 includes at least one processor 2 a and a memory 2 b communicably connected to the processor 2 a.”):
…
for a following function in a following layer in an artificial network (see fig. 7 that shows the pooling layer and activation layer providing statistical information to determine a quantization range to be fed to a previous layer)
the artificial neural network including a leading function in a leading layer and the following function in the following layer, which is a subsequent layer of the artificial neural network (see fig. 7 that shows the pooling layer and activation layer providing statistical information to determine a quantization range to be fed to a previous layer).
Mori does not explicitly teach to determine a practical domain or set a first quantization range of an output activation of the leading function based on the practical domain.
Kim, however, analogously teaches to determine a practical domain (see pg. 3 section 2.3: “Therefore, calculating the LUT index from the input value 𝑥 is intuitive, i.e., taking several most significant digits from the input value by dividing the input range into several intervals”. Also see pg. 3 section 2.3: “Our approach searches all the LUT entries one by one, so the execution time increases proportionally to the entries. However, since all the vector ALUs perform the search simultaneously, i.e., by a lockstep manner, the large-scaled vector ALUs diminish the search overhead.”) and
set a first quantization range of an output activation of the leading function based on the practical domain (see pg. 4 section 3.1: “For the calculation of 𝑓(𝑥), we search the LUT entry to which the input 𝑥 belongs one by one by sequentially accessing all the LUT entries from address 5‘b 00000. We compute 𝑥−𝑥𝑖 and identify the desired entry by detecting the sign bit from the computation result. For example, as shown in Fig. 2, an incoming input 𝑥 is sequentially subtracted from 𝑥0 at address 5‘b 00000. When a sign bit of the result value changes from positive to negative, the address holds the range where the input 𝑥 belongs. The input 𝑥(0.875) satisfies 𝑥22(0.83984375)≤0.875<𝑥23(0.90234375). ”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Mori and Kim before him or her, to modify the apparatus of claim 1 to include attributes of determine a practical domain or set a first quantization range of an output activation of the leading function based on the practical domain in order to reduce average errors (see Kim at pg. 1 abstract: “Our accuracy analysis shows that with only an entry size of 1/8 compared to state-of-the-art 16-bit fixed-point LUT methods and the small logic overhead, FP-LUT reduces the average errors by 51.8%, 28.4%, 14.4%, and 26.1% ”).
Regarding claims 8 and 15:
Claims 8 and 15 recite analogous limitations to claim 1 and therefore are rejected on the same grounds as claim 1.
Regarding claim 2:
Mori in view of Kim teaches the apparatus of claim 1.
Mori does not explicitly teach in which the at least one processor is further configured to calculate the practical domain based on an inverse of the following function and a plurality of tolerances.
Kim, however, analogously teaches in which the at least one processor is further configured to calculate the practical domain based on an inverse of the following function and a plurality of tolerances (see pg. 1 abstract: “we apply the principle of an inverse function for our design, especially targeting bounded DNN activation functions. We divide a range of function output values linearly by the number of LUT entry sizes and store the corresponding input values in the LUT.”)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Mori and Kim before him or her, to modify the apparatus of claim 2 to include attributes of one processor is further configured to calculate the practical domain based on an inverse of the following function and a plurality of tolerances in order to reduce average errors (see Kim at pg. 1 abstract: “Our accuracy analysis shows that with only an entry size of 1/8 compared to state-of-the-art 16-bit fixed-point LUT methods and the small logic overhead, FP-LUT reduces the average errors by 51.8%, 28.4%, 14.4%, and 26.1% ”).
Regarding claims 9 and 16:
Claims 9 and 16 recite analogous limitations to claim 2 and therefore are rejected on the same grounds as claim 2.
Regarding claim 4:
Mori in view of Kim teaches the apparatus of claim 1.
Mori further teaches in which the at least one processor is further configured to set the first quantization range based on a maximum value of a predetermined quantization range and a minimum value of the predetermined quantization range (see para [0100]: “Specifically, the quantization-range determination step S22 determines the quantization range for the layer parameters of the convolution layer 11 in accordance with the retrieved maximum and minimum values from each of the layers 11, 12, and 13 such that at least part of the frequency distribution range of the layer parameters of the convolution layer 11 is excluded from the determined quantization range for the layer parameters of the convolution layer 11; ”. Also see para [0118]: “Specifically, the present disclosure can use a predetermined low percentile, which is substantially equivalent to the minimum value, such as the 3rd percentile, of the frequency distribution range of the layer parameters as the minimum value thereof, and use a predetermined high percentile, which is substantially equivalent to the maximum value, such as the 97th percentile, of the frequency distribution range of the layer parameters as the maximum value thereof.”).
Regarding claims 11 and 18:
Claims 11 and 18 recite analogous limitations to claim 4 and therefore are rejected on the same grounds as claim 4.
Regarding claim 5:
Mori in view of Kim teaches the apparatus of claim 1.
Mori further teaches in which the at least one processor is further configured to: set a practical range of the leading function in the leading layer based on the first quantization range (see para [0027]: The apparatus includes a determiner configured to determine, based on the statistical information, a quantization range for the layer parameters related to the quantization target layer.”. See also fig. 7 that shows the pooling layer and activation layer providing statistical information to determine a quantization range to be fed to a previous layer).
Mori does not explicitly teach determine a practical domain of the leading function of the leading layer based on the practical range of the leading function; and set a second quantization range for a prior function of a prior layer, which is immediately before the leading layer, based on the practical domain of the leading function.
Kim, however, analogously teaches determine a practical domain of the leading function of the leading layer based on the practical range of the leading function (see pg. 4 section 3.1: “Our FP-LUT method evenly divides the function output range by the LUT entry size and stores the input values corresponding to each entry in the LUT. … Fig. 2 shows how to calculate 𝑦=tanh(𝑥=0.875) with the 32-entry FP-LUT. The hyper-tangent function has a symmetry point of [0,0] and a maximum value of 1; thus, the function output varies from 0 to 1, i.e., from the symmetry point to the maximum. The FP-LUT method equally divides the output range into 32 intervals, and each interval increases by 1/32. The method stores the input values corresponding to each interval 𝑖 to LUT, i.e., 𝑓−1(𝑦𝑖)”.); and
set a second quantization range for a prior function of a prior layer, (see pg. 4 section 3.1: “For the calculation of 𝑓(𝑥), we search the LUT entry to which the input 𝑥 belongs one by one by sequentially accessing all the LUT entries from address 5‘b 00000. We compute 𝑥−𝑥𝑖 and identify the desired entry by detecting the sign bit from the computation result. For example, as shown in Fig. 2, an incoming input 𝑥 is sequentially subtracted from 𝑥0 at address 5‘b 00000. When a sign bit of the result value changes from positive to negative, the address holds the range where the input 𝑥 belongs. The input 𝑥(0.875) satisfies 𝑥22(0.83984375)≤0.875<𝑥23(0.90234375). ”).
Mori teaches setting a quantization range for a prior function of a prior layer immediately before the leading layer (See also fig. 7 that shows the pooling layer and activation layer providing statistical information to determine a quantization range to be fed to a previous layer).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Mori and Kim before him or her, to modify the apparatus of claim 5 to include attributes of determine a practical domain of the leading function of the leading layer based on the practical range of the leading function; and set a second quantization range for a prior function of a prior layer, which is immediately before the leading layer, based on the practical domain of the leading function. in order to reduce average errors (see Kim at pg. 1 abstract: “Our accuracy analysis shows that with only an entry size of 1/8 compared to state-of-the-art 16-bit fixed-point LUT methods and the small logic overhead, FP-LUT reduces the average errors by 51.8%, 28.4%, 14.4%, and 26.1% ”).
Regarding claims 12 and 19:
Claims 12 and 19 recite analogous limitations to claim 5 and therefore are rejected on the same grounds as claim 5.
Regarding claim 6:
Mori in view of Kim teaches the apparatus of claim 1.4
Mori further teaches in which the following function comprises an activation function having a many-to-one mapping (see paras [0094] – [0099]: “Specifically, the first embodiment results in each of the selected layer parameters, which is an N-bit floating-point value, of the convolution layer 11 being quantized to a corresponding one of lower bitwidth values, i.e., L-bit integer values, using the symmetric quantization in accordance with the following expression (2); the number N is for example 32, and the number L is for example 8:x f→Δx x q where: xf represents an original floating-point value (layer parameter) of the convolution layer 11, Δx represents the quantization interval, xq represents a corresponding quantized integer, and the symbol “→” represents mapping of the left-side value to the right-side value”. Also see para [0100]: “Specifically, the quantization-range determination step S22 determines the quantization range for the layer parameters of the convolution layer 11 in accordance with the retrieved maximum and minimum values from each of the layers 11, 12, and 13 such that at least part of the frequency distribution range of the layer parameters of the convolution layer 11 is excluded from the determined quantization range for the layer parameters of the convolution layer 11;”)
Regarding claims 13 and 20:
Claims 13 and 20 recite analogous limitations to claim 6 and therefore are rejected on the same grounds as claim 6.
Regarding claim 7:
Mori in view of Kim teaches the apparatus of claim 6.
Mori further teaches in which the activation function comprises a non-linear activation function (see para [0193]: “Specifically, the activation function according to the third embodiment has the non-saturation region S01 in its input-output characteristic, and serves as a non-linear function that nonlinearly transforms an input value to an output value when the input value lies within the non-saturation region S01.”).
Allowable Subject Matter
Claims 3, 10, and 17 are objected to as being dependent upon a rejected base claim, but it would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims provided the 101 rejections are overcome.
Regarding claims 3, 10, and 17, the closest prior art of record to the limitations of the aforementioned claims, Kim et al. (“Low-overhead inverted LUT design for bounded DNN activation functions on floating-point vector ALUs”) recites error-tolerant outputs, inverse-function values, and taking the activation function range in intervals however it does not disclose four separate tolerances as disclosed in the instant case.
The examiner has found that the distinct features of the applicant’s claimed invention over the prior art is the explicit claiming of the aforementioned limitations, specified in claims 3, 10, and 17. When viewed individually or in combination with other prior art of record, the limitations specified in the aforementioned claims are distinct.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Andrew A Bracero whose telephone number is (571) 270-0592. The examiner can normally be reached Monday - Friday 9:00a.m. - 5:00 p.m. ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi at (571) 270-7519 can be reached at Monday - Friday 9:00a.m. - 5:00 p.m. ET. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW BRACERO/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
