DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1,8, and 19-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
For claims 1, 8, and 19-20:
Step 2A prong 1: The claims are considered an exception because they recite concepts that can be performed in the human mind (including an observation, evaluation, judgment, opinion such as: “obtaining a data variation range”, “determining a target iteration interval”, “adjust a quantization parameter”, and “implement quantization of data”) and/or using pen and paper, and they do not necessarily require generic or any computer components to be completed. Although the claims do recite computer components that execute the operations/steps performed (“neural network”, “iteration”, “non-transitory computer readable storage medium”, “computer program”, and “device”), this does not change the fact that the operations/steps can still be performed in the mind/pen-paper only, and that the computer components are not necessary.
Step 2A prong 2: (i)The recited additional elements specified above do not integrate the judicial exception into a practical application because the generic elements recited do not add a meaningful limitation to the abstract idea as they amount to simply implementing the abstract idea utilizing a computer; (ii) Also, the object of the observing and evaluating in the claims, is a neural network. The neural network is not positively claimed (it is just observed and evaluated); (iii)There is also no indication what the systems that employs the neural network specifically is, and as such the claim is not practical, but rather preemptive of all systems and applications that use neural networks; (iv) the objects of the determining and evaluating operations are not sufficiently complex and/or laborious that would mandate the use of a computing components.
Step 2B: the claim recites the additional limitation of “neural network”. This additional limitation is well-understood, routine, and conventional (see Wikipedia “neural network” page).
For claim 8, the claim further recites obtaining a variation range, which is still an observation or calculation. The obtaining appears to be in the planning stage of the method as it is based on quantities not necessarily calculated by a computer yet. Thus, they can be performed in the mind/pen-paper (Step 2A prong 1). The operations are not sufficiently complex (Step 2A prong 2), and that the obtaining is well-understood, routine, and conventional (Step 2B).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
Initially it should be noted that this application has two same inventors (out of eight) as the patent US 12,001,955 (LIU, Shaoli and ZHANG, Xishan) and the assignee of this application (Shanghai Cambricon Information Technology Co., Ltd) is a sister company to ANHUI Cambricon Information Technology Co., Ltd. both subsidiaries or parts of the larger Cambricon Technologies.
Claims 1 and 19-20 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 5 of U.S. Patent No. 12,001,955. Although the conflicting claims are not identical, they are not patentably distinct from each other because of the following:
For claim 1, U.S. 12,001,955 claim 5 anticipates claim 1 of the instant application as demonstrated by the following table:
Instant Application
U.S. 12,001,955
comments
1. A quantization parameter
adjustment method of
a neural network,
1. A neural network
comprising: obtaining a data variation range of data to be quantized; and determining a target iteration interval according to the data variation range of the data to be quantized, so as to adjust a quantization parameter in a neural network operation according to the target iteration interval, wherein the target iteration interval comprises at least one iteration, and the quantization parameter of the neural network is configured to implement quantization of the data to be quantized in the neural network operation
quantization method applied to a terminal, wherein the terminal is configured to run a neural network to perform a target task, and the method comprises: according to a precision requirement of the target task and/or the terminal, determining a plurality of pieces of data to be quantized from target data, wherein each piece of data to be quantized is a subset of the target data, the target data is data to be operated and quantized in a layer of the neural network to be quantized, and the data to be operated and quantized includes at least one of an input neuron, a weight, a bias, and a gradient; quantizing each piece of data to be quantized according to a corresponding quantization parameter to obtain a piece of quantized data; during a fine-tuning stage and/or training stage of a neural network operation, obtaining a variation range of the data to be quantized in a current iteration and historical iterations, wherein the historical iterations are iterations before the current iteration; determining, according to the variation range of the data to be quantized, a target iteration interval corresponding to the data to be quantized to enable the layer to be quantized to update the quantization parameter of the data to be quantized according to the target iteration interval, wherein the target iteration interval includes at least one iteration; according to a data bit width of the data to be quantized in the current iteration, determining the data bit width of the data to be quantized in iterations within the target iteration interval to enable the neural network to determine the quantization parameter according to the data bit width of the data to be quantized in the iterations within the target iteration interval; according to a point position of the data to be quantized in the current iteration, determining the point position of the data to be quantized in the iterations within the target iteration interval, wherein the point position includes a first type of the point position and/or a second type of the point position; wherein obtaining the variation range of the data to be quantized in the current iteration and historical iterations includes: according to the point position of the data to be quantized in the current iteration and a point position in an historical iteration, computing a moving average of the point position of the data to be quantized in each iteration interval, wherein the point position of the data to be quantized in each iteration interval includes the first type of the point position and/or the second type of the point position; obtaining a first data variation range according to a first moving average of the point position of the data to be quantized in the current iteration and a second moving average of the point position in an iteration corresponding to a previous iteration interval; wherein the target iteration interval corresponding to the data to be quantized is determined according to the first data variation range; and wherein obtaining the first data variation range according to the first moving average of the point position of the data to be quantized in the current iteration and the second moving average of the point position in the iteration corresponding to the previous iteration interval includes computing a difference between the first moving average and the second moving average, and determining an absolute value of the difference as the first data variation range; and obtaining a quantization result of the target data according to the piece of quantized data corresponding to each piece of data to be quantized, so that an operation is performed in the layer to be quantized according to the quantization result of the target data.
All elements of instant application claim 1 are in the patent claim 5
5. The method of claim 1, further comprising: determining a quantization error corresponding to each piece of data to be quantized according to the piece of quantized data corresponding to each piece of data to be quantized, adjusting the data bit width of each piece of data to be quantized to obtain an adjusted bit width corresponding to each piece of data to be quantized according to the quantization error and an error threshold corresponding to each piece of data to be quantized, updating the data bit width corresponding to each piece of data to be quantized to a corresponding adjusted bit width, and computing a corresponding adjusted quantization parameter according to each piece of data to be quantized and the corresponding adjusted bit width, and quantizing each piece of data to be quantized according to the corresponding adjusted quantization parameter.
For claims 19-20, the claim is anticipated by respective claim 5 for the same reasons. for claim 1.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitations uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: "obtaining unit configured to obtaining" and "iteration interval determination unit configured to determine" in claim 20. The functions are deemed non-specialized functions.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
A review of the specification shows that the following appears to be the corresponding structure described in the specification for the 35 U.S.C. 112(f) limitation: “processor”.
If applicant wishes to provide further explanation or dispute the examiner’s interpretation of the corresponding structure, applicant must identify the corresponding structure with reference to the specification by page and line number, and to the drawing, if any, by reference characters in response to this Office action.
If applicant does not intend to have the claim limitation(s) treated under 35 U.S.C. 112(f), applicant may amend the claim(s) so that it/they will clearly not invoke 35 U.S.C. 112(f), or present a sufficient showing that the claim recites/recite sufficient structure, material, or acts for performing the claimed function to preclude application of 35 U.S.C. 112(f).
For more information, see MPEP § 2173 et seq. and Supplementary Examination Guidelines for Determining Compliance With 35 U.S.C. 112 and for Treatment of Related Issues in Patent Applications, 76 FR 7162, 7167 (Feb. 9, 2011).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 2-7 and 9-19 are rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention.
For claim 2, the term “the iteration” is indefinite because there is an option for more than one iteration to be declared earlier and so it is unclear to which the claim refers to. It is suggested the claim be amended to “the one or more iteration”.
For claims 2 and 4-5, the term “the point location” is indefinite because two “point location” instances were declared earlier, and so it is unclear to which the claim refers to. It is suggested the claim be amended to “the determined point location”.
For claim 2, the term “the adjust the point location” lack sufficient antecedent basis in the claim. It is suggested the claim be amended to “the [[adjust]] the determined point location”.
For claim 3, the term “the iteration of the target iteration interval” lack sufficient antecedent basis in the claim. It is suggested the claim be amended to “the [[iteration]] respective iterations of the target iteration interval”.
For claim 3, the term “(adjust) the point location (of the neural network operation)” is indefinite because more than one instance of “point location” were declared earlier, and so it is unclear to which the claim refers to. It is suggested the claim be amended to “[[the]] a point location”.
For claims 6 and 17, the term “the quantized data of the current verify iteration” (more than one instance) lacks sufficient antecedent basis in the claim. “data to be quantized of the current verify iteration” was declared earlier but not “the quantized data of the current verify iteration”. It is suggested the first instance be amended to “[[the]] a quantized data of the current verify iteration”.
For claims 7 and 15-16, the term “the data bit width” (multiple instances) lacks sufficient antecedent basis in the claim. “target data bit width” was declared earlier but not “the data bit width”. It is suggested the first instance for each claim be amended to “[[the]] a data bit width”.
For claim 9, the term “the previous iteration interval” lacks sufficient antecedent basis in the claim. It is suggested the claim be amended to “[[the]] a previous iteration interval”.
For claims 9-11, the term “the historical verify iterations” lacks sufficient antecedent basis in the claim. It is suggested the first instance be amended to “[[the]] historical verify iterations” or all instances be amended to “the historical [[verify]] iterations”.
For claims 9 and 12, the term “the target data bit width” lacks sufficient antecedent basis in the claim. It is suggested claim 9 be amended to “[[the]] a target data bit width”.
For claim 10, the term “the preset number of verify iterations” lacks sufficient antecedent basis in the claim. “preset number of intermediate moving mean values “ was declared earlier but not “preset number of verify iterations”. It is suggested the claim be amended to “[[the]] a preset number of verify iterations”.
For claim 10, the term “the current verify” lacks sufficient antecedent basis in the claim. It is suggested the claim be amended to “the current verify iteration”.
For claim 16-17, the terms “the first error” and “the obtained first error” (multiple instances) lack sufficient antecedent basis in the claim. It is suggested the first instance be amended to “[[the]] an obtained first error” and “[[the]] a first error”.
For claim 18, the terms “the first preset iteration interval” and “the second preset iteration interval” (multiple instances) lack sufficient antecedent basis in the claim. It is suggested the first instances be amended to “[[the]] a first preset iteration interval” and “[[the]] a second preset iteration interval”.
For claim 18, the term “the convergence” lacks sufficient antecedent basis in the claim. It is suggested the claim be amended to “[[the]] a convergence”.
For claim 19, the term “the steps” lacks sufficient antecedent basis in the claim. It is suggested the claim be amended to “[[the]] a steps”.
Dependent claims inherit rejections.
Allowable Subject Matter
Claims 1-20 would be allowable if all 35 USC § 112, Double Patenting, and 35 USC § 101 rejections are overcome.
Reasons for Allowability
For Independent claims 1 and 19-20,
The claim is allowable because although prior art has been found to teach quantization using iterations for neural networks, no prior art was found to teach: adjusting the quantization parameters as described in the claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YAIR LEIBOVICH whose telephone number is (571)270-3796. The examiner can normally be reached 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ashish Thomas can be reached at 571-272-0631. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YAIR LEIBOVICH/Primary Examiner, Art Unit 2114
Prosecution Insights