DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is responsive to amendment filed on 02/25/2026. Claims 1-27 are pending. The amendment have overcome the drawing objection and claim objections set forth in the previous office action.
Response to Arguments
In response to applicant’s argument regarding rejection under 35 U.S.C. 102 on page 11, “Applicant submits that Pasca does not describe, either expressly or inherently, each and every element as set forth in the claims. For example, Pasca does not anticipate or suggest "the configurable nonlinear activation function circuit further comprises a bypass circuit configured to selectively bypass at least one of the successive linear approximators," as recited in amended independent Claim 1 (emphasis added) and similar features recited in amended independent Claims 10 and 19. Rather, Pasca is silent with respect to a bypass circuit, and especially to a bypass circuit configured to selectively bypass at least one of the successive linear approximators.”
Examiner respectfully disagrees because Pasca teach or suggest a bypass circuit as a mux 216E in figure 13 to selectively bypass multiply-add blocks 212A and 212B when the decode 602B sent control value of 1 or 2 to approximate the positive or negative saturation region. See rejection below for details.
Claim Objections
Claims 1-27 are objected to because of the following informalities:
Claim 1 line 14-15; claim 10 line 14-15; claim 19 line 16-17 “the successive linear approximators” should be “the at least two successive linear approximators” as antecedently recited.
Dependent claims are also objected for inheriting the same deficiencies in which claims they depend on.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 4-11, 13-20, and 22-27 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Pasca - US 20220230057.
Regarding claim 1, Pasca teaches a processor (Pasca figure 1 illustrates a system 10 [i.e., a processor]), comprising:
a configurable nonlinear activation function circuit (Pasca [0021] figure 1 illustrates an integrated circuit 12 having RNN to implement sigmoid and hyperbolic tangent function, also see figures 3 and 11 illustrate sigmoid and tanh(x) as non-linear. [0072] figure 13 illustrates a fused activation function approximation circuit to implement sigmoid and tanh functions. Thus, the integrated circuit 12 corresponds to a configurable nonlinear activation function circuit]) configured to:
determine, based on a selected nonlinear activation function, a set of parameters for the selected nonlinear activation function (Pasca figure 13 [0073] describes in the case of the sigmoid function, the polynomial evaluated may be represented as (C0+x*(C1+C2*x)), and in the case of the hyperbolic tangent, the polynomial evaluated is represented as (x*(C1+x.sup.2*(C3+C5*x.sup.2))). Figure 13 illustrates the coefficients C0-C2 [i.e., a set of parameters] are determined based on the sigmoid function [i.e., a selected nonlinear activation function]); and
generate output data based on application of the set of parameters for the selected nonlinear activation function (Pasca figure 13, the fused activation function approximation circuit is configured to generate output based on application of the coefficients for the selected function), wherein:
the configurable nonlinear activation function circuit comprises at least one nonlinear approximator (Pasca figure 13 illustrates the fused activation function approximation circuit 600 [i.e., at least one nonlinear approximator] for approximating functions of tanh and sigmoid) comprising at least two successive linear approximators (Pasca figure 13 the fused circuit 600 includes multiply-add blocks 212A and 212B and tables 202 are connected in series or successively connected [i.e., at least two successive linear approximators]), and each linear approximator of the at least two successive linear approximators is configured to approximate a linear function using one or more function parameters of the set of parameters (Pasca figure 13 each multiply-add block connected to tables 202 configured to approximate a linear function, such as (C1+C2x) for sigmoid function, wherein C2 corresponds to one or more function parameters of the set of parameters); and
the configurable nonlinear activation function circuit further comprises a bypass circuit configured to selectively bypass at least one of the successive linear approximators (Pasca figure 13, the fused activation function approximation circuit 600 is part of the integrated circuit 12 and comprises a mux 216E [i.e., a bypass circuit], wherein the mux 216E receives output from 212C, value 1, and output of 216F, wherein [0079] describes the value 1 represents an approximation of the hyperbolic tangent and sigmoid in a positive saturation interval [i.e., such as area 107 for sigmoid in figure 3 and area 506 for tanh for figure 11] and the output of 216F represents the approximation of negative saturation region (see figure 3 and 11 for negative saturation region). Thus, the mux 216E corresponds to the bypass circuit configured to selectively bypass output from 212C, which includes the multiply-add blocks 212A-212B and table 202 [i.e., at least one of the successive linear approximator] when the mux 216E receives control value 1 or 2 for approximation of positive or negative saturation region [0079]).
Regarding claim 2, Pasca teaches the processor of Claim 1, wherein each linear approximator of the at least two successive linear approximators comprises: a stage input; a coefficient input; a constant input; and a stage output (Pasca figure 13 [0077] describes multiply-add block 212A output (c.sub.M+z*c.sub.R), wherein c.sub.M corresponds to a constant input, z corresponds to a stage input, c.sub.R corresponds to a coefficient input and (c.sub.M+z*c.sub.R) corresponds to stage output. Similarly, [0077] describes multiply-add block 212B may output c.sub.L+z(c.sub.M+z*c.sub.R), wherein c.sub.L corresponds to constant input, z corresponds to stage input, (c.sub.M+z*c.sub.R) corresponds to coefficient input, and c.sub.L+z(c.sub.M+z*c.sub.R) corresponds to stage output).
Regarding claim 4, Pasca teaches the processor of Claim 1, wherein the at least one nonlinear approximator comprises a quadratic approximator comprising two successive linear approximators (Pasca figure 13 [0037] describes a second degree polynomial function is used for sigmoid approximation C0+C1*x+C2*x^2. Figure 13 also illustrates the fused circuit 600 includes 212A and 212B [i.e., a quadratic approximator comprising two successive linear approximators]).
Regarding claim 5, Pasca teaches the processor of Claim 2, wherein: the at least one nonlinear approximator further comprises a region finder component configured to determine an input value region (Pasca figure 13 illustrates the fused circuit 600 includes a barrel shifter 204 [i.e., a region finder component] generate input that map uniquely to a sub-interval [i.e., an input value region]), and each linear approximator of the at least two successive linear approximators is further configured to determine the coefficient input and constant input based on the input value region (Pasca figure 13 [0075] describes the data used in operation for the multiply-add blocks are stored in table 202 for different sub-intervals or input regions. Thus, based on the sub-interval determined by the barrel shifter 204, coefficients and constant input stored in the tables are determined for the multiply-add blocks. For example, block 212A performs (C1+C2x) for sigmoid function using the corresponding coefficient C2 and constant C1 stored in the tables 202).
Regarding claim 6, Pasca teaches the processor of Claim 1, wherein the at least one nonlinear approximator further comprises a sign and offset corrector component configured to modify a stage output from at least one linear approximator of the at least two successive linear approximators (Pasca figure 13 the fuse circuit 600 includes subtractor 214, mux 216C, and block 212C [i.e., a sign and offset corrector component], wherein [0078] describes the subtractor 214 and mux 216C are configured to generate either output of the second multiply-add block 212B (e.g., c.sub.L+z(c.sub.M+z*c.sub.R)) or the negative of the output of the second multiply-add block 212B (e.g., −(c.sub.L+z(c.sub.M+z*c.sub.R))) [i.e., modified stage output of at least one linear approximator], which is sign and the adder of block 212C configured to add 0 or 1, which is an offset).
Regarding claim 7, Pasca teaches the processor of Claim 6, wherein the sign and offset corrector component is further configured to invert a sign of the stage output in order to modify the stage output from the at least one linear approximator of the at least two successive linear approximators (Pasca figure 13 the fuse circuit 600 includes subtractor 214, mux 216C, and block 212C [i.e., the sign and offset corrector component], wherein [0078] describes the subtractor 214 and mux 216C are configured to generate the negative of the output of the second multiply-add block 212B (e.g., −(c.sub.L+z(c.sub.M+z*c.sub.R))), which inverts a sign of the stage output to modify the stage output of the block 212B].
Regarding claim 8, Pasca teaches the processor of Claim 6, wherein the sign and offset corrector component is further configured to add an offset to the stage output in order to modify the stage output from the at least one linear approximator of the at least two successive linear approximators (Pasca figure 13 the fuse circuit 600 includes subtractor 214, mux 216C, and block 212C [i.e., the sign and offset corrector component] configured to add 0 or 1 [i.e., an offset] to c.sub.L+z(c.sub.M+z*c.sub.R) [i.e., the stage output] of block 212B [i.e., the at least one linear approximator] to modify the stage output when the sigmoid function is selected. See [0078]).
Regarding claim 9, Pasca teaches the processor of Claim 1, wherein each linear approximator of the at least two successive linear approximators is further configured to select the one or more function parameters based at least in part on selecting one or more non-uniform segments of a function approximation in order for each linear approximator of the at least two successive linear approximators to approximate a linear function using one or more function parameters of the set of parameters (Pasca figure 13 illustrates each of blocks 212A and 212B and tables 202 configured to select the coefficients stored based on the determined sub-interval, wherein figure 6 [0030] describes the non-linear function, such as sigmoid is split into a number of non-uniformly size sub-intervals. Thus, each of the blocks 212 and 202 are configured to select parameters based on one or more nonuniform segments to approximate a linear function).
Claims 10-11, 13-18 recite method claims that would be practiced by the apparatus claims 1-2, 4-9. Thus, they are rejected for the same reasons.
Claims 19-20, 22-27 recite product claims having similar limitations as the method claims 10-11, 13-18. Thus, they are rejected for the same reasons. Claim 19 further recites a non-transitory computer-readable medium comprising computer-executable instructions that, when executed by a processor, cause the processor to perform a method (Pasca [0023] describes designers may implement their high level designs using design software and use compiler 16 to convert the high level program into a low level description, wherein the compiler 16 may provide machine-readable instructions to the integrated circuit device 12. Also see claims 1 that claims a tangible, non-transitory, machine-readable medium, comprising machine-readable instructions that, when executed by one or more processors, cause the processors to perform operations).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 12, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Pasca in view of Azadet - US 20140086361.
Regarding claim 3, Pasca teaches the processor of Claim 1, wherein at least one nonlinear approximator comprises a quadratic approximator comprising two successive linear approximators (Pasca [0037] describes a second degree polynomial function is used for sigmoid approximation C0+C1*x+C2*x^2. Figure 13 also illustrates the fused circuit 600 includes 212A and 212B [i.e., a quadratic approximator comprising two successive linear approximators]). Pasca does not teach the at least one nonlinear approximator comprises a cubic approximator comprising three successive linear approximators. However, Azadet teaches at least one nonlinear approximator comprises a cubic approximator comprising three successive linear approximators (Azadet figure 6 [0044] describes a Tayler sum computation block [i.e., at least one nonlinear approximator] to approximate a user-defined non-linear function as illustrated in figure 5A, wherein figure 6 illustrates a circuit includes multiplier 610-1 and adder 620-1 [i.e., a linear approximator], multiplier 610-2 and adder 620-2 [i.e., another linear approximator], and multiplier 610-4 and adder 620-3 [i.e., another linear approximator], wherein the three approximator are successively connected to implement equation 3 [0043-0044], which is a cubic polynomial).
It would have been obvious for one of ordinary skills in the art before the effective filing date of the claimed invention to modify the circuit as illustrated in figure 13 of Pasca to approximate a non-linear function using degree of three or cubic polynomial as disclosed in Azadet. This modification would have been obvious because both references discloses concept of approximating nonlinear function. Furthermore, [0034] Pasca discloses while 2nd or quadratic degree polynomial maybe suitable, but a higher degree polynomial function may be utilized because as recognized by Pasca [0033] higher degree polynomial may produce a more accurate approximation than lower degree polynomial. Accordingly, modifying figure 13 to include another multiply-add block 212 to perform cubic degree polynomial as illustrated in figure 6 of Azadet would increase the accuracy of approximation. Moreover, Azadet also recognized in [0043-0044] that implementing cubic degree polynomial using successive linear approximator decreases the number of multiplier and adder to 6 components because of the factorization Horner algorithm as illustrated in equation 3.
Claims 12 and 21 recite method and product claims having similar limitations as the apparatus claim 3. Thus, they are rejected for the same reasons.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUY DUONG whose telephone number is (571)272-2764. The examiner can normally be reached Mon-Friday 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Caldwell can be reached at (571) 272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUY DUONG/Examiner, Art Unit 2182 (571)272-2764
/ANDREW CALDWELL/Supervisory Patent Examiner, Art Unit 2182