Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office Action is sent in response to Applicant’s Communication received
on 09/30/2025 for application number 17/597066.
Response to Amendments
Claims 11, 12, 15, 16, 19, and 20 have been amended.
Claims 13, 14, 17, 18, 21, and 22 have been canceled.
Claims 11, 12, 15, 16, 19, and 20 remain pending in the application.
The amendment filed 09/30/2025 is sufficient to overcome the 101 rejections to
claims 15 and 16 for being directed towards non-statutory subject matter. The previous rejections have been withdrawn.
Argument 1, regarding the 101 rejections, applicant argues that amendments made to claims 15 and 16 overcome their 101 rejections for being directed towards non-statutory subject matter. Examiner agrees and the rejections are withdrawn.
Applicant also argues that tangible hardware components are used to execute operations such as implementing physical multiply-add arrays, so the claim should not be characterized as a mathematical concept. Examiner respectfully disagrees because even if the hardware is tangible, the multiply-add arrays are being used to perform calculations which result in the claim being directed towards a mathematical concept.
Applicant also argues that “reading M*N feature map data of all input channels and weights of a preset number of output channels, wherein a value of M*N and a value of the preset number are respectively determined by preset Y*Y weights, and M, N and Y are all positive integers” should not be considered generic data handling or mere data gathering because a hardware-software codesign is used to dynamically calibrate input and output dimensions to the convolution kernel size and minimizing redundant data transfers. Examiner respectfully disagrees because even though these limitations are reflected in the specification, they are not reflected in the claims.
Applicant also argues that the invention is directed towards solutions including reducing storage access, reducing unnecessary calculations and SRAM reads, reducing bandwidth consumption, and reducing power usage. Examiner notes that even though these solutions may be reflected in the specification, they are not reflected in the claims. To evaluate an improvement to a computer or technical field, the specification must set for than improvement in technology and the claim itself must reflect the disclosed improvement. See MPEP 2106.04(d)(1) and 2106.05(a).
The full 101 rejections are outlined below.
Argument 2, regarding the 103 rejections, applicant argues that Chen does not teach “reading m*n feature map data of all input channels” or that “data of all input channels” is the input for subsequent convolution calculations. Applicant argues that Chen recites “reading the weight values on the corresponding third predetermined number m′ of pointwise convolution channels”, so not all channels are read. Examiner respectfully disagrees because the rest of the citation recites “in the fourth predetermined number n of pointwise convolution kernels correspondingly to the intermediate feature values of the first predetermined number p of points on the third predetermined number m′ of depthwise convolution output channels”, meaning m’ convolution channels are read for n’ kernels, meaning m*n feature map data is read.
Applicant also argues Chen does not teach “wherein a value of M*N and a value of the preset number are respectively determined by preset Y*Y weights” because the way to determine the predetermined values for M and N is not recited. Examiner respectfully disagrees because the way the values M and N in the claims is not recited beyond preset weights being used to determined M and N. Chen teaches weight values corresponding to predetermined values for M and N being used for convolution calculations, P00113.
Applicant also argues that Yang does not teach M, N, and Y being positive integers. Examiner notes that Yang teaches M, N and Y are all positive integers (The number of output channels, weights, and feature maps are positive integers, P0090-P0091). Applicant argues the teaching in Yang does not teach positive attributes of M, N, or Y or the relationship between YY and M and N. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Examiner notes that Yang teaches dimensions of feature map data being positive integers as described in P0090-P0091 of Yang, with Chen teaching the relationship between weights and dimensions M and N.
Applicant also argues that Chen and Yang do not teach a feature described in P0028-P0030 of the specification detailing clarifying the determination logic of the feature map data size and the number of output channels, ensuring that the convolution kernel size matches the input data dimension, and enabling the convolution calculation to be efficiently performed according to preset rules. Examiner notes that these features are not explicitly recited in the claims.
Applicant also argues that Chen does not teach “wherein a mode of the convolution calculation comprises: not performing the convolution calculation in a case that the feature map data or the weights of the output channels are zero” because there is no mention of feature map data or channel weights. Examiner respectfully disagrees because P0118 of Chen describes not performing a convolution calculation when p points are at zero. These points are used to determine and are correlated with the feature map data and weights as described in P0116 of Chen which recites “multiplication calculations are respectively performed on the m′ intermediate feature values of each point and the m′ weight values in a first convolution kernel of the pointwise convolution kernels (11), and the m′ multiplication calculation results are accumulated, to obtain the current pointwise convolution partial sums of p points in the first output feature map”. Thus, the p points are reflective of a feature map and m weight values. The entire calculation is described in more detail in P0111-P0118 of Chen.
Applicant also argues that Chen does not teach “selecting one from a plurality of same values to perform the convolution calculation in a case that there are a plurality of feature map data with the same values” because Chen does not mention “selecting one from a plurality of same values to perform the convolution calculation in a case that there are a plurality of feature map data with the same values”. Examiner respectfully disagrees because Chen teaches “performing pointwise convolution calculation on the intermediate feature values of the first predetermined number p of points on the third predetermined number m′ of depthwise convolution output channels with the weight values on the corresponding third predetermined number m′ of pointwise convolution channels in the fourth predetermined number n of pointwise convolution kernels” as described in P0113 of Chen. In this calculation, multiple feature maps contain the same values for n and m but feature map data from separate maps and kernels may be chosen to perform the convolution calculation as further described in figure 8 and P0114.
Applicant also argues that Chen does not teach “inputting, in the second cycle, the M* 1 feature map data of the second line of the first input channel and the weights of the preset number of output channels into the calculation array of the preset number of output channels, using the second group of Z*1 multiply-add units to perform the multiply-add calculation and then obtaining an intermediate result of Z points of a next line, shifting the feature map data of the first line to a left side and making all multiply-add calculations in a same output point are implemented in a same multiply-add unit” because the bit shifting taught in Chen is different from shifting feature map data of the first line to a left side and making all multiply-add calculations in a same output point are implemented in a same multiply-add unit. Applicant argues this limitation has a purpose of hardware resource scheduling, whereas the bit shifting taught in Chen has a purpose of adjusting quantization precision. Examiner notes that hardware resource scheduling is not recited in the claims, and that shifting feature map data and making multiply-add calculations in a same output point implemented with the same multiply-add unit may be interpreted under the broadest reasonable interpretation which includes bit shifting so that outputs of bit operations result in the same unit. Applicant also argues that the current inventions use of shifting has a purpose of reducing power consumption and SRAM read times, whereas Chen’s use of bit shifting does not recite cyclical data shifting. Examiner notes that the claims as they are currently recited do not include any language directed towards reducing power consumption or SRAM read time.
Chen teaches inputting, in the second cycle, the M* 1 feature map data of the second line of the first input channel and the weights of the preset number of output channels into the calculation array of the preset number of output channels, using the second group of Z*1 multiply-add units to perform the multiply-add calculation and then obtaining an intermediate result of Z points of a next line, shifting the feature map data of the first line to a left side and making all multiply-add calculations in a same output point are implemented in a same multiply-add unit (the steps of reading the intermediate feature values of predetermined input and output channels, inputting them into pointwise convolution calculations and accumulation calculations, and performing the pointwise convolution calculations and accumulation calculations to obtain multiply-accumulate (MAC) units may be repeated multiple times, P0120, P0130. Bit shifting may be used to obtain low precision or high precision results, making the units the same, P0134).
The full 103 rejections are outlined below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 11, 12, 15, 16, 19, and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: The claims recite a method, storage medium, and device, each being one of the four categories of eligible subject matter.
Claims 11, 15, and 19
Step 2A Prong 1: The claims recite the following limitations:
wherein a mode of the convolution calculation comprises: not performing the convolution calculation in a case that the feature map data or the weights of the output channels are zero, and selecting one from a plurality of same values to perform the convolution calculation in a case that there are a plurality of feature map data with the same values (Mathematical Concept);… using a first group of Z*1 multiply-add units to perform a multiply-add calculation and then obtaining Z calculation results, wherein Z is determined by the preset Y*Y weights (Mathematical Concept); … and using a second group of Z*1 multiply-add units to perform the multiply-add calculation and then obtaining Z calculation results, until after a multiply-add calculation of a Y-th cycle is completed once a reading operation is performed (Mathematical Concept), … and using a Y+2 group of Z* 1 multiply-add units to perform the multiply-add calculation and then obtaining Z calculation results, until after a Y*Yth cycle of the reading operation is performed, all multiply-add calculations of Z data in the first line of the first input channel are completed (Mathematical Concept); … performing the multiply-add calculation for the M* 1 feature map data of each line sequentially, until after Y*Y* preset number of cycles once the reading operation is performed, all the multiply-add calculations of Z data in the first line are completed (Mathematical Concept), … and repeating a same operation as completing all the multiply-add calculation of Z data in the first line, until the feature map data of all the input channels are calculated (Mathematical Concept)… using the second group of Z*1 multiply-add units to perform the multiply-add calculation and then obtaining an intermediate result of Z points of a next line, shifting the feature map data of the first line to a left side and making all multiply-add calculations in a same output point are implemented in a same multiply-add unit (Mathematical Concept); …until after the multiply-add calculation of the Y-th cycle is completed once the reading operation is performed, in a Y+lth cycle, inputting M* 1 feature map data of a Y+lth line, performing the same operation as the previous last cycle and replacing the M*N feature map data of the first input channel as a whole (Mathematical Concept).
Performing a convolution calculation is a mathematical concept under the broadest reasonable interpretation of the claim language. Performing various multiply-add calculations are mathematical concepts under the broadest reasonable interpretation of the claim language.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application. The claims recite the additional limitations:
reading M*N feature map data of all input channels and weights of a preset number of output channels, wherein a value of M*N and a value of the preset number are respectively determined by preset Y*Y weights, and M, N and Y are all positive integers; inputting read feature map data and the weights of the preset number of output channels into a multiply-add array of the preset number of output channels for a convolution calculation;…outputting a result of the convolution calculation…inputting, in a first cycle, M* 1 feature map data of a first line of a first input channel and the weights of the preset number of output channels into a calculation array of the preset number of output channels,… inputting, in a second cycle, M* 1 feature map data of a second line and the weights of the preset number of output channels into the calculation array of the preset number of output channels,… M*N feature map data of the first input channel is replaced as a whole, wherein the reading operation is an operation of reading the M*N feature map data of all the input channels and the weights of the preset number of output channels; inputting, in a Y+2 cycle, M* 1 feature map data of a Y+2 line and the weights of the preset number of output channels into the calculation array of the preset number of output channels,… inputting the M*N feature map data of the preset number of input channels into the calculation array sequentially, and for the feature map data of each of the input channels,… and outputting a calculation result; reading the M*N feature map data of all the input channels sequentially…inputting, in the second cycle, the M* 1 feature map data of the second line of the first input channel and the weights of the preset number of output channels into the calculation array of the preset number of output channels,… inputting, in a third cycle, M* 1 feature map data of a third line, and performing the same operation as a previous last cycle.
Reading, inputting, and outputting data is mere data gathering, which is an insignificant extra-solution activity as discussed in MPEP 2106.05(g). The claims are directed to abstract ideas.
Step 2B: The claims do not include additional elements that are sufficient to
amount to significantly more than the judicial exception. Reading, inputting, and outputting data is mere data gathering which is an insignificant extra-solution activity as discussed in MPEP 2106.05(g). The claims are not patent eligible.
Dependent Claims
Claims 12, 16, and 20
Step 2A Prong 1: The judicial exceptions of claims 11, 15, and 19 are incorporated.
Accordingly, the claims recite abstract ideas.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical
application. The claims recite the following limitations:
wherein reading M*N feature map data of all the input channels and weights of the preset number of output channels comprises: reading the M*N feature map data of all the input channels and saving them in a memory; reading the weights of the preset number of output channels and saving them in the memory.
Reading M*N feature map data and weights of the output channels and saving them into memory is mere data gathering, which is an insignificant extra-solution activity as discussed in MPEP 2106.05(g). The claims are directed to abstract ideas.
Step 2B: The claims do not include additional elements that are sufficient to
amount to significantly more than the judicial exception. Reading M*N feature map data and weights of the output channels and saving them into memory is mere data gathering, which is an insignificant extra-solution activity as discussed in MPEP 2106.05(g). The claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11, 12, 15, 16, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al (Pub. No.: US 20190197083 A1), hereafter Chen, in view of Yang et al (Pub. No.: US 20190087725 A1), hereafter Yang.
Regarding claims 11, 15, and 19, Chen teaches a data processing method and device, comprising steps of reading M*N feature map data of all input channels and weights of a preset number of output channels (“reading the weight values on the corresponding third predetermined number m′ of pointwise convolution channels in the fourth predetermined number n of pointwise convolution kernels correspondingly to the intermediate feature values of the first predetermined number p of points on the third predetermined number m′ of depthwise convolution output channels”, P0111), wherein a value of M*N and a value of the preset number are respectively determined by preset Y*Y weights (predetermined weight values for M and N are used in the convolution calculations, P0113), …: inputting read feature map data and the weights of the preset number of output channels into a multiply-add array of the preset number of output channels for a convolution calculation (“For depthwise convolution, firstly calculating multiplication and accumulation results of p(p<=H*W) points and m(m<=M) channels, the accumulation here being the accumulation performed in the direction of the length and width of the convolution kernel, as R and S shown in FIG. 6, and here p*m multiply-accumulate (MAC) units being shared, and p*m multiply-accumulate results being obtained.”, P0130); wherein a mode of the convolution calculation comprises: not performing the convolution calculation in a case that the feature map data or the weights of the output channels are zero (“since the previous accumulation calculation results of this p points do not exist or are 0, therefore, there is no need to perform accumulation, or say, the accumulation calculation results are the multiplication calculation results themselves”, P0118), and selecting one from a plurality of same values to perform the convolution calculation in a case that there are a plurality of feature map data with the same values (“performing pointwise convolution calculation on the intermediate feature values of the first predetermined number p of points on the third predetermined number m′ of depthwise convolution output channels with the weight values on the corresponding third predetermined number m′ of pointwise convolution channels in the fourth predetermined number n of pointwise convolution kernels”, P0113); outputting a result of the convolution calculation (“… to obtain the current pointwise convolution partial sums of the first predetermined number p of points on the fourth predetermined number n of pointwise convolution output channels corresponding to the fourth predetermined number n of pointwise convolution kernels”, P0113) wherein inputting read feature map data and the weights of the preset number of output channels into the multiply-add array of the preset number of output channels for a convolution calculation comprises: inputting, in a first cycle, M* 1 feature map data of a first line of a first input channel and the weights of the preset number of output channels into a calculation array of the preset number of output channels (calculation includes multiplier-adder units, output channels and input feature maps, P0132, P0144), and using a first group of Z*1 multiply-add units to perform a multiply-add calculation and then obtaining Z calculation results, wherein Z is determined by the preset Y*Y weights (“firstly calculating multiplication and accumulation results of p(p<=H*W) points and m(m<=M) channels, the accumulation here being the accumulation performed in the direction of the length and width of the convolution kernel, as R and S shown in FIG. 6, and here p*m multiply-accumulate (MAC) units being shared, and p*m multiply-accumulate results being obtained.”, P0130); inputting, in a first cycle, M* 1 feature map data of a first line of a first input channel and the weights of the preset number of output channels into a calculation array of the preset number of output channels, and using a first group of Z*1 multiply-add units to perform a multiply-add calculation and then obtaining Z calculation results, wherein Z is determined by the preset Y*Y weights; inputting, in a second cycle, M* 1 feature map data of a second line and the weights of the preset number of output channels into the calculation array of the preset number of output channels, and using a second group of Z*1 multiply-add units to perform the multiply-add calculation and then obtaining Z calculation results, until after a multiply-add calculation of a Y-th cycle is completed once a reading operation is performed, M*N feature map data of the first input channel is replaced as a whole, wherein the reading operation is an operation of reading the M*N feature map data of all the input channels and the weights of the preset number of output channels; inputting, in a Y+2 cycle, M* 1 feature map data of a Y+2 line and the weights of the preset number of output channels into the calculation array of the preset number of output channels, and using a Y+2 group of Z* 1 multiply-add units to perform the multiply-add calculation and then obtaining Z calculation results, until after a Y*Yth cycle of the reading operation is performed, all multiply-add calculations of Z data in the first line of the first input channel are completed; inputting the M*N feature map data of the preset number of input channels into the calculation array sequentially, and for the feature map data of each of the input channels, performing the multiply-add calculation for the M* 1 feature map data of each line sequentially, until after Y*Y* preset number of cycles once the reading operation is performed, all the multiply-add calculations of Z data in the first line are completed, and outputting a calculation result; reading the M*N feature map data of all the input channels sequentially, and repeating a same operation as completing all the multiply-add calculation of Z data in the first line, until the feature map data of all the input channels are calculated (“repeating above operations (i.e. substeps 1-4), reading the intermediate feature values of the first predetermined number p of points on a next third predetermined number m′ of depthwise convolution output channels, reading weight values on a corresponding next third predetermined number m′ of pointwise convolution channels in a fourth predetermined number n of pointwise convolution kernels, and correspondingly performing subsequent operations until the pointwise convolution calculations and accumulation calculations are completed on the intermediate feature values of the first predetermined number p of points on all depthwise convolution output channels, the final accumulation calculation results of the first predetermined number p of points being the output feature values of the first predetermined number p of points on the fourth predetermined number n of pointwise convolution output channels corresponding to the fourth predetermined number n of pointwise convolution kernels”, P0120) wherein inputting, in the second cycle, the M* 1 feature map data of the second line of the first input channel and the weights of the preset number of output channels into the calculation array of the preset number of output channels, using the second group of Z*1 multiply-add units to perform the multiply-add calculation and then obtaining an intermediate result of Z points of a next line, shifting the feature map data of the first line to a left side and making all multiply-add calculations in a same output point are implemented in a same multiply-add unit (the steps of reading the intermediate feature values of predetermined input and output channels, inputting them into pointwise convolution calculations and accumulation calculations, and performing the pointwise convolution calculations and accumulation calculations to obtain multiply-accumulate (MAC) units may be repeated multiple times, P0120, P0130. Bit shifting may be used to obtain low precision or high precision results, making the units the same, P0134); inputting, in a third cycle, M* 1 feature map data of a third line, and performing the same operation as a previous last cycle; until after the multiply-add calculation of the Y-th cycle is completed once the reading operation is performed, in a Y+lth cycle, inputting M* 1 feature map data of a Y+lth line, performing the same operation as the previous last cycle and replacing the M*N feature map data of the first input channel as a whole (steps of performing multiplication accumulation operations are repeated with each input feature map and output channels, until the pointwise convolution calculations and accumulation calculations are completed on the intermediate feature values of the predetermined convolution output channels, the final accumulation calculation results of the first predetermined output feature values of the predetermined convolution output channels corresponding to the fourth predetermined convolution kernels, P0120).
Chen does not appear to explicitly teach M, N and Y are all positive integers.
Yang teaches M, N and Y are all positive integers (The number of output channels, weights, and feature maps are positive integers, P0090-P0091).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Chen and Yang before them, to include Yang’s specific teaching of the number of channels, weights, and feature maps being positive integers in Chen’s system of Convolution Calculation in Neural Networks. One would have been motivated to make such a combination of the number of channels, weights, and feature maps being positive integers (see Yang P0090-P0091) and performing convolution calculations using rows, columns, the depth of the network, and the width of the neural network, with no negative coefficients, to obtain output results of the N channels (see Chen P0036-P0039).
Regarding claims 12, 16, and 20, Chen in view of Yang teaches the limitations of claims 11, 15, and 19 as outlined above. Chen further teaches wherein reading M*N feature map data of all the input channels and weights of the preset number of output channels comprises: reading the M*N feature map data of all the input channels and saving them in a memory (“reading an input feature map, depthwise convolution kernels and pointwise convolution kernels from a dynamitic random access memory (DRAM);”, P0007); reading the weights of the preset number of output channels and saving them in the memory (“storing the output feature values of a first predetermined number p of points on all pointwise convolution output channels into an on-chip memory”, P0007).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHAN MOUNDI whose telephone number is (703)756-1547. The examiner can normally be reached 8:30 A.M. - 5 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached on (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.M./ Examiner, Art Unit 2141
/KIEU D VU/ Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights