DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/11/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
The broadest reasonable interpretation of a method (or process) claim having contingent limitations requires only those steps that must be performed and does not include steps that are not required to be performed because the condition(s) precedent are not met. For example, assume a method claim requires step A if a first condition happens and step B if a second condition happens. If the claimed invention may be practiced without either the first or second condition happening, then neither step A or B is required by the broadest reasonable interpretation of the claim. If the claimed invention requires the first condition to occur, then the broadest reasonable interpretation of the claim requires step A. If the claimed invention requires both the first and second conditions to occur, then the broadest reasonable interpretation of the claim requires both steps A and B. See MPEP 2111.04 II for more information.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3, 5, 7, 9-12 and 14-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Under Step 1, Claims 1, 3, 5, 7, 9, 16 and 21 recite a series of steps and, therefore, is a process. Claims 10-12, 14-15, 20 and 22 recite a system and, therefore, is a machine. Claims 17 and 19 recite a non-transitory computer readable storage medium and, therefore, is an article of manufacture. Claim 18 recites a non-transitory computer readable store medium and, therefore, is an article of manufacture.
Under Step 2A prong 1, claim 1 recites
A method of implementing in hardware a dilated convolution, comprising convolving a kernel with input data, using a given dilation rate, the method comprising:
mapping the dilated convolution to a plurality of constituent convolutions;
evaluating the plurality of constituent convolutions using the hardware, to produce a plurality of partial results; and
combining the plurality of partial results to produce a result of the dilated convolution,
wherein the result of the dilated convolution is provided as an output of a neural network comprising the dilated convolution or as an input to a subsequent layer in the neural network,
wherein the mapping comprises:
splitting the kernel into a plurality of constituent kernels, each constituent kernel comprising a single coefficient per input channel, each constituent kernel to be applied in a respective one of the plurality of constituent convolutions, wherein combining the plurality of partial results comprises summing the plurality of partial results to produce the result of the dilated convolution.
The above underlined limitations of performing dilated convolution amounts to processing mathematical relationships/calculations and falls within the “Mathematical Concepts” and “Mental Processes” grouping of abstract ideas. The steps of “mapping”, “evaluating” and “combining” is a process that under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, other than reciting “hardware”, nothing in the claim element precludes the steps from practically being performed in the human mind. For example, but for the “hardware” language, the claim encompasses manually splitting a 3x3 kernel into nine 1x1 kernels; multiplying each nine 1x1 kernels with its corresponding input data elements to produce a plurality of partial results and adding the plurality of partial results to obtain a result of the dilated convolution as described in paragraphs [0073-0074] and Fig. 3 using pen and paper. Accordingly, the claim is directed to recite an abstract idea.
Under step 2A prong 2, the claim recites the following additional elements: hardware and wherein the result of the dilated convolution is provided as an output of a neural network comprising the dilated convolution or as an input to a subsequent layer in the neural network. However, the additional element of “hardware” is recited at a high-level of generality (i.e., as a generic hardware for implementing a dilated convolution operation) such that it amounts to no more than merely reciting the words “apply it” (or an equivalent) with the judicial exception or merely as a tool to implement the abstract idea. See MPEP 2106.05(f) for more information. The additional element of “wherein the result of the dilated convolution is provided as an output of a neural network comprising the dilated convolution or as an input to a subsequent layer in the neural network” is merely generally linking the use of the judicial exception to a particular technological environment or field of use of a neural network by limiting the result of the dilated convolution as an output of a neural network or as an input to a subsequent layer of the neural network. See MPEP 2106.05(h) for more information. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claim is not integrated into a practical application.
Under step 2B, claim 1 does not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “hardware” is recited at a high-level of generality (i.e., as a generic hardware for implementing a dilated convolution operation) such that it amounts to no more than merely reciting the words “apply it” (or an equivalent) with the judicial exception or merely as a tool to implement the abstract idea. See MPEP 2106.05(f) for more information. The additional element of “wherein the result of the dilated convolution is provided as an output of a neural network comprising the dilated convolution or as an input to a subsequent layer in the neural network” is merely generally linking the use of the judicial exception to a particular technological environment or field of use of a neural network by limiting the result of the dilated convolution as an output of a neural network or as an input to a subsequent layer of the neural network. See MPEP 2106.05(h) for more information. The claim does not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claim does not amount to significantly more than the abstract idea.
Under step 2A prong 1, claims 3, 5, 7, 9, 17 and 21 recite the same abstract idea as claim 1 by reason of dependence. Further, claim 21 recites further details of the abstract idea of mapping the dilated convolution “wherein the mapping comprises: selecting among a number of potential candidate mappings, each candidate mapping being associated with a respective plurality of constituent convolutions; and implementing the dilated convolution based on the selected candidate mapping, comprising: evaluating the plurality of constituent convolutions of the selected candidate mapping to produce the plurality of partial results; and combining the partial results to produce the result of the dilated convolution”; claim 3 recites further details of the abstract idea of the mapping “wherein the selecting is based at least in part on the given dilation rate, wherein, responsive to the given dilation rate being above a predetermined threshold, the selected candidate mapping comprises splitting the kernel into a plurality of constituent kernels, each constituent kernel comprising a single coefficient per input channel, each constituent kernel to be applied in a respective one of the plurality of constituent convolutions, wherein combining the plurality of partial results comprises summing the plurality of partial results to produce the result of the dilated convolution”; claim 5 recites further details of the abstract idea of the mapping “wherein the selecting is based at least in part on the given dilation rate, wherein, if the given dilation rate is below a predetermined threshold, the selected candidate mapping comprises dividing the input data into a plurality of parts, each part to be subjected to a respective one of the plurality of constituent convolutions; and combining the plurality of partial results comprises interleaving the plurality of partial results to produce the result of the dilated convolution”; claim 7 recites further details of the abstract idea of the mapping “wherein the selecting is based at least in part on the given dilation rate, and at least in part on a type of the dilated convolution, wherein: if the dilated convolution contains a separate filter for each of a plurality of input channels of the input data, then if the given dilation rate is above a predetermined threshold, the selected candidate mapping comprises dividing the input data into a plurality of parts, each part to be subjected to a respective one of the plurality of constituent convolutions; and combining the plurality of partial results comprises interleaving the plurality of partial results to produce the result of the dilated convolution”; claim 9 recites further details of the abstract idea of the mapping “wherein the mapping comprises: defining a set of candidate mappings, each candidate mapping comprising a plurality of constituent convolutions; predicting a performance metric for each candidate mapping; selecting a candidate mapping with a highest predicted performance from the set of candidate mappings; and implementing the dilated convolution based on the selected candidate mapping, comprising: evaluating the plurality of constituent convolutions of the selected candidate mapping using the hardware, to produce the plurality of partial results; and combining the plurality of partial results to produce the result of the dilated convolution” which falls within the “Mathematical Concepts” and “Mental Processes” grouping of abstract ideas. In particular claims 3, 5, 7, 9 and 21 do not include additional elements that would require further analysis under step 2A prong 2 and step 2B. Accordingly, the claims are directed to recite an abstract idea.
Under step 2A prong 2, claim 17 recites the following additional elements: a non-transitory computer readable storage medium having stored thereon computer readable code. However, the additional element of “a non-transitory computer readable storage medium” is recited at a high-level of generality (i.e., as a generic computer component for storing computer readable code) such that it amounts to no more than mere instructions using a generic computer component or merely as a tool to implement the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See 2106.05(f) for more information. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claim is not integrated into a practical application.
Under step 2B, claim 17 does not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a non-transitory computer readable storage medium” is recited at a high-level of generality (i.e., as a generic computer component for storing computer readable code) such that it amounts to no more than mere instructions using a generic computer component or merely as a tool to implement the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See 2106.05(f) for more information. The claim does not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claim does not amount to significantly more than the abstract idea.
Under Step 2A prong 1, claim 10 recites
A data processing system for implementing a dilated convolution comprising convolving a kernel with input data, using a given dilation rate, the system comprising:
a controller, configured to map the dilated convolution to a plurality of constituent convolutions; and
a hardware accelerator, wherein the hardware accelerator is configured to:
evaluate the plurality of constituent convolutions to produce a respective plurality of partial results, and
combine the plurality of partial results to produce a result of the dilated convolution,
wherein the controller is configured to map the dilated convolution to the plurality of constituent convolutions by splitting the kernel into a plurality of constituent kernels, each constituent kernel comprising a single coefficient per input channel, each constituent kernel to be applied in a respective one of the plurality of constituent convolutions, and wherein the hardware accelerator is configured to combine the plurality of partial results by summing the plurality of partial results to produce the result of the dilated convolution.
The above underlined limitations of performing dilated convolution amounts to processing mathematical relationships/calculations and falls within the “Mathematical Concepts” and “Mental Processes” grouping of abstract ideas. The steps of “map”, “evaluate” and “combine” is a process that under its broadest reasonable interpretation, covers performance of the limitation in the mind. That is, other than reciting “a controller” and “a hardware accelerator”, nothing in the claim element precludes the step from practically being performed in the human mind. For example, but for the “a controller” and “a hardware accelerator”, language, the claim encompasses manually splitting a 3x3 kernel into nine 1x1 kernels; multiplying each nine 1x1 kernels with its corresponding input data elements to produce a plurality of partial results and adding the plurality of partial results to obtain a result of the dilated convolution as described in paragraphs [0073-0074] and Fig. 3 using pen and paper. Accordingly, the claim is directed to recite an abstract idea.
Under step 2A prong 2, the claim recites the following additional elements: a controller and a hardware accelerator. However, the additional elements of “a controller” and “a hardware accelerator”, are recited at a high-level of generality (i.e., as a generic computer component for mapping a dilated convolution to a plurality of smaller convolution operations; and as a generic hardware for implementing the dilated convolution) such that they amount to no more than merely reciting the words “apply it” (or an equivalent) with the judicial exception or merely as tools to implement the abstract idea. See MPEP 2106.05(f) for more information. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claim is not integrated into a practical application.
Under step 2B, claim 10 does not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a controller” and “a hardware accelerator”, are recited at a high-level of generality (i.e., as a generic computer component for mapping a dilated convolution to a plurality of smaller convolution operations; and as a generic hardware for implementing the dilated convolution) such that they amount to no more than merely reciting the words “apply it” (or an equivalent) with the judicial exception or merely as tools to implement the abstract idea. See MPEP 2106.05(f) for more information. The claim does not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claim does not amount to significantly more than the abstract idea.
Under step 2A prong 1, claims 11-12, 14-16, 18-20 and 22 recite the same abstract idea as claim 10 by reason of dependence. Further, claim 11 recites further details of the abstract idea of evaluating the plurality of constituent convolutions “multiply a set of one or more input data values and a set of one or more weights, in each cycle of a plurality of hardware cycles to evaluate the plurality of constituent convolutions”; claim 12 recites further details of the abstract idea of evaluating the plurality of constituent convolutions “multiply a weight by an input data value; and sum outputs”; claim 22 recites further details of the abstract idea of the mapping “select among a number of potential candidate mappings, each candidate mapping being associated with a respective plurality of constituent convolutions; and implement the dilated convolution based on the selected candidate mapping to: evaluate the plurality of constituent convolutions of the selected candidate mapping, to produce the plurality of partial results; and combine the partial results to produce the result of the dilated convolution”; claim 14 recites further details of the abstract idea of the mapping “select among the potential candidate mappings based on one or more of: a size of the kernel; the given dilation rate; and a type of the dilated convolution”; claim 15 recites further details of the abstract idea of the mapping “define a set of candidate mappings, each candidate mapping comprising a plurality of constituent convolutions; predict a performance metric for each candidate mapping; select a candidate mapping with a highest predicted performance from the set of candidate mappings; and implement the dilated convolution based on the selected candidate mapping to: evaluate the plurality of constituent convolutions of the selected candidate mapping, to produce the plurality of partial results; and combine the plurality of partial results to produce the result of the dilated convolution” which falls within the “Mathematical Concepts” and “Mental Processes” grouping of abstract ideas. In particular claims 14, 15 and 22 do not include additional elements that would require further analysis under step 2A prong 2 and step 2B. Accordingly, the claims are directed to recite an abstract idea.
Under step 2A prong 2, claim 11 recites the following additional elements: a plurality of convolution engines; claim 12 recites the following additional elements: a plurality of elements of multiply logic; and a plurality of elements of addition logic; claim 16 recites the following additional elements: a method of manufacturing, using an integrated circuit manufacturing system, the data processing system as claimed in claim 10, the method comprising: processing, using a layout processing system, a computer readable description of the data processing system so as to generate a circuit layout description of an integrated circuit embodying the data processing system; and manufacturing, using an integrated circuit generation system, the data processing system according to the circuit layout description; claim 18 recites the following additional elements: a non-transitory computer readable store medium having stored thereon an integrated circuit definition dataset that, when processed in an integrated circuit manufacturing system, configures the integrated circuit manufacturing system to manufacture the data processing system as claimed in claim 10; claim 19 recites the following additional elements: a non-transitory computer readable storage medium having stored thereon a computer readable description of the data processing system as claimed in claim 10 that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the data processing system; claim 20 recites the following additional elements: an integrated circuit manufacturing system comprising: a non-transitory computer readable storage medium having stored thereon a computer readable description of the data processing system as claimed in claim 10;a layout processing system configured to process the computer readable description so as to generate a circuit layout description of an integrated circuit embodying the data processing system; and an integrated circuit generation system configured to manufacture the data processing system according to the circuit layout description. However, the additional elements of “a plurality of convolution engines” in claim 11; “a plurality of elements of multiply logic” and “a plurality of elements of addition logic” in claim 12; “an integrated circuit manufacturing system”, “a layout processing system”, “an integrated circuit generation system” in claim 16; “a non-transitory computer readable store medium” and “an integrated circuit manufacturing system” in claim 18; “a non-transitory computer readable storage medium” and “an integrated circuit manufacturing system” in claim 19; and “an integrated circuit manufacturing system”, “a non-transitory computer readable storage medium”, “a layout processing system” and “an integrated circuit generation system” in claim 20 are recited at a high-level of generality (i.e., as generic convolution engines for performing convolution; as generic multiply logic elements for performing multiplication; as generic addition logic elements for performing addition; as a generic integrated circuit manufacturing system for manufacturing an integrated circuit; as a generic layout processing system for generating a circuit layout; as a generic integrated circuit generation system for generating an integrated circuit; and as a generic storage medium for storing data) such that they amount to no more than mere instructions using a generic computer component or merely as tools to implement the abstract idea or merely reciting the words "apply it" (or an equivalent) with the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See MPEP 2106.05(f) for more information. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claims are not integrated into a practical application.
Under step 2B, claims 11-12, 16 and 18-20 do not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a plurality of convolution engines” in claim 11; “a plurality of elements of multiply logic” and “a plurality of elements of addition logic” in claim 12; “an integrated circuit manufacturing system”, “a layout processing system”, “an integrated circuit generation system” in claim 16; “a non-transitory computer readable store medium” and “an integrated circuit manufacturing system” in claim 18; “a non-transitory computer readable storage medium” and “an integrated circuit manufacturing system” in claim 19; and “an integrated circuit manufacturing system”, “a non-transitory computer readable storage medium”, “a layout processing system” and “an integrated circuit generation system” in claim 20 are recited at a high-level of generality (i.e., as generic convolution engines for performing convolution; as generic multiply logic elements for performing multiplication; as generic addition logic elements for performing addition; as a generic integrated circuit manufacturing system for manufacturing an integrated circuit; as a generic layout processing system for generating a circuit layout; as a generic integrated circuit generation system for generating an integrated circuit; and as a generic storage medium for storing data) such that they amount to no more than mere instructions using a generic computer component or merely as tools to implement the abstract idea or merely reciting the words "apply it" (or an equivalent) with the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See MPEP 2106.05(f) for more information. The claims do not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claims do not amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5, 7, 10-12, 14, 17 and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Yadav et al. (US 20200218936 A1), hereinafter Yadav, in view of Huynh et al. (US 20200410036 A1), hereinafter Huynh.
Regarding claim 1, Yadav teaches a method of implementing in hardware a dilated convolution, comprising convolving a kernel with input data, using a given dilation rate, the method comprising (Yadav Figs. 2A, 3, 4H and paragraphs [0070] “FIG. 3 illustrates a flowchart illustrating method of convolution in a neural network with variable dilation rate”; paragraphs [0068, 0085]; kernel – kernel; input data – input image; hardware – computing system/convolution module):
mapping the dilated convolution to a plurality of constituent convolutions (Yadav Figs. 3 steps 302-304 and paragraphs [0073, 0076-0081]; Fig. 4H and paragraph [0082]; plurality of constituent convolutions - convolution of each of the one or more disintegrated kernels (403) and the one or more blocks (215) of the input image (101)”);
evaluating the plurality of constituent convolutions using the hardware, to produce a plurality of partial results (Yadav Fig. 4H and paragraphs [0082-0085] “determining the output image (103) includes generating the one or more matrices by multiplying one or more kernel values (105) of each of the one or more disintegrated kernels (403) with a plurality of pixel values (104) corresponding to each block of the one or more blocks (215) of the input image (101)”; plurality of partial results - one or more matrices); and
combining the plurality of partial results to produce a result of the dilated convolution (Yadav Fig. 4H and paragraphs [0082-0085] “adding the one or more matrices to determine the output image (103)”; result – output image),
wherein the result of the dilated convolution is provided as an output (Yadav Fig. 3 step 305 and paragraphs [0082-0083]),
wherein the mapping comprises:
splitting the kernel into a plurality of constituent kernels, each constituent kernel comprising a single coefficient per input channel, each constituent kernel to be applied in a respective one of the plurality of constituent convolutions, wherein combining the plurality of partial results comprises summing the plurality of partial results to produce the result of the dilated convolution (Yadav Fig. 4H and paragraph [0083] “consider an input image (101) with a size of 20x24 and one or more disintegrated kernels (403) of size 1x1 as shown in FIG. 4H. The one or more blocks (215) of the input image (101) is multiplied with the corresponding one or more disintegrated kernels (403) to generate the one or more matrices as shown in FIG. 4H. The one or more matrices are added to determine the output image (103) as shown in FIG. 4H”; plurality of constituent kernels - one or more disintegrated kernels (403) of size 1x1).
Yadav does not explicitly teach wherein the result of the dilated convolution is provided as an output of a neural network comprising the dilated convolution or as an input to a subsequent layer in the neural network.
However, on the same the same field of endeavor, Huynh discloses a result of a convolution is provided as an output of a neural network comprising the convolution or as an input to a subsequent layer in the neural network (Huynh Fig. 2A and paragraphs [0032-0033] “each processing node of layer 209 may generate the intermediate output based on the scaling of pixel values from a group of processing nodes of layers 207 … Rectangular blocks of the convolution outputs can be further grouped, and convolution operations can be performed at layer 211 between the groups of convolution outputs and another set of filter weights to generate another set of convolution outputs”).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Yadav using Huynh and perform the dilated convolution as part of processing a layer of a neural network such that the result of the dilated convolution is provided as an output of the neural network comprising the dilated convolution or as an input to a subsequent layer of the neural network in order to implement a multi-layer neural network such as a deep neural network (DNN) or a convolutional neural network (CNN) (Yadav paragraph [0028]).
Therefore, the combination of Yadav as modified in view of Huynh teaches wherein the result of the dilated convolution is provided as an output of a neural network comprising the dilated convolution or as an input to a subsequent layer in the neural network.
Regarding claim 21, Yadav as modified in view of Huynh teaches all the limitations of claim 1 as stated above. Further, Yadav as modified in view of Huynh teaches wherein the mapping comprises:
selecting among a number of potential candidate mappings, each candidate mapping being associated with a respective plurality of constituent convolutions (Yadav Fig. 4D and paragraph [0076]); and
implementing the dilated convolution based on the selected candidate mapping, comprising:
evaluating the plurality of constituent convolutions of the selected candidate mapping, using the hardware, to produce the plurality of partial results (Yadav Fig. 4H and paragraphs [0082-000083]); and
combining the partial results to produce the result of the dilated convolution (Yadav Fig. 4H and paragraphs [0082-000083]).
Regarding claim 3, Yadav as modified in view of Huynh teaches all the limitations of claim 21 as stated above. Further, Yadav as modified in view of Huynh teaches wherein the selecting is based at least in part on the given dilation rate (Yadav Fig. 4D and paragraph [0076]) “Based on the value of "n" and the dilation rate, the corresponding size of the one or more disintegrated kernels is determined from the table (402) as shown in FIG. 4D. For example, if the dilation rate is 14, the value of "n" is selected as "1" and the size of the one or more disintegrated kernels is 3x7”),
wherein, responsive to the given dilation rate being above a predetermined threshold, the selected candidate mapping comprises splitting the kernel into a plurality of constituent kernels, each constituent kernel comprising a single coefficient per input channel, each constituent kernel to be applied in a respective one of the plurality of constituent convolutions, wherein combining the plurality of partial results comprises summing the plurality of partial results to produce the result of the dilated convolution (This is a contingent limitation that is not required to be performed if the given dilation rate is equal or below the predetermined threshold. See claim interpretation section above for more information).
Regarding claim 5, Yadav as modified in view of Huynh teaches all the limitations of claim 21 as stated above. Further, Yadav as modified in view of Huynh teaches wherein the selecting is based at least in part on the given dilation rate (Yadav Fig. 4D and paragraph [0076]) “Based on the value of "n" and the dilation rate, the corresponding size of the one or more disintegrated kernels is determined from the table (402) as shown in FIG. 4D. For example, if the dilation rate is 14, the value of "n" is selected as "1" and the size of the one or more disintegrated kernels is 3x7”),
wherein, if the given dilation rate is below a predetermined threshold, the selected candidate mapping comprises dividing the input data into a plurality of parts, each part to be subjected to a respective one of the plurality of constituent convolutions; and combining the plurality of partial results comprises interleaving the plurality of partial results to produce the result of the dilated convolution (This is a contingent limitation that is not required to be performed if the given dilation rate is equal or above the predetermined threshold. See claim interpretation section above for more information).
Regarding claim 7, Yadav as modified in view of Huynh teaches all the limitations of claim 21 as stated above. Further, Yadav as modified in view of Huynh teaches wherein the selecting is based at least in part on the given dilation rate, and at least in part on a type of the dilated convolution (Yadav Fig. 4D and paragraph [0076]) “Based on the value of "n" and the dilation rate, the corresponding size of the one or more disintegrated kernels is determined from the table (402) as shown in FIG. 4D. For example, if the dilation rate is 14, the value of "n" is selected as "1" and the size of the one or more disintegrated kernels is 3x7”),
wherein: if the dilated convolution contains a separate filter for each of a plurality of input channels of the input data, then if the given dilation rate is above a predetermined threshold, the selected candidate mapping comprises dividing the input data into a plurality of parts, each part to be subjected to a respective one of the plurality of constituent convolutions; and combining the plurality of partial results comprises interleaving the plurality of partial results to produce the result of the dilated convolution (These are contingent limitations that are not required to be performed if the dilated convolution does not contain a separate filter for each of a plurality of input channels of the input data and/or given dilation rate is equal or below the predetermined threshold. See claim interpretation section above for more information).
Regarding claim 10, Yadav teaches
a data processing system for implementing a dilated convolution comprising convolving a kernel with input data, using a given dilation rate, the system comprising (Yadav Figs 2A, 3, 4H and paragraphs [0070, 0082-0083] system – computing system; kernel – kernel; input data – input images):
(Yadav Fig. 2A, 3 steps 302-304 and paragraphs [0073, 0076-0081]; Fig. 4H and paragraph [0082]; plurality of constituent convolutions - convolution of each of the one or more disintegrated kernels (403) and the one or more blocks (215) of the input image (101)”); and
a hardware accelerator, wherein the hardware accelerator is configured to (Yadav Figs. 2A. and paragraph [0068] hardware accelerator – convolution module 213):
evaluate the plurality of constituent convolutions to produce a respective plurality of partial results (Yadav Fig. 4H and paragraphs [0082-0085] “determining the output image (103) includes generating the one or more matrices by multiplying one or more kernel values (105) of each of the one or more disintegrated kernels (403) with a plurality of pixel values (104) corresponding to each block of the one or more blocks (215) of the input image (101)”; plurality of partial results - one or more matrices), and
combine the plurality of partial results to produce a result of the dilated convolution (Yadav Fig. 4H and paragraphs [0082-0085] “adding the one or more matrices to determine the output image (103)”; result – output image),
results to produce the result of the dilated convolution (Yadav Fig. 4H and paragraph [0083] “consider an input image (101) with a size of 20x24 and one or more disintegrated kernels (403) of size 1x1 as shown in FIG. 4H. The one or more blocks (215) of the input image (101) is multiplied with the corresponding one or more disintegrated kernels (403) to generate the one or more matrices as shown in FIG. 4H. The one or more matrices are added to determine the output image (103) as shown in FIG. 4H”; plurality of constituent kernels - one or more disintegrated kernels (403) of size 1x1).
Yadav does not explicitly teach a controller.
However, on the same field of endeavor, Huynh discloses a computing system comprising a controller and an array of processing engines (Huynh Fig. 4A and paragraph [0048] controller – controller 402).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Yadav using Huynh and configure the system to include a controller for mapping the dilated convolution to the plurality of constituent convolutions evaluated by the convolution module in order to control the overall operations of the computing system including the computations on the kernel and input data (Huynh paragraphs [0048, 0064]).
Therefore, the combination of Yadav as modified in view of Huynh teaches a controller, configured to map the dilated convolution to a plurality of constituent convolutions; wherein the controller is configured to map the dilated convolution to the plurality of constituent convolutions by splitting the kernel into a plurality of constituent kernels, each constituent kernel comprising a single coefficient per input channel, each constituent kernel to be applied in a respective one of the plurality of constituent convolutions, and wherein the hardware accelerator is configured to combine the plurality of partial results by summing the plurality of partial results to produce the result of the dilated convolution.
Regarding claim 11, Yadav as modified in view of Huynh teaches all the limitations of claim 10 as stated above.
Yadav does not explicitly teach wherein the hardware accelerator comprises a plurality of convolution engines, each configured to multiply a set of one or more input data values and a set of one or more weights, in each cycle of a plurality of hardware cycles, wherein the plurality of convolution engines is configured to evaluate the plurality of constituent convolutions.
However, on the same field of endeavor, Huynh discloses a hardware accelerator comprises a plurality of convolution engines each configured to multiply a set of one or more input data values and a set of one or more weights, in each cycle of a plurality of hardware cycles, wherein the plurality of convolution engines is configured to evaluate the plurality of constituent convolutions (Huynh Figs. 4A-4C and paragraphs [0053-0057 and 0066] “The processing engine array 410 includes multiple processing engines 411, arranged in rows and columns, such that results output by one processing engine 411 can be input directly into another processing engine 411 … the processing engine array 410 uses systolic execution, in which data arrives at each processing engine 411 from different directions at regular intervals”; plurality of convolution engines – columns of processing engines 411).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Yadav using Huynh and configure the convolution module as an array of processing engines each including a multiplier-accumulator circuit for performing the multiplication operations between the kernel values and input data streams at regular intervals/clock cycles and to perform accumulation operations to generate the result in order to implement a systolic execution engine that accelerates convolution operations (Huynh paragraph 0053-0054]).
Therefore, the combination of Yadav as modified in view of Huynh teaches wherein the hardware accelerator comprises a plurality of convolution engines, each configured to multiply a set of one or more input data values and a set of one or more weights, in each cycle of a plurality of hardware cycles, wherein the plurality of convolution engines is configured to evaluate the plurality of constituent convolutions.
Regarding claim 12, Yadav as modified in view of Huynh teaches all the limitations of claim 11 as stated above. Further, Yadav as modified in view of Huynh teaches wherein each convolution engine comprises: a plurality of elements of multiply logic, each configured to multiply a weight by an input data value; and a plurality of elements of addition logic, configured to sum outputs of the plurality of elements of multiply logic (Huynh Figs. 4A, 4C and paragraph [0056] plurality of elements of multiply logic – multipliers in each column; plurality of elements of addition logic – accumulator in each column).
Regarding claim 22, Yadav as modified in view of Huynh teaches all the limitations of claim 10 as stated above. Further, Yadav as modified in view of Huynh teaches wherein the controller is configured to:
select among a number of potential candidate mappings, each candidate mapping being associated with a respective plurality of constituent convolutions (Yadav Fig. 4D and paragraph [0076]); and
control the hardware accelerator to implement the dilated convolution based on the selected candidate mapping, wherein the hardware accelerator is configured to:
evaluate the plurality of constituent convolutions of the selected candidate mapping, to produce the plurality of partial results (Yadav Fig. 4H and paragraphs [0082-000083]); and
combine the partial results to produce the result of the dilated convolution (Yadav Fig. 4H and paragraphs [0082-0083]).
Regarding claim 14, Yadav as modified in view of Huynh teaches all the limitations of claim 22 as stated above. Further, Yadav as modified in view of Huynh teaches wherein the controller is configured to select among the potential candidate mappings based on one or more of: a size of the kernel; the given dilation rate; and a type of the dilated convolution (Yadav Fig. 4D and paragraph [0076]) “Based on the value of "n" and the dilation rate, the corresponding size of the one or more disintegrated kernels is determined from the table (402) as shown in FIG. 4D”).
Regarding claim 17, it is directed to a non-transitory computer readable storage medium having stored thereon computer readable code to cause the method of claim 1 to be performed. Claim 1 analysis applies equally to claim 17. Further, Yadav as modified in view of Huynh teaches a non-transitory computer readable storage medium (Yadav paragraph [0098]).
Claims 9 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Yadav in view of Huynh as applied to claims 1 and 10 above, and further in view of Chen et al. (NPL – “Adaptive Fractional Dilated Convolution Network for Image Aesthetics Assessment”), hereinafter Chen.
Regarding claim 15, Yadav as modified in view of Huynh teaches all the limitations of claim 10 as stated above. Further, Yadav as modified in view of Huynh teaches
wherein the controller is configured to: define a set of candidate mappings, each candidate mapping comprising a plurality of constituent convolutions (Yadav Fig. 4 and paragraph [0076]);
;
; and
control the hardware accelerator to implement the dilated convolution based on the selected candidate mapping, wherein the hardware accelerator is configured to (Yadav Figs. 3 and 4H and paragraphs [0082-0083]):
evaluate the plurality of constituent convolutions of the selected candidate mapping, to produce the plurality of partial results (Yadav Figs. 3 and 4H and paragraphs [0082-0083]); and
combine the plurality of partial results to produce the result of the dilated convolution (Yadav Figs. 3 and 4H and paragraphs [0082-0083]).
Yadav does not explicitly teach predict a performance metric for each candidate mapping; and select a candidate mapping with a highest predicted performance from the set of candidate mappings.
However, on the same field of endeavor, Chen discloses predicting a performance metric for a plurality of neural network models each model having a different kernel dilation rate and selecting a model with a highest predicted performance from the set of models (Chen page 14118 left col bottom paragraph “Training efficiency can be optimized by grouping batches, e.g. training with three dilation kernels for the most batches, DilationRates = {(2,1), (1,1), (1,2)}”; page 14119 section 4.1 “we train the model with three dilation kernels, 1 × 2, 1 × 1, 2 × 1 on the grouped images since the aspect ratios for 97.8% training and validation images fall between [1/2, 2]. Then we train the model with seven dilation kernels, 1 × 4, 1 × 3, 1 × 2, 1 × 1, 2 × 1, 3 × 1, 4 × 1, for the remaining 6 epochs and select the best model from the results in the validation dataset”).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Yadav and generalize the teaching of Chen and configure the controller to predict a performance metric for each candidate mapping; and select a candidate mapping with a highest predicted performance from the set of candidate mappings in order to select and implement the best mapping that will reduce the total number of memory fetches (i.e., read cycles) which reduces the time to implement the dilated convolution (Chen page 14119 section 4.1; Yadav paragraph [0085]).
Therefore, the combination of Yadav as modified in view of Huynh teaches predict a performance metric for each candidate mapping; and select a candidate mapping with a highest predicted performance from the set of candidate mappings.
Regarding claim 9, it is directed to a method practiced by the system of claim 15. All steps performed by the method of claim 9 would be practiced by the system of claim 15. Claim 15 analysis applies equally to claim 9.
Claims 16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yadav in view of Huynh as applied to claim 10 above, and further in view of Martin (US 20190147327 A1).
Regarding claim 16, Yadav as modified in view of Huynh teaches all the limitations of claim 10 as stated above. Further, Yadav as modified in view of Huynh teaches the data processing system as claimed in claim 10 (see 103 analysis above).
Yadav does not explicitly teach a method of manufacturing, using an integrated circuit manufacturing system, the data processing system as claimed in claim 10, the method comprising: processing, using a layout processing system, a computer readable description of the data processing system so as to generate a circuit layout description of an integrated circuit embodying the data processing system; and manufacturing, using an integrated circuit generation system, the data processing system according to the circuit layout description.
However, on the same field of endeavor, Martin discloses a method of manufacturing, using an integrated circuit manufacturing system, a data processing system, the method comprising: processing, using a layout processing system, a computer readable description of the data processing system so as to generate a circuit layout description of an integrated circuit embodying the data processing system; and manufacturing, using an integrated circuit generation system, the data processing system according to the circuit layout description (Martin Fig. 9 and paragraph [0266-0268, 0272] integrated circuit manufacturing system - integrated circuit (IC) manufacturing system 1002; data processing system - hardware; layout processing system - layout processing system 1004; computer readable description - integrated circuit definition dataset; integrated circuit generation system - IC generation system 1006).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Yadav in view Huynh using Martin and manufacture the system of claim 10 using the manufacturing method of Martin in order to implement a semiconductor device fabrication process to generate the system for accelerating dilated convolution computations (Martin paragraphs [0268, 0271]).
Therefore, the combination of Yadav as modified in view Huynh and Martin teaches a method of manufacturing, using an integrated circuit manufacturing system, the data processing system as claimed in claim 10, the method comprising: processing, using a layout processing system, a computer readable description of the data processing system so as to generate a circuit layout description of an integrated circuit embodying the data processing system; and manufacturing, using an integrated circuit generation system, the data processing system according to the circuit layout description.
Regarding claim 18, Yadav as modified in view of Huynh teaches all the limitations of claim 10 as stated above. Further, Yadav as modified in view of Huynh teaches a data processing system as claimed in claim 10 (see 103 analysis above).
Yadav does not explicitly teach a non-transitory computer readable store medium having stored thereon an integrated circuit definition dataset that, when processed in an integrated circuit manufacturing system, configures the integrated circuit manufacturing system to manufacture a data processing system as claimed in claim 10.
However, on the same field of endeavor, Martin discloses a non-transitory computer readable store medium having stored thereon an integrated circuit definition dataset that, when processed in an integrated circuit manufacturing system, configures the integrated circuit manufacturing system to manufacture a data processing system (Martin paragraphs [0068-0071]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Yadav in view Huynh using Martin and provide a non-transitory computer readable store medium having stored thereon an integrated circuit definition dataset that, when processed in an integrated circuit manufacturing system, configures the integrated circuit manufacturing system to manufacture the data processing system as claimed in claim 10 in order to implement a semiconductor device fabrication system for generating the system for accelerating dilated convolution computations (Martin paragraphs [0268, 0271]).
Therefore, the combination of Yadav as modified in view Huynh and Martin teaches a non-transitory computer readable store medium having stored thereon an integrated circuit definition dataset that, when processed in an integrated circuit manufacturing system, configures the integrated circuit manufacturing system to manufacture a data processing system as claimed in claim 10.
Regarding claim 19, Yadav as modified in view of Huynh teaches all the limitations of claim 10 as stated above. Further, Yadav as modified in view of Huynh teaches the data processing system as claimed in claim 10 (see 103 analysis above).
Yadav does not explicitly teach a non-transitory computer readable storage medium having stored thereon a computer readable description of the data processing system as claimed in claim 10 that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the data processing system.
However, on the same field of endeavor, Martin discloses a non-transitory computer readable storage medium having stored thereon a computer readable description of a data processing system that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the data processing system (Martin paragraphs [0068-0071]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Yadav in view Huynh using Martin and provide a non-transitory computer readable storage medium having stored thereon a computer readable description of a data processing system as claimed in claim 10 that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the data processing system in order to implement a semiconductor device fabrication system for generating the system for accelerating dilated convolution computations (Martin paragraphs [0268, 0271]).
Therefore, the combination of Yadav as modified in view Huynh and Martin teaches a non-transitory computer readable storage medium having stored thereon a computer readable description of the data processing system as claimed in claim 10 that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the data processing system.
Regarding claim 20, Yadav as modified in view of Huynh teaches all the limitations of claim 10 as stated above. Further, Yadav as modified in view of Huynh teaches the data processing system as claimed in claim 10 (see 103 analysis above).
Yadav does not explicitly teach an integrated circuit manufacturing system comprising: a non-transitory computer readable storage medium having stored thereon a computer readable description of the data processing system as claimed in claim 10; a layout processing system configured to process the computer readable description so as to generate a circuit layout description of an integrated circuit embodying the data processing system; and an integrated circuit generation system configured to manufacture the data processing system according to the circuit layout description.
However, on the same field of endeavor, Martin discloses an integrated circuit manufacturing system comprising: a non-transitory computer readable storage medium having stored thereon a computer readable description of a data processing system; a layout processing system configured to process the computer readable description so as to generate a circuit layout description of an integrated circuit embodying the data processing system; and an integrated circuit generation system configured to manufacture the data processing system according to the circuit layout description (Martin Fig. 9 and paragraphs [0068, 0073, 0266-0268, 0272] integrated circuit manufacturing system - integrated circuit (IC) manufacturing system 1002; non-transitory computer readable storage medium - non-transitory computer readable storage medium; data processing system -hardware; layout processing system - layout processing system 1004; computer readable description - integrated circuit definition dataset; integrated circuit generation system - IC generation system 1006).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Yadav in view Huynh using Martin and implement a manufacturing system for manufacturing the system of claim 10 using the manufacturing method of Martin in order to implement a semiconductor device fabrication system for generating the system for accelerating dilated convolution computations (Martin paragraphs [0268, 0271]).
Therefore, the combination of Yadav as modified in view Huynh and Martin teaches an integrated circuit manufacturing system comprising: a non-transitory computer readable storage medium having stored thereon a computer readable description of the data processing system as claimed in claim 10; a layout processing system configured to process the computer readable description so as to generate a circuit layout description of an integrated circuit embodying the data processing system; and an integrated circuit generation system configured to manufacture the data processing system according to the circuit layout description.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Carlo Waje whose telephone number is (571)272-5767. The examiner can normally be reached 9:00-6:00 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Carlo Waje/Examiner, Art Unit 2151 (571)272-5767
Prosecution Insights