DETAILED ACTION
This action is in response to the application filed on 10/23/2023. Claims 1-17 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless – (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 7-9; 10-11, 14-15 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Lau et al. (US 20190392297 A1, as disclosed in the IDS, hereinafter “Lau”)
Regarding Claim 1,
Lau discloses An integrated circuit, including: a computer unit configured for executing a neural network; (Lau [0044]; “a machine learning computing system may be provided that includes an application-specific integrated circuit (ASIC)-based deep learning hardware (DLH) device provided that is designed to accelerate computations for deep learning applications.”
Lau [0086]; “In some embodiments, a matrix processing chip 1620 may be implemented using an integrated circuit, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), and/or any other type of circuitry.”
Lau [0116]; “Output engine 1737 may then be used to compute a result for the particular matrix operation 1701. For example, output engine 1737 may perform the appropriate matrix operation 1701 using the matrix operands generated by slice engine 1736a (e.g., the matrix operands stored in MRBs 1738b and 1738c).
In some embodiments, for example, output engine 1737 may first identify an associated matrix subroutine corresponding to the particular matrix operation, and output engine 1737 may then obtain that matrix subroutine from matrix subroutine memory 1739. Matrix subroutine memory 1739, for example, may be a memory component used to store matrix subroutines that are used by output engine 1737. A matrix subroutine, for example, may be a programmable subroutine for a matrix processor that is designed to perform a particular matrix operation when executed by the matrix processor. For example, a matrix subroutine may include a series of instructions and/or commands, supported by a particular matrix processor, and designed to perform a desired matrix operation when executed by the matrix processor. In some embodiments, for example, a matrix processor may be designed to support a set of instructions and/or commands for performing various fundamental operations. For example, in some embodiments, a matrix processor may support instructions for processing data, performing various arithmetic operations, and/or identifying matrix operands and outputs for the various instructions and operations. In this manner, the fundamental instructions and/or commands supported by the matrix processor can be used to program matrix subroutines for more complex matrix operations, such as distributed matrix multiplication and/or convolution operations, dimension shuffle operations, reshape operations, and so forth.” wherein the output engine’s performance of matrix and convolutional operations for output of the model thus reads on a computer unit executing a neural network)
a first memory configured to store parameters of the neural network to be executed (Lau [0110]; “Memory resource blocks (MRBs) 1738 may be memory components designed to efficiently store and retrieve matrix data. In some embodiments, for example, MRBs 1738 may be memory resource blocks on a particular matrix processing cluster. In those embodiments, for example, MRBs 1738 may be used to store and retrieve matrix data associated with matrix operations performed on the particular cluster”
Lau [0112]; “In the illustrated example, matrix processing engine 1700 is being used to perform convolution related operations, and thus the matrix data is associated with the image(s) and filters involved in those operations”
Lau [0113]; “Read engine 1735 may then store the matrix data retrieved from HBM 1740a in certain MRBs 1738a of its associated cluster. In some embodiments, for example, read engine 1735 may use two MRBs 1738a to store the associated matrix data. For example, read engine 1735 may use one MRB to store matrix data associated with an image, and may use another MRB to store matrix data associated with a filter used for convolution related operations on that image.” wherein the first MRB storing matrix data associated with an image thus reads on storing parameters of the neural network to be executed (since the matrix data associated with the images used to perform convolution related operations implicitly reads on stored parameters of the neural network))
a second memory configured to store data supplied at an input of the computer unit to be executed or generated by the neural network (Lau [0110]; “Memory resource blocks (MRBs) 1738 may be memory components designed to efficiently store and retrieve matrix data. In some embodiments, for example, MRBs 1738 may be memory resource blocks on a particular matrix processing cluster. In those embodiments, for example, MRBs 1738 may be used to store and retrieve matrix data associated with matrix operations performed on the particular cluster”
Lau [0112]; “In the illustrated example, matrix processing engine 1700 is being used to perform convolution related operations, and thus the matrix data is associated with the image(s) and filters involved in those operations”
Lau [0113]; “Read engine 1735 may then store the matrix data retrieved from HBM 1740a in certain MRBs 1738a of its associated cluster. In some embodiments, for example, read engine 1735 may use two MRBs 1738a to store the associated matrix data. For example, read engine 1735 may use one MRB to store matrix data associated with an image, and may use another MRB to store matrix data associated with a filter used for convolution related operations on that image.” wherein the second MRB storing matrix data associated with a filter thus reads on storing parameters of the neural network to be executed (since the matrix data associated with the filters used to perform convolution related operations implicitly reads on stored parameters of the neural network))
a first barrel shifter circuit between an output of the second memory and the input of the computer unit, the first barrel shifter circuit being configured to transmit the data from the output of the second memory to computer unit; (Lau [0124]; “FIG. 18 illustrates an example embodiment of memory 1800 for storing matrices using data shifting. In some embodiments, for example, memory 1800 may be used to provide efficient storage of matrices in a matrix processing system. For example, memory 1800 could be used to implement the memory resource blocks (MRBs) of an example processing cluster.”
Lau [0128]; “Memory 1800 also includes a memory controller 1806. Memory controller 1806 may be used for efficiently storing and retrieving rows and columns of a matrix from memory modules 1801, as described further below. Moreover, in the illustrated embodiment, memory 1800 includes barrel shifters 1804a and 1804b. Barrel shifters 1804 may be used for shifting matrix data in order to provide efficient access to rows and columns of a matrix, as described further below”
Lau [0137]; “After a particular row 1812 or column 1814 of matrix 1810 is read from memory modules M1, M2 and M3, the elements are out-of-order and thus must be shifted back to their original order, which is performed by barrel shifter 1804b. The shifting required when reading a particular row 1812 or column 1814 is simply the opposite of the shifting performed when the row 1812 or column 1814 was originally written. Accordingly, when reading out the rows 1812 of matrix 1810 from memory 1800, the first row is shifted by 0 elements (e.g., no shifting is performed), the second row is shifted by −1 element, the third row is shifted by −2 elements, and so forth. Similarly, when reading out the columns 1814 of matrix 1810 from memory 1800, the first column is shifted by 0 elements (e.g., no shifting is performed), the second column is shifted by −1 element, the third column is shifted by −2 elements, and so forth.”
Lau [Figure 18];
PNG
media_image1.png
259
535
media_image1.png
Greyscale
wherein the first barrel shifter 1804a performed on output from one of the plurality of MRBs transmitting shifted elements to be accessed and read by the computer unit)
a second barrel shifter circuit between an output of the computer unit and the second memory, the second barrel shifter circuit being configured to deliver data generated during the execution of the neural network; (Lau [0124]; “FIG. 18 illustrates an example embodiment of memory 1800 for storing matrices using data shifting. In some embodiments, for example, memory 1800 may be used to provide efficient storage of matrices in a matrix processing system. For example, memory 1800 could be used to implement the memory resource blocks (MRBs) of an example processing cluster.”
Lau [0128]; “Memory 1800 also includes a memory controller 1806. Memory controller 1806 may be used for efficiently storing and retrieving rows and columns of a matrix from memory modules 1801, as described further below. Moreover, in the illustrated embodiment, memory 1800 includes barrel shifters 1804a and 1804b. Barrel shifters 1804 may be used for shifting matrix data in order to provide efficient access to rows and columns of a matrix, as described further below”
Lau [0137]; “After a particular row 1812 or column 1814 of matrix 1810 is read from memory modules M1, M2 and M3, the elements are out-of-order and thus must be shifted back to their original order, which is performed by barrel shifter 1804b. The shifting required when reading a particular row 1812 or column 1814 is simply the opposite of the shifting performed when the row 1812 or column 1814 was originally written. Accordingly, when reading out the rows 1812 of matrix 1810 from memory 1800, the first row is shifted by 0 elements (e.g., no shifting is performed), the second row is shifted by −1 element, the third row is shifted by −2 elements, and so forth. Similarly, when reading out the columns 1814 of matrix 1810 from memory 1800, the first column is shifted by 0 elements (e.g., no shifting is performed), the second column is shifted by −1 element, the third column is shifted by −2 elements, and so forth.”
Lau [Figure 18];
PNG
media_image1.png
259
535
media_image1.png
Greyscale
wherein the second barrel shifter 1804b performed on the out-of-order elements of the second memory MRB output to shift such elements back to their original order in the second MRB thus reads on a second barrel shifter configured to deliver data generated during the execution of the neural network (since the executed and read data being reorganized to their original order still reads on generation of the executed data))
and a control unit configured to control the computer unit, the first and second barrel shifter circuits as well as the accesses to the first memory and to the second memory (Lau [0044]; “These instructions may be processed by the control logic of the DLH to feed the other units (MPU, memory, etc.). These instructions may include data movement (e.g. from off-chip memory into on-chip memory, operands in on-chip memory, and the arithmetic operations). This data may be stored and transferred as tensors in on-chip and off-chip memory, and between the host and the chip.” wherein directing data movement reads on controlled activity of the first and second barrel shifter circuits and general control of the computer unit
Lau [0113]; “For example, read engine 1735 may use one MRB to store matrix data associated with an image, and may use another MRB to store matrix data associated with a filter used for convolution related operations on that image. In some embodiments, read engine 1735 may use the master control CPU (MCC) 1732 on its respective cluster for storing and retrieving data on HBMs 1740 and MRBs 1738.”
Lau [0118]; “In some embodiments, output engine 1737 may use the master control CPU (MCC) 1732 on its respective cluster to retrieve matrix subroutines from matrix subroutine memory 1739, and to specify or supply any remaining information and/or fields for the particular matrix subroutine (e.g., the size and/or location of matrix operands)” wherein the master control CPU’s control on matrix subroutines, operands, and fields thus reads on controlled access to the first and second memories)
Regarding Claim 2,
Lau teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Lau already discloses wherein the computer unit comprises a bank of processing elements configured to parallelize execution of the neural network, and wherein the first barrel shifter circuit is configured to transmit the data from the second memory to the different processing elements (Lau [0111]; “Matrix processing engine 1700 performs matrix operations using read engine 1735, slice engines 1736, and output engine 1737, as described further below. In the illustrated example, matrix processing engine 1700 is performing multiple matrix operations 1701 and 1702 in parallel. For example, as noted above, in some embodiments matrix processing engine 1700 may be implemented on a particular matrix processing cluster, and the particular matrix processing cluster may include multiple MPUs 1734. In the illustrated example, matrix processing engine 1700 is implemented on a cluster with two MPUs 1734a-b. Accordingly, matrix processing engine 1700 can perform two matrix operations 1701 and 1702 in parallel using the respective MPUs 1734” wherein the plurality of MPUs read on a bank of processing elements configured to parallelize execution; wherein the first barrel shifter’s matrix operations performed by such MPUs thus reads on the first barrel shifter circuit transmitting data from the second memory MRB to its respective parallel MPU)
Regarding Claim 7,
Lau teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Lau already discloses wherein the second memory is configured to store data matrices supplied at the input of the computer unit to be executed or generated by the computer unit (Lau [0113]; “Read engine 1735 may then store the matrix data retrieved from HBM 1740a in certain MRBs 1738a of its associated cluster. In some embodiments, for example, read engine 1735 may use two MRBs 1738a to store the associated matrix data. For example, read engine 1735 may use one MRB to store matrix data associated with an image, and may use another MRB to store matrix data associated with a filter used for convolution related operations on that image.”)
wherein each data matrix has several data channels, the data of each data matrix being grouped together in the second memory in at least one data group, the data groups being stored in different banks of the second memory, the data of each data group configured for processing in parallel by the different processing elements of the computer unit (Lau [0067]; “An MRB (e.g., 830a-n) may be implemented to store and retrieve matrix data (and other tensor data) efficiently. For instance, each MRB may be configured to read and write 32 matrix values either row-wise or column-wise every cycle. As an example, a MRB (e.g., 830a-n) may be composed of 16 logical memories with individual addressing and input and output data rotation to support both the row and column access, among other example implementations” wherein the matrix values reads implicitly on at least some plurality of data channels; wherein the data of the matrices are grouped within the MRB across 16 logical memories stored in individual addresses of the second memory
Lau [0111]; “In the illustrated example, matrix processing engine 1700 is implemented on a cluster with two MPUs 1734a-b. Accordingly, matrix processing engine 1700 can perform two matrix operations 1701 and 1702 in parallel using the respective MPUs 1734” wherein the plurality of respective MPUs performing the matrix operations in parallel thus reads on the data of each data group configured for processing in parallel by different processing elements of the computer unit)
Regarding Claim 8,
Lau teaches the method of Claim 7 (and thus the rejection of Claim 7 is incorporated). Lau already discloses wherein each data group of a data matrix includes data of at least one position of the data matrix for at least one channel of the data matrix (Lau [0129]; “In the illustrated example of FIG. 18, memory 1800 is used to store matrix 1810. Matrix 1810 includes a plurality of matrix elements A-I arranged into rows 1812 and columns 1814. For example, matrix 1810 includes three rows 1812 (r1, r2, and r3) and three columns 1814 (c1, c2, and c3) for storing matrix elements A-I. Matrix 1810 may be stored in memory 1800 using memory modules 1801. For example, each element A-I of matrix 1810 may be stored in a particular entry 1802 of memory modules 1801.”)
Regarding Claim 9,
Lau teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Lau already discloses A system-on-chip including an integrated circuit (Lau [0080]; “In some embodiments, the matrix processing functionality described throughout this disclosure may be implemented using a matrix processing architecture, such as the matrix processing architecture of FIGS. 16A-16C. Matrix processing architectures, such as the matrix processing architecture of FIGS. 16A-16C, may be implemented or used in a variety of systems, devices, and/or components, such as those described throughout this disclosure, including system 100 of FIG. 1 and/or any of its associated components (e.g., cloud services 120/datacenter servers, edge devices 110, matrix processing nodes 130). In some embodiments, the matrix processing architecture of FIGS. 16A-16C may be used to implement artificial intelligence and machine learning in neural networks. The matrix processing architecture illustrated in FIGS. 16A-16C is merely one example embodiment for performing the matrix processing functionality described throughout this disclosure. Other embodiments may use different types, arrangements, and/or numbers of components. For example, other embodiments may include any number of matrix processing chips 1620, matrix processing clusters 1630, matrix processing units (MPUs) 1634, high bandwidth memory (HBM) modules 1640, and/or memory resource blocks (MRBs) 1638. Moreover, all or part of any component of the matrix processing architecture of FIGS. 16A-16C (e.g., any component of matrix processing system 1600, matrix processing chips 1620, and/or matrix processing clusters 1630) may be implemented as a separate or stand-alone component or chip, or may be integrated with other components or chips, such as a system-on-a-chip (SoC) that integrates various computer components into a single chip”)
Regarding Claim 10,
Lau discloses An integrated circuit, comprising: a computer unit having a first input, a second input and an output; (Lau [0044]; “a machine learning computing system may be provided that includes an application-specific integrated circuit (ASIC)-based deep learning hardware (DLH) device provided that is designed to accelerate computations for deep learning applications.”
Lau [0086]; “In some embodiments, a matrix processing chip 1620 may be implemented using an integrated circuit, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), and/or any other type of circuitry.”
Lau [0116]; “Output engine 1737 may then be used to compute a result for the particular matrix operation 1701. For example, output engine 1737 may perform the appropriate matrix operation 1701 using the matrix operands generated by slice engine 1736a (e.g., the matrix operands stored in MRBs 1738b and 1738c).
In some embodiments, for example, output engine 1737 may first identify an associated matrix subroutine corresponding to the particular matrix operation, and output engine 1737 may then obtain that matrix subroutine from matrix subroutine memory 1739. Matrix subroutine memory 1739, for example, may be a memory component used to store matrix subroutines that are used by output engine 1737. A matrix subroutine, for example, may be a programmable subroutine for a matrix processor that is designed to perform a particular matrix operation when executed by the matrix processor. For example, a matrix subroutine may include a series of instructions and/or commands, supported by a particular matrix processor, and designed to perform a desired matrix operation when executed by the matrix processor. In some embodiments, for example, a matrix processor may be designed to support a set of instructions and/or commands for performing various fundamental operations. For example, in some embodiments, a matrix processor may support instructions for processing data, performing various arithmetic operations, and/or identifying matrix operands and outputs for the various instructions and operations. In this manner, the fundamental instructions and/or commands supported by the matrix processor can be used to program matrix subroutines for more complex matrix operations, such as distributed matrix multiplication and/or convolution operations, dimension shuffle operations, reshape operations, and so forth.” wherein the output engine’s performance of matrix and convolutional operations for output of the model thus reads on a computer unit executing a neural network
Lau [0044]; “Instructions of the MPUs may take tensors as inputs or operands. These instructions may be sent from a general purpose host processor to the DLH device. The instructions, as sent down from the host processor, may also operate on tensors. These instructions may be processed by the control logic of the DLH to feed the other units (MPU, memory, etc.). … For instance, data to be fetched or written to using the MPUs may be stored in tensor form, among other example features. Further, workloads involving a convolution or matrix multiplication operation may be performed by orchestrating portions of the work to be performed substantially in parallel by multiple MPUs. Data transferred between MPUs or even between multiple DLHs (e.g., as in the example of FIG. 2) may be transferred as tensors. Additionally, specialized memory blocks may be provided, with access to the memory shared by the multiple MPUs to limit data exchanges and simplify and expedite workloads involving multiple cooperating MPUs, among other example functions and advantages” wherein a plurality of tensors as inputs or operands for the model as well as tensor-transmitted data to be fetched or written by the parallel MPUs thus reads on a first and second input of the computing unit as well as transmitted outputs)
a first memory configured to store first data; (Lau [0110]; “Memory resource blocks (MRBs) 1738 may be memory components designed to efficiently store and retrieve matrix data. In some embodiments, for example, MRBs 1738 may be memory resource blocks on a particular matrix processing cluster. In those embodiments, for example, MRBs 1738 may be used to store and retrieve matrix data associated with matrix operations performed on the particular cluster”
Lau [0112]; “In the illustrated example, matrix processing engine 1700 is being used to perform convolution related operations, and thus the matrix data is associated with the image(s) and filters involved in those operations”
Lau [0113]; “Read engine 1735 may then store the matrix data retrieved from HBM 1740a in certain MRBs 1738a of its associated cluster. In some embodiments, for example, read engine 1735 may use two MRBs 1738a to store the associated matrix data. For example, read engine 1735 may use one MRB to store matrix data associated with an image, and may use another MRB to store matrix data associated with a filter used for convolution related operations on that image.” wherein the first MRB storing matrix data associated with an image thus reads on storing parameters of the neural network to be executed (since the matrix data associated with the images used to perform convolution related operations implicitly reads on stored parameters of the neural network))
a second memory configure to store second data applied to the second input of the computer unit; (Lau [0110]; “Memory resource blocks (MRBs) 1738 may be memory components designed to efficiently store and retrieve matrix data. In some embodiments, for example, MRBs 1738 may be memory resource blocks on a particular matrix processing cluster. In those embodiments, for example, MRBs 1738 may be used to store and retrieve matrix data associated with matrix operations performed on the particular cluster”
Lau [0112]; “In the illustrated example, matrix processing engine 1700 is being used to perform convolution related operations, and thus the matrix data is associated with the image(s) and filters involved in those operations”
Lau [0113]; “Read engine 1735 may then store the matrix data retrieved from HBM 1740a in certain MRBs 1738a of its associated cluster. In some embodiments, for example, read engine 1735 may use two MRBs 1738a to store the associated matrix data. For example, read engine 1735 may use one MRB to store matrix data associated with an image, and may use another MRB to store matrix data associated with a filter used for convolution related operations on that image.” wherein the second MRB storing matrix data associated with a filter thus reads on storing parameters of the neural network to be executed (since the matrix data associated with the filters used to perform convolution related operations implicitly reads on stored parameters of the neural network))
a first barrel shifter unit having an input configured to receive first data from the first memory and an output configured to deliver barrel shifted first data to the first input of the computer unit; (Lau [0124]; “FIG. 18 illustrates an example embodiment of memory 1800 for storing matrices using data shifting. In some embodiments, for example, memory 1800 may be used to provide efficient storage of matrices in a matrix processing system. For example, memory 1800 could be used to implement the memory resource blocks (MRBs) of an example processing cluster.”
Lau [0128]; “Memory 1800 also includes a memory controller 1806. Memory controller 1806 may be used for efficiently storing and retrieving rows and columns of a matrix from memory modules 1801, as described further below. Moreover, in the illustrated embodiment, memory 1800 includes barrel shifters 1804a and 1804b. Barrel shifters 1804 may be used for shifting matrix data in order to provide efficient access to rows and columns of a matrix, as described further below”
Lau [0137]; “After a particular row 1812 or column 1814 of matrix 1810 is read from memory modules M1, M2 and M3, the elements are out-of-order and thus must be shifted back to their original order, which is performed by barrel shifter 1804b. The shifting required when reading a particular row 1812 or column 1814 is simply the opposite of the shifting performed when the row 1812 or column 1814 was originally written. Accordingly, when reading out the rows 1812 of matrix 1810 from memory 1800, the first row is shifted by 0 elements (e.g., no shifting is performed), the second row is shifted by −1 element, the third row is shifted by −2 elements, and so forth. Similarly, when reading out the columns 1814 of matrix 1810 from memory 1800, the first column is shifted by 0 elements (e.g., no shifting is performed), the second column is shifted by −1 element, the third column is shifted by −2 elements, and so forth.”
Lau [Figure 18];
PNG
media_image1.png
259
535
media_image1.png
Greyscale
wherein the first barrel shifter 1804a performed on output from one of the plurality of MRBs transmitting shifted elements to be accessed and input by the computer unit)
a second barrel shifter unit having an input configured to receive output data from the output of the computer unit and an output configured to deliver barrel shifted output data for storage in the first memory; (Lau [0124]; “FIG. 18 illustrates an example embodiment of memory 1800 for storing matrices using data shifting. In some embodiments, for example, memory 1800 may be used to provide efficient storage of matrices in a matrix processing system. For example, memory 1800 could be used to implement the memory resource blocks (MRBs) of an example processing cluster.”
Lau [0128]; “Memory 1800 also includes a memory controller 1806. Memory controller 1806 may be used for efficiently storing and retrieving rows and columns of a matrix from memory modules 1801, as described further below. Moreover, in the illustrated embodiment, memory 1800 includes barrel shifters 1804a and 1804b. Barrel shifters 1804 may be used for shifting matrix data in order to provide efficient access to rows and columns of a matrix, as described further below”
Lau [0137]; “After a particular row 1812 or column 1814 of matrix 1810 is read from memory modules M1, M2 and M3, the elements are out-of-order and thus must be shifted back to their original order, which is performed by barrel shifter 1804b. The shifting required when reading a particular row 1812 or column 1814 is simply the opposite of the shifting performed when the row 1812 or column 1814 was originally written. Accordingly, when reading out the rows 1812 of matrix 1810 from memory 1800, the first row is shifted by 0 elements (e.g., no shifting is performed), the second row is shifted by −1 element, the third row is shifted by −2 elements, and so forth. Similarly, when reading out the columns 1814 of matrix 1810 from memory 1800, the first column is shifted by 0 elements (e.g., no shifting is performed), the second column is shifted by −1 element, the third column is shifted by −2 elements, and so forth.”
Lau [Figure 18];
PNG
media_image1.png
259
535
media_image1.png
Greyscale
wherein the second barrel shifter 1804b performed on the out-of-order elements of the first memory MRB output to shift such elements back to their original order in the first MRB thus reads on a second barrel shifter configured to deliver barrel shifted data for storage in the first MRB (since the re-ordered elements of MRB thus reads on their delivery to the MRB in question))
and a control circuit configured to control execution operation by the computer unit, barrel shifting operation by the first and second barrel shifter unit and read/write operation of the first and second memories (Lau [0044]; “These instructions may be processed by the control logic of the DLH to feed the other units (MPU, memory, etc.). These instructions may include data movement (e.g. from off-chip memory into on-chip memory, operands in on-chip memory, and the arithmetic operations). This data may be stored and transferred as tensors in on-chip and off-chip memory, and between the host and the chip.” wherein directing data movement reads on controlled activity of the first and second barrel shifter circuits and general control of the computer unit
Lau [0113]; “For example, read engine 1735 may use one MRB to store matrix data associated with an image, and may use another MRB to store matrix data associated with a filter used for convolution related operations on that image. In some embodiments, read engine 1735 may use the master control CPU (MCC) 1732 on its respective cluster for storing and retrieving data on HBMs 1740 and MRBs 1738.”
Lau [0118]; “In some embodiments, output engine 1737 may use the master control CPU (MCC) 1732 on its respective cluster to retrieve matrix subroutines from matrix subroutine memory 1739, and to specify or supply any remaining information and/or fields for the particular matrix subroutine (e.g., the size and/or location of matrix operands)” wherein the master control CPU’s control on matrix subroutines, operands, and fields thus reads on controlled access to the first and second memories, specifically read/write operations of storage and retrieval on HBMs by the control circuit)
Regarding Claim 11,
Lau teaches the method of Claim 10 (and thus the rejection of Claim 10 is incorporated). Lau already discloses wherein the first data comprises input data for neural network process executed by the computer network and the second data comprises parameter data for configuring the neural network process (Lau [0110]; “Memory resource blocks (MRBs) 1738 may be memory components designed to efficiently store and retrieve matrix data. In some embodiments, for example, MRBs 1738 may be memory resource blocks on a particular matrix processing cluster. In those embodiments, for example, MRBs 1738 may be used to store and retrieve matrix data associated with matrix operations performed on the particular cluster”
Lau [0112]; “In the illustrated example, matrix processing engine 1700 is being used to perform convolution related operations, and thus the matrix data is associated with the image(s) and filters involved in those operations”
Lau [0113]; “Read engine 1735 may then store the matrix data retrieved from HBM 1740a in certain MRBs 1738a of its associated cluster. In some embodiments, for example, read engine 1735 may use two MRBs 1738a to store the associated matrix data. For example, read engine 1735 may use one MRB to store matrix data associated with an image, and may use another MRB to store matrix data associated with a filter used for convolution related operations on that image.” wherein the first MRB stored first data comprising matrix data correspondent to the image input data for the neural network processes; wherein the second MRB stored second data comprising filter data comprises parameter data for configuration of the convolutional process of the neural network)
Regarding Claim 14,
Lau teaches the method of Claim 10 (and thus the rejection of Claim 10 is incorporated). Lau already discloses wherein the computer unit comprises a plurality of processing units executing in parallel (Lau [0111]; “Matrix processing engine 1700 performs matrix operations using read engine 1735, slice engines 1736, and output engine 1737, as described further below. In the illustrated example, matrix processing engine 1700 is performing multiple matrix operations 1701 and 1702 in parallel. For example, as noted above, in some embodiments matrix processing engine 1700 may be implemented on a particular matrix processing cluster, and the particular matrix processing cluster may include multiple MPUs 1734. In the illustrated example, matrix processing engine 1700 is implemented on a cluster with two MPUs 1734a-b. Accordingly, matrix processing engine 1700 can perform two matrix operations 1701 and 1702 in parallel using the respective MPUs 1734” wherein the plurality of MPUs read on a bank of processing elements configured to parallelize execution)
Regarding Claim 15,
Lau teaches the method of Claim 14 (and thus the rejection of Claim 14 is incorporated). Lau already discloses a shift circuit configured to shift second data from the second memory for application to ones of the processing units (Lau [0111]; “Matrix processing engine 1700 performs matrix operations using read engine 1735, slice engines 1736, and output engine 1737, as described further below. In the illustrated example, matrix processing engine 1700 is performing multiple matrix operations 1701 and 1702 in parallel. For example, as noted above, in some embodiments matrix processing engine 1700 may be implemented on a particular matrix processing cluster, and the particular matrix processing cluster may include multiple MPUs 1734. In the illustrated example, matrix processing engine 1700 is implemented on a cluster with two MPUs 1734a-b. Accordingly, matrix processing engine 1700 can perform two matrix operations 1701 and 1702 in parallel using the respective MPUs 1734” wherein the plurality of MPUs read on a bank of processing elements configured to parallelize execution; wherein the first barrel shifter’s matrix operations performed by such MPUs thus reads on the first barrel shifter circuit transmitting data from the second memory to its respective parallel MPU)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-5; 13, 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Lau et al. (US 20190392297 A1, as disclosed in the IDS, hereinafter “Lau”) in view of Krishnamurthy et al. (US 5845099 A, hereinafter “Krishnamurthy”).
Regarding Claim 3,
Lau teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Lau fails to explicitly disclose but Krishnamurthy discloses including a first multiplexer stage, wherein an input of the first barrel shifter circuit is connected to the second memory via the first multiplexer stage, and wherein the first multiplexer stage is configured to deliver to the first barrel shifter circuit a data vector from the data stored in the second memory, the first barrel shifter circuit being configured to shift the data vector of the first multiplexer stage (Krishnamurthy [Figure 3];
PNG
media_image2.png
428
706
media_image2.png
Greyscale
wherein the first 7x1 multiplexer alongside buffer 302 is indicative of a first multiplexer stage; wherein barrel shifter 330 connected via the first 7x1 multiplexer to a plurality of array memory reads on an input of the first barrel shifter circuit connected to the second memory via the first multiplexer stage
Krishnamurthy [Abstract]; “The circuit includes a first pointing unit to indicate a position of a first byte of an instruction of the stream of N successive instruction bytes. The circuit also includes a second pointing unit to store a vector having a length N” wherein the combination of elements 318 and 302 interpreted as a first multiplexer stage both deliver the initial RP vector as well as the length to be shifted, thus interpreted as a first multiplexer stage delivering to the first barrel shifter a data vector from the data stored in memory; wherein the first barrel shifter 330 shifting RP to RP2 reads on shifting the data vector of the first multiplexer stage)
It would have been obvious to modify Lau’s barrel shifters to include Krishnamurthy’s additional multiplexer stage in its connection to second memory. One would have been motivated to do so because “In response to this control signal, the multiplexer 326 selects an input that corresponds” (Krishnamurthy [Column 8 Line 24]) thus allowing Lau’s barrel shifters to only be performed on particular select input.
Regarding Claim 4,
Lau/Krishnamurthy teaches the method of Claim 3 (and thus the rejection of Claim 3 is incorporated). Lau/Krishnamurthy already discloses a second multiplexer stage, wherein the input of the computer unit is connected to the first barrel shifter circuit via the second multiplexer stage, and wherein the second multiplexer stage is configured to deliver the data vector shifted by the first barrel shifter circuit to the computer unit (Krishnamurthy [Figure 3];
PNG
media_image2.png
428
706
media_image2.png
Greyscale
wherein the shifted data vector output by the first barrel shifter is connected as input into the 6x1 MUX thus interpreted as a second multiplexer stage wherein the first barrel shifter is connected to the input of the computer unit; wherein the shifted data vector RP2 being input into the multiplexer which selects one of RP2 or RP1 as output to be delivered to the computing unit as the updated current read pointer thus reads on the second multiplexer stage configured to deliver the data vector shifted by the first barrel shifter to the computing unit)
Regarding Claim 5,
Lau teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Lau fails to explicitly disclose but Krishnamurthy discloses including a buffer memory, wherein the second barrel shifter circuit is connected to the computer unit via the buffer memory, and wherein the buffer memory is configured to temporarily store the data generated by the computer unit during the execution of the neural network before the second barrel shifter circuit delivers the data to the second memory (Krishnamurthy [Figure 3];
PNG
media_image2.png
428
706
media_image2.png
Greyscale
Krishnamurthy [Column 5 Line 43]; “FIG. 3 illustrates an alternative embodiment of an instruction alignment unit (IAU) 300 according to the present invention. Instruction alignment unit 300 includes a first pointing unit 302 (hereinafter read pointer buffer 302) which stores a decoded 32 bit read pointer. The decoded read pointer is a binary vector which indicates the position of a first byte of an instruction stored in an instruction buffer 310. “ wherein element 302 representing a buffer storing pointer vector RP and directly connected as input to the second barrel shifter circuit 328 before the second barrel shifter circuit eventually delivers the data as an updated pointer thus reads on a buffer memory connecting the barrel shifter circuit to the computer unit and temporarily storing generated data of the computer unit)
It would have been obvious to modify Lau’s barrel shifters to include Krishnamurthy’s buffer memory connected to its second barrel shifter circuit to temporarily store data. One would have been motivated to do so because a buffer has the capacity to “indicates a position of a last byte of at least one instruction which is included in the stream of successive instruction bytes stored” (Krishnamurthy [Column 3 Line 50]).
Regarding Claim 13,
Lau teaches the method of Claim 10 (and thus the rejection of Claim 10 is incorporated). Lau fails to explicitly disclose but Krishnamurthy discloses a buffer circuit coupled between the output of the computer unit and the input of the second barrel shifter unit, said buffer circuit configured to buffer store the output data (Krishnamurthy [Figure 3];
PNG
media_image2.png
428
706
media_image2.png
Greyscale
Krishnamurthy [Column 5 Line 43]; “FIG. 3 illustrates an alternative embodiment of an instruction alignment unit (IAU) 300 according to the present invention. Instruction alignment unit 300 includes a first pointing unit 302 (hereinafter read pointer buffer 302) which stores a decoded 32 bit read pointer. The decoded read pointer is a binary vector which indicates the position of a first byte of an instruction stored in an instruction buffer 310. “ wherein element 302 representing a buffer storing pointer vector RP and directly connected as input to the second barrel shifter circuit 328 before the second barrel shifter circuit eventually delivers the data as an updated pointer thus reads on a buffer memory connecting the barrel shifter circuit to the computer unit and temporarily storing generated data of the computer unit)
It would have been obvious to modify Lau’s barrel shifters to include Krishnamurthy’s buffer circuit connected to its second barrel shifter circuit to temporarily store data. One would have been motivated to do so because a buffer has the capacity to “indicates a position of a last byte of at least one instruction which is included in the stream of successive instruction bytes stored” (Krishnamurthy [Column 3 Line 50]).
Regarding Claim 16,
Lau teaches the method of Claim 10 (and thus the rejection of Claim 10 is incorporated). Lau fails to explicitly disclose but Krishnamurthy discloses wherein the first data comprises a first data vector, and further comprising: a first multiplexing circuit having an input configured to receive the first data vector and an output coupled to the input of the first barrel shifter unit and configured to generate a shifted data vector for input to the first barrel shifter unit; and a second multiplexing circuit having an input coupled to the output of the first barrel shifter unit and an output configured to generate a second data vector for input to the computer unit (Krishnamurthy [Figure 3];
PNG
media_image2.png
428
706
media_image2.png
Greyscale
wherein the first 7x1 multiplexer alongside buffer 302 is indicative of a first multiplexer stage; wherein barrel shifter 330 connected via the first 7x1 multiplexer to a plurality of array memory reads on an input of the first barrel shifter circuit connected to the second memory via the first multiplexer stage
Krishnamurthy [Abstract]; “The circuit includes a first pointing unit to indicate a position of a first byte of an instruction of the stream of N successive instruction bytes. The circuit also includes a second pointing unit to store a vector having a length N” wherein the combination of elements 318 and 302 interpreted as a first multiplexer stage both deliver the initial RP vector as well as the length to be shifted, thus interpreted as a first multiplexer stage delivering to the first barrel shifter a data vector from the data stored in memory; wherein the first barrel shifter 330 shifting RP to RP2 reads on shifting the data vector of the first multiplexer stage; wherein the shifted data vector output by the first barrel shifter is connected as input into the 6x1 MUX thus interpreted as a second multiplexer stage wherein the first barrel shifter is connected to the input of the computer unit; wherein the shifted data vector RP2 being input into the multiplexer which selects one of RP2 or RP1 as output to be delivered to the computing unit as the updated current read pointer thus reads on the second multiplexer stage configured to deliver the data vectors RP1 and RP2 shifted by the first barrel shifter to the computing unit)
It would have been obvious to modify Lau’s barrel shifters to include Krishnamurthy’s additional multiplexer stages in its connection to memory and computing unit inputs. One would have been motivated to do so because “In response to this control signal, the multiplexer 326 selects an input that corresponds” (Krishnamurthy [Column 8 Line 24]) thus allowing Lau’s barrel shifters to only be performed on particular select input.
Regarding Claim 17,
Lau/Krishnamurthy teaches the method of Claim 16 (and thus the rejection of Claim 16 is incorporated). Lau/Krishnamurthy already discloses wherein the computer unit comprises a plurality of processing units executing in parallel and configured to receive the second data vector (Lau [0111]; “Matrix processing engine 1700 performs matrix operations using read engine 1735, slice engines 1736, and output engine 1737, as described further below. In the illustrated example, matrix processing engine 1700 is performing multiple matrix operations 1701 and 1702 in parallel. For example, as noted above, in some embodiments matrix processing engine 1700 may be implemented on a particular matrix processing cluster, and the particular matrix processing cluster may include multiple MPUs 1734. In the illustrated example, matrix processing engine 1700 is implemented on a cluster with two MPUs 1734a-b. Accordingly, matrix processing engine 1700 can perform two matrix operations 1701 and 1702 in parallel using the respective MPUs 1734” wherein the plurality of MPUs read on a bank of processing elements configured to parallelize matrix operations including data transmission, thus Lau’s plurality of MPUs are configured to process and thus receive the second data vectors (RP2) output by Krishnamurthy’s second-stage multiplexer)
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Lau et al. (US 20190392297 A1, as disclosed in the IDS, hereinafter “Lau”) in view of Krishnamurthy et al. (US 5845099 A, hereinafter “Krishnamurthy”) further in view of Coenen (US 20210287074 A1)
Regarding Claim 6,
Lau/Krishnamurthy teaches the method of Claim 5 (and thus the rejection of Claim 5 is incorporated). Lau/Krishnamurthy fails to explicitly disclose but Coenen discloses a pruning stage between the buffer memory and the second barrel shifter circuit, wherein the pruning stage is configured to delete some data among the data generated by the computer unit. (Coenen [Figure 1A];
PNG
media_image3.png
364
515
media_image3.png
Greyscale
Coenen [0005]; “The accelerator includes a decompression logic circuit configured to retrieve the encoded weights from the weight memory, and decode the encoded weights to obtain a sequence of one or more pruned weight words and one or more non-pruned weight words, where the pruned weight words include zero-value weight words, and the non-pruned weight words include non-zero-value weight words. The accelerator includes a plurality of input-weight multipliers configured to receive the sequence of the pruned weight words and the non-pruned weight words”
Coenen [0006]; “The device includes a control logic configured to control the first weight decoder and the second weight decoder, a shifter configured to receive the first group of decoded weight words from the first weight decoder and the second group of decoded weight words from the second weight decoder, and a plurality of weight registers coupled to the shifter, where each weight register of the plurality of weight registers is configured to receive a separate decoded weight word from the shifter”
Coenen [0067]; “In some examples, the shifters 540 include one or more digital circuits that can shift a data word by a specified number of bits. In some examples, the shifters 540 are a sequence of multiplexers. In some examples, the shifters 540 includes a cascade of flip flops. In some examples, the shifters 540 include one or more barrel shifters.” wherein the output data of the neural network trainer having its weights pruned before input into an accelerator comprising barrel shifters thus reads on a pruning stage between the output of the computer unit and the input of the second barrel shifter unit)
It would have been obvious to modify the circuit architecture of Lau/Krishnamurthy to incorporate Coenen’s pruning circuit for removal of useless data from the output data. One would have been motivated to do so because “Since the number of weights is the product of the number of neurons in both layers, the amount of weights can be relatively large (thereby increasing the amount of memory to store these weights) and the amount of multiplications can be relatively large (thereby increasing the amount of processing power to perform the multiplications).” (Coenen [0002]) and thus, pruning would consequently decrease the amount of memory to store such weights as well as decrease the amount of processing power required.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Lau et al. (US 20190392297 A1, as disclosed in the IDS, hereinafter “Lau”) in view of Coenen (US 20210287074 A1)
Lau teaches the method of Claim 10 (and thus the rejection of Claim 10 is incorporated). Lau fails to explicitly disclose but Coenen discloses a pruning circuit coupled between the output of the computer unit and the input of the second barrel shifter unit, said pruning circuit configured to prune useless data from the output data (Coenen [Figure 1A];
PNG
media_image3.png
364
515
media_image3.png
Greyscale
Coenen [0005]; “The accelerator includes a decompression logic circuit configured to retrieve the encoded weights from the weight memory, and decode the encoded weights to obtain a sequence of one or more pruned weight words and one or more non-pruned weight words, where the pruned weight words include zero-value weight words, and the non-pruned weight words include non-zero-value weight words. The accelerator includes a plurality of input-weight multipliers configured to receive the sequence of the pruned weight words and the non-pruned weight words”
Coenen [0006]; “The device includes a control logic configured to control the first weight decoder and the second weight decoder, a shifter configured to receive the first group of decoded weight words from the first weight decoder and the second group of decoded weight words from the second weight decoder, and a plurality of weight registers coupled to the shifter, where each weight register of the plurality of weight registers is configured to receive a separate decoded weight word from the shifter”
Coenen [0067]; “In some examples, the shifters 540 include one or more digital circuits that can shift a data word by a specified number of bits. In some examples, the shifters 540 are a sequence of multiplexers. In some examples, the shifters 540 includes a cascade of flip flops. In some examples, the shifters 540 include one or more barrel shifters.” wherein the output data of the neural network trainer having its weights pruned before input into an accelerator comprising barrel shifters thus reads on a pruning circuit coupled between the output of the computer unit and the input of the second barrel shifter unit)
It would have been obvious to modify the circuit architecture of Lau to incorporate Coenen’s pruning circuit for removal of useless data from the output data. One would have been motivated to do so because “Since the number of weights is the product of the number of neurons in both layers, the amount of weights can be relatively large (thereby increasing the amount of memory to store these weights) and the amount of multiplications can be relatively large (thereby increasing the amount of processing power to perform the multiplications).” (Coenen [0002]) and thus, pruning would consequently decrease the amount of memory to store such weights as well as decrease the amount of processing power required.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“System-on-chip, Data Processing Method Thereof, and Neural Network Device” (US 11392377 B2) which discloses a data processing circuit comprising a system-on-a-chip including a first and second memory for neural network acceleration .
“Neural Network Processor with a Window Expander Circuit” (US 11151445 B2) which discloses a circuit for processing artificial neural networks including multiplexer-processed data vector input.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN J KIM whose telephone number is (571) 272-0523.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN J KIM/Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights