DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 09/26/2023 and 12/16/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Interpretation
Claims 1, 14, and 17 state the limitation “a wideband data connection”. Applicant’s Specification filed 09/26/2023, Paragraph [0074] states that a wideband data connection includes any data connection with a bandwidth of a predetermined threshold, such as PCIe. Thus, Examiner will interpret a wideband data connection as any high bandwidth data connection protocol.
Furthermore, claims 1, 14, and 17 state the limitation “persistent memory”. Applicant’s Specification filed 09/26/2023, Paragraph [0066] states that persistent memory includes any non-volatile memory. Thus, Examiner will interpret persistent memory as any non-volatile memory type.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-16 and 19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Under Step 1, claims 1-13 recite a system comprising circuitry, and therefore is a machine. Claims 14-16 recite a method, and therefore is a process. Claim 19 recites a device comprising circuitry, and therefore is a machine.
Under Step 2A, prong 1, claim 1 recites control and inference circuit that is configured to perform arithmetic operations.
Since the claim recites a generic computer element/circuit to perform arithmetic operations, the act of performing arithmetic operations is a mental step. Applicant’s Specification filed 09/26/2023 discloses that arithmetic operations include multiplication operations (See Paragraphs [0071] and [0075]). The judicial exception is therefore mathematical/logical operations. Accordingly, the claim is directed to reciting an abstract idea.
Under step 2A, prong 2, the generic computer elements are not sufficiently integrating the act of performing arithmetic operations into a practical application. These judicial exceptions are not integrated into a practical application because the claims merely incorporate the steps into a system comprising persistent memory and inference circuitry. Accordingly, the claim is not integrated into a practical application.
Under Step 2B, claims 1-13 do not recite additional elements that alone or in combination amount to an inventive concept. For example, the claim merely discloses transferring data over wideband data connections, which is well-understood, routine, and conventional activity. US PGPUB 2021/0373790 to Henderson discloses in Paragraph [0022] that using a wideband data connection such as PCIe is well-known and conventional in machine learning and artificial intelligence training. These judicial exceptions are not integrated into a practical application because the claims merely incorporate these steps into a system containing a memory and a controller, which is mere instructions to apply an exception. See MPEP 2106.05(f) (2) Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) and See MPEP 2106.05(f)(2)(iii) A process for monitoring audit log data that is executed on a general-purpose computer where the increased speed in the process comes solely from the capabilities of the general-purpose computer, FairWarning IP, LLC v. Iatric Sys., 839 F.3d 1089, 1095, 120 USPQ2d 1293, 1296 (Fed. Cir. 2016). Accordingly, the claims do not amount to significantly more than the abstract idea.
Under Step 2A, prong 1, claim 14 recites performing inference operation of a neural network by reading a weight, multiplying the weight by an element of an input feature map to form a first product, and calculating an activation based on the first product.
Since the claim recites a generic computer element/circuit to perform the operations, the act of reading, multiplying, and calculating are mental steps. Applicant’s Specification filed 09/26/2023 discloses that these operations include multiplication operations (See Paragraphs [0071] and [0075]). The judicial exception is therefore an algorithm and/or mathematical/logical operations. Accordingly, the claim is directed to reciting an abstract idea.
Under step 2A, prong 2, the generic computer elements are not sufficiently integrating the act of performing arithmetic operations into a practical application. These judicial exceptions are not integrated into a practical application because the claims merely incorporate the steps into a system comprising persistent memory and inference circuitry. Accordingly, the claim is not integrated into a practical application.
Under Step 2B, claims 14-16 do not recite additional elements that alone or in combination amount to an inventive concept. For example, the claim merely discloses transferring data over wideband data connections, which is well-understood, routine, and conventional activity. US PGPUB 2021/0373790 to Henderson discloses in Paragraph [0022] that using a wideband data connection such as PCIe is well-known and conventional in machine learning and artificial intelligence training. These judicial exceptions are not integrated into a practical application because the claims merely incorporate these steps into a system containing a memory and a controller, which is mere instructions to apply an exception. See MPEP 2106.05(f) (2) Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) and See MPEP 2106.05(f)(2)(iii) A process for monitoring audit log data that is executed on a general-purpose computer where the increased speed in the process comes solely from the capabilities of the general-purpose computer, FairWarning IP, LLC v. Iatric Sys., 839 F.3d 1089, 1095, 120 USPQ2d 1293, 1296 (Fed. Cir. 2016). Accordingly, the claims do not amount to significantly more than the abstract idea.
Under Step 2A, prong 1, claim 19 recites performing an operation selected from the group consisting of pruning, sparsity, compression, quantization, and approximation.
Since the claim recites a generic computer element/circuit to perform the operations consisting of pruning, sparsity, compression, quantization, and approximation are mental steps. Applicant’s Specification filed 09/26/2023 discloses that these operations are mathematical operations (See Paragraph [0084]). The judicial exception is therefore an algorithm and/or mathematical/logical operations. Accordingly, the claim is directed to reciting an abstract idea.
Under step 2A, prong 2, the generic computer elements are not sufficiently integrating the act of performing arithmetic operations into a practical application. These judicial exceptions are not integrated into a practical application because the claims merely incorporate the steps into a system comprising persistent memory and inference circuitry. Accordingly, the claim is not integrated into a practical application.
Under Step 2B, claim 19 does not recite additional elements that alone or in combination amount to an inventive concept. For example, the claim merely discloses transferring data over wideband data connections, which is well-understood, routine, and conventional activity. US PGPUB 2021/0373790 to Henderson discloses in Paragraph [0022] that using a wideband data connection such as PCIe is well-known and conventional in machine learning and artificial intelligence training. These judicial exceptions are not integrated into a practical application because the claims merely incorporate these steps into a system containing a memory and a controller, which is mere instructions to apply an exception. See MPEP 2106.05(f) (2) Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) and See MPEP 2106.05(f)(2)(iii) A process for monitoring audit log data that is executed on a general-purpose computer where the increased speed in the process comes solely from the capabilities of the general-purpose computer, FairWarning IP, LLC v. Iatric Sys., 839 F.3d 1089, 1095, 120 USPQ2d 1293, 1296 (Fed. Cir. 2016). Accordingly, the claims do not amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-7, 14-15, and 17-18 are rejected under 35 U.S.C. 102(a)(1) and 35 U.S.C. 102(a)(2) as being anticipated by Shah (US 2020/0167098).
Regarding claim 1, Shah teaches a system (Fig. 5, System; Paragraph 0019, IG. 5 depicts an example inference and storage system), comprising: a first persistent memory (Fig. 5, Media 550 includes persistent memory such as SSD; Paragraph 0046, media 550 includes SSDs 1-8); and a control and inference circuit (Fig. 5, Inference engine 510 controls inference models), wherein: the first persistent memory is connected to the control and inference circuit by a wideband data connection (Fig. 5, Media 550 (i.e. first persistent memory) is connected to inference engine 510 via PCIe interface 545 which is a wideband data connection; Paragraph 0041, Media interface 545 can be compatible with one or more of: DDR4, DDR5, PCIe… Paragraph 0094, High speed interconnects can be… Peripheral Component Interconnect express (PCIe)… Paragraph 0097, high-speed link 1018, such as a 40 Gigabit/second (Gb/s)); and the control and inference circuit is configured to perform arithmetic operations (Fig. 5, Inference engine 510 performs inference operations using arithmetic on inference model weights; Paragraph 0020, use inference engine 510 to perform inference operations. Inference engine 510 can use one or more of: a decode and crop engine 512, compute engine 514… Paragraph 0066, an inference model (e.g., weights) and an address of the decoded data (to be processed using the inference model)… Paragraph 0023, output value of a preceding neuron can be multiplied by the weight of its connection).
Regarding claim 2, Shah teaches the system of claim 1. Shah teaches the system further comprising wherein: the control and inference circuit comprises a first persistent memory controller (Figs. 5 and 8, State machine 516 is a microprocessor that controls storage operations for persistent media 550 and is synonymous with memory controller 822 of Figure 8; Paragraph 0020, State machine 516 can be implemented as a microprocessor… Paragraph 0066, state machine issues a Read command for an inference model (and optionally, weights) and decoded data from the media… Paragraph 0086, memory controller to generate and issue commands to memory 830); the system further comprises: an interface circuit (Fig. 8, Interface 812), a second persistent memory (Fig. 8, Storage 884 is a second persistent memory; Paragraph 0090, storage device(s) 884, which can be or include any conventional medium for storing large amounts of data in a nonvolatile manner, such as one or more magnetic, solid state), and a second persistent memory controller (Fig. 8, Controller 882; Paragraph 0090, storage subsystem 880 includes controller 882 to interface with storage 884); the second persistent memory is connected to the second persistent memory controller (Fig. 8, Controller 882 connected to storage 884); the control and inference circuit is connected to the interface circuit (Fig. 8, Memory subsystem 820 is the control and inference circuit and is coupled to interface 812); and the second persistent memory controller is connected to the interface circuit (Fig. 8, Controller 882 is coupled to interface 814 which further transfers data to interface 812 and thus are connected to each other; Paragraph 0088, system 800 includes interface 814, which can be coupled to interface 812).
Regarding claim 3, Shah teaches the system of claim 1. Shah teaches the system further comprising: an interface circuit (Figs. 4 and 5, Memory board with inference in Figure 4 has interface 503 in Figure 5 which is serial), wherein: the interface circuit comprises a serial interface (Fig. 5, Interface 503 is PCIe or SATA which are both serial; Paragraph 0014, various specifications for M.2 include PCI Express M.2 Specification Revision 3.0, Version 1.2 (2013) and Serial ATA International Organization (SATA-IO) revision 3.2 specification); and the control and inference circuit is connected to the interface circuit (Fig. 5, Inference engine 510 coupled to interface 503).
Regarding claim 4, Shah teaches the system of claim 1. Shah teaches the system further comprising: an interface circuit (Fig. 8, Interface 812); a second persistent memory (Fig. 8, Storage 884); and a second persistent memory controller (Fig. 8, Controller 882), wherein: the control and inference circuit comprises a first persistent memory controller (Fig. 8, Memory controller 822); the interface circuit comprises a serial interface (Fig. 8, Interface 812 is serial; Paragraph 0094, system 800 can be implemented using interconnected compute sleds of processors, memories, storages, network interfaces, and other components. High speed interconnects can be… Peripheral Component Interconnect express (PCIe)); the control and inference circuit is connected to the interface circuit (Fig. 8, Memory subsystem 820 coupled coupled to interface 812 which is the synonymous with Figure 5 where inference engine 510 coupled to interface 503); and the second persistent memory is connected to the interface circuit (Fig. 8, Storage 884 coupled to interface 812 via interface 814).
Regarding claim 5, Shah teaches the system of claim 1. Shah teaches the system further comprising: a second persistent memory (Fig. 8, Storage 884), and a second persistent memory controller (Fig. 8, Controller 882), wherein: the control and inference circuit comprises a first persistent memory controller (Fig. 8, Memory controller 822), the first persistent memory comprises a flash memory (Fig. 8, Memory 830 is flash; Paragraph 0086, Memory subsystem 820 can include one or more memory devices 830 such as read-only memory (ROM), flash memory), the second persistent memory comprises a flash memory (Fig. 8, Storage 884 is flash NAND; Paragraph 0059, nonvolatile memory storage (e.g., flash memory or byte addressable non-volatile memory (e.g., Intel Optane® or Samsung Z-NAND®))… Paragraph 0090, storage device(s) 884, which can be or include any conventional medium for storing large amounts of data in a nonvolatile manner), the first persistent memory controller comprises a first flash memory controller (Fig. 8, Memory controller 822 controls flash memory 830 and thus is a flash memory controller), and the second persistent memory controller comprises a second flash memory controller (Fig. 8, Controller 882 controls flash memory 884 and thus is a flash memory controller).
Regarding claim 6, Shah teaches the system of claim 1. Shah teaches the system comprising wherein the wideband data connection comprises a serial connection (Fig. 5, PCIe interface 545 which is a wideband data connection uses PCIe which is a serial protocol; Paragraph 0041, Media interface 545 can be compatible with one or more of: DDR4, DDR5, PCIe).
Regarding claim 7, Shah teaches the system of claim 1. Shah teaches the system further comprising: a first interface circuit (Fig. 5, Interface 503), and an artificial intelligence accelerator (Fig. 5, Host 502 is an AI accelerator; Paragraph 0019, host 502 can include one or more processors (e.g., CPU, cores, GPU, accelerators… Paragraph 0047, Host 502 can issue a compute or storage command to processing system 504… Paragraph 0048, compute command can include or be associated with an inference model (e.g., model type) and include or reference weights stored in media 550), the artificial intelligence accelerator comprising an artificial intelligence processing circuit and a second interface circuit (Fig. 3, Host 302 is same as host 502 of Figure 5 and includes accelerator circuitry and host fabric interface (i.e. second interface circuit)), wherein: the first interface circuit is connected to the second interface circuit (Fig. 3, Host fabric interface coupled to interface of inference engine 304 with connection 300), the artificial intelligence accelerator is configured to perform inference operations of a neural network, with assistance from: the first persistent memory, and the control and inference circuit (Fig. 5, Host 502 transmits compute commands to perform inference of a neural network to inference engine 510 and media 550; Paragraph 0048, compute command can include or be associated with an inference model (e.g., model type) and include or reference weights stored in media 550… Paragraph 0085, Accelerators 842 can provide multiple neural networks, CPUs, processor cores, general purpose graphics processing units, or graphics processing units can be made available for use by artificial intelligence (AI) or machine learning (ML) models).
Regarding claim 14, Shah teaches a method (Fig. 5, Method performed in system 504), comprising: performing an inference operation of a neural network (Fig. 5, Inference engine 510 includes compute engine 514 that performs inference operation of a neural network; Paragraph 0023, Compute engine 514 can use a neural network to perform inferences or analysis), the performing comprising: reading a weight from a persistent memory into a random-access memory (Fig. 5, Inference weights are read from persistent memory 550 to local random-access memory 518; Paragraph 0015, To run an inference operation with a CPU or inference cards being used as an inference accelerator, the data is fetched from storage solid-state drive (SSD)… and then stored in static random access memory (SRAM)… Paragraph 0066, host issues a compute command with an address in the media of an inference model (e.g., weights)… the processing system copies the model and decoded data from media into local memory (e.g., SRAM)); multiplying the weight by an element of an input feature map to form a first product (Fig. 5, Compute engine 514 performs multiplication of weights of a feature map to obtain a product; Paragraph 0063, After the data and model are received by processing system 504, processing system 504 will execute the operation (e.g., Convolution, Relu, MaxPool). For example, Convolution can refer to application of a filter to an input that results in an activation. Repeated application of the same filter to an input results in a map of activations called a feature map… result of the operation can be written to media 550); and calculating an activation based on the first product (Fig. 5, Compute engine 514 can perform further processing such as convolution via multiplication, wherein a multiplication product is obtained and used to determine an activation; Paragraph 0025, Examples of a compute command supported by compute engine 514 include, but are not limited to… Paragraph 0027, 2. Matrix-Filter Convolve… Paragraph 0063, Convolution can refer to application of a filter to an input that results in an activation… host 502 can issue another command such as to retrieve the result or perform further processing on the result), wherein the reading of the weight from the persistent memory into the random- access memory comprises reading the weight from the persistent memory into the random-access memory (Figs. 5 and 6, Memory 518 (i.e. random-access memory) and media 550 (i.e. persistent memory) in Figure 5 are same embodiment as Figure 6 where weights are read in step 608 from media to memory; Paragraph 0066, an inference model (e.g., weights)… processing system copies the model and decoded data from media into local memory (e.g., SRAM)… Paragraph 0022, Data provided by decode and crop engine 512 can be stored in memory 518, which is accessible to compute engine 514. Data generated by compute engine 514 can be stored in memory 518) through a wideband data connection (Fig. 5, Media 550 (i.e. first persistent memory) is connected to inference engine 510 via PCIe interface 545 which is a wideband data connection, wherein the inference engine includes memory 518 which is the local SRAM; Paragraph 0041, Media interface 545 can be compatible with one or more of: DDR4, DDR5, PCIe… Paragraph 0094, High speed interconnects can be… Peripheral Component Interconnect express (PCIe)… Paragraph 0097, high-speed link 1018, such as a 40 Gigabit/second (Gb/s)).
Regarding claim 15, Shah teaches the method of claim 14. Shah teaches the method further comprising storing the activation in the random-access memory (Figs. 5 and 6, Memory 518 (i.e. random-access memory) stores activation; Paragraph 0066, an inference model (e.g., weights)… processing system copies the model and decoded data from media into local memory (e.g., SRAM)).
Regarding claim 17, Shah teaches a device (Fig. 4, Memory board with inference), comprising: a connector (Fig. 4, Memory board with inference is a memory card with a connector; Paragraph 0018, a board with memory and inference capabilities is available. The board with memory and inference capabilities can be M.2 compatible. A storage or memory device and inference engine can be mounted using an M.2 compatible circuit board. M.2 is a specification for internally mounted computer expansion cards and associated connectors for coupling to other devices in a server, rack, or blade architecture); a first persistent memory (Figs. 4 and 5, Memory board with inference in Figure 4 is the same embodiment as processing system 504 in Figure 5, which shows media 550 which is persistent memory; Paragraph 0018, Reference to storage or memory device can refer to any type of volatile or non-volatile memory); and a control and inference circuit (Fig. 5, Inference engine 510; Paragraph 0020, processing system 504 can use inference engine 510 to perform inference operations), wherein: the first persistent memory is connected to the control and inference circuit by a wideband data connection (Fig. 5, Media interface 545 is PCIe wideband connection coupled to inference engine 510; Paragraph 0041, Media interface 545 can be compatible with one or more of: DDR4, DDR5, PCIe… Paragraph 0094, High speed interconnects can be… Peripheral Component Interconnect express (PCIe)… Paragraph 0097, high-speed link 1018, such as a 40 Gigabit/second (Gb/s)); and the connector is suitable for connecting the device to a mobile computing device (Fig. 4, Memory board with inference is a card capable of connecting to a smartphone, tablets, personal computers which are mobile (i.e. they can be physically moved, such as a laptop); Paragraph 0099, Embodiments herein may be implemented in various types of computing, smart phones, tablets, personal computers).
Regarding claim 18, Shah teaches the device of claim 17. Shah teaches the device further comprising wherein: the control and inference circuit comprises a first persistent memory controller (Figs. 5 and 8, State machine 516 is a microprocessor that controls storage operations for persistent media 550 and is synonymous with memory controller 822 of Figure 8; Paragraph 0020, State machine 516 can be implemented as a microprocessor… Paragraph 0066, state machine issues a Read command for an inference model (and optionally, weights) and decoded data from the media… Paragraph 0086, memory controller to generate and issue commands to memory 830); the device further comprises: an interface circuit (Fig. 8, Interface 812), a second persistent memory (Fig. 8, Storage 884 is a second persistent memory; Paragraph 0090, storage device(s) 884, which can be or include any conventional medium for storing large amounts of data in a nonvolatile manner, such as one or more magnetic, solid state), and a second persistent memory controller (Fig. 8, Controller 882; Paragraph 0090, storage subsystem 880 includes controller 882 to interface with storage 884); the second persistent memory is connected to the second persistent memory controller (Fig. 8, Controller 882 connected to storage 884); the control and inference circuit is connected to the interface circuit (Fig. 8, Memory subsystem 820 is the control and inference circuit and is coupled to interface 812); and the second persistent memory controller is connected to the interface circuit (Fig. 8, Controller 882 is coupled to interface 814 which further transfers data to interface 812 and thus are connected to each other; Paragraph 0088, system 800 includes interface 814, which can be coupled to interface 812).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 8 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Shah (US 2020/0167098) in view of Schardt (US 2023/0061268).
Regarding claim 8, Shah teaches the system of claim 1. Shah teaches the system further comprising: a first interface circuit (Fig. 5, Interface 503), and an artificial intelligence accelerator (Fig. 5, Host 502 is an AI accelerator; Paragraph 0019, host 502 can include one or more processors (e.g., CPU, cores, GPU, accelerators… Paragraph 0047, Host 502 can issue a compute or storage command to processing system 504… Paragraph 0048, compute command can include or be associated with an inference model (e.g., model type) and include or reference weights stored in media 550), wherein: the artificial intelligence accelerator is configured to perform inference operations of a neural network, with assistance from: the first persistent memory, and the control and inference circuit (Fig. 5, Host 502 transmits compute commands to perform inference of a neural network to inference engine 510 and media 550; Paragraph 0048, compute command can include or be associated with an inference model (e.g., model type) and include or reference weights stored in media 550… Paragraph 0085, Accelerators 842 can provide multiple neural networks, CPUs, processor cores, general purpose graphics processing units, or graphics processing units can be made available for use by artificial intelligence (AI) or machine learning (ML) models); and the first persistent memory and the control and inference circuit are configured to perform an operation selected from the group consisting of compression (Fig. 5, Inference engine 510 and media 550 used for compressing/decompressing; Paragraph 0020, Decode and crop engine 512 can in addition, or alternatively, decompress data stored in media 550. Decompression can include decompressing media (e.g., video, still image, or audio)… Paragraph 0085, an accelerator among accelerators 842 can provide compression (DC) capability).
Shah does not teach the system comprising to perform an operation selected from the group consisting of pruning, sparsity, compression, quantization, and approximation.
Schardt teaches the system comprising to perform an operation selected from the group consisting of pruning, sparsity, compression, quantization, and approximation (Paragraph 0019, Examples of compression techniques include pruning, quantization, low-rank approximation and sparsity).
Shah and Schardt are analogous arts because they are in the same field of endeavor of performing compression via inference using neural networks.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s system to incorporate the teachings of Schardt and enable the host accelerator of Shah to perform pruning, sparsity, compression, quantization, and approximation via artificial intelligence operations.
One of ordinary skill in the art would be motivated to make the modifications in order to perform model compression to simplify large, complex models to produce lightweight counterpart models (See Schardt: Paragraph 0018).
Regarding claim 19, Shah teaches the device of claim 17. Shah teaches the device comprising wherein: the first persistent memory and the control and inference circuit are configured to perform an operation selected from the group consisting of compression (Fig. 5, Inference engine 510 and media 550 used for compressing/decompressing; Paragraph 0020, Decode and crop engine 512 can in addition, or alternatively, decompress data stored in media 550. Decompression can include decompressing media (e.g., video, still image, or audio)… Paragraph 0085, an accelerator among accelerators 842 can provide compression (DC) capability).
Shah does not teach the device comprising to perform an operation selected from the group consisting of pruning, sparsity, compression, quantization, and approximation.
Schardt teaches the device comprising to perform an operation selected from the group consisting of pruning, sparsity, compression, quantization, and approximation (Paragraph 0019, Examples of compression techniques include pruning, quantization, low-rank approximation and sparsity).
Shah and Schardt are analogous arts because they are in the same field of endeavor of performing compression via inference using neural networks.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s device to incorporate the teachings of Schardt and enable the host accelerator of Shah to perform pruning, sparsity, compression, quantization, and approximation via artificial intelligence operations.
One of ordinary skill in the art would be motivated to make the modifications in order to perform model compression to simplify large, complex models to produce lightweight counterpart models (See Schardt: Paragraph 0018).
Claims 9-12 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shah (US 2020/0167098) in view of Kwak (US 2012/0317332).
Regarding claim 9, Shah teaches the system of claim 1. Shah teaches the system comprising wherein: the control and inference circuit is part of a second semiconductor die (Fig. 5, Inference engine 510 is an integrated circuit die; Paragraph 0040, Inference engine 510 can include application specific integrated circuits (ASICs), Intel processor(s), Advanced RISC Machines (ARM) compatible processor(s), CPUs, GPUs).
Shah does not teach the system comprising wherein: the first persistent memory is part of a first semiconductor die.
Kwak teaches the system comprising wherein: the first persistent memory is part of a first semiconductor die (Fig. 2, Persistent memory die 110; Paragraph 0039, a non-volatile memory package 110).
Shah and Kwak are analogous arts because they are in the same field of endeavor of creating integrated systems between random access memory, memory controllers, and persistent memory.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s system to incorporate the teachings of Kwak and include the persistent memory and random access memory as semiconductor chips within a stacked package with the control and inference chip of Shah.
One of ordinary skill in the art would be motivated to make the modifications in order to improve thermal characteristics, increase reliability, and reduce footprint in semiconductor systems (See Kwak: Paragraphs 0003 and 0036).
Regarding claim 10, Shah teaches the system of claim 1. Shah teaches the system comprising wherein: the control and inference circuit is part of a second semiconductor die (Fig. 5, Inference engine 510 is an integrated circuit die; Paragraph 0040, Inference engine 510 can include application specific integrated circuits (ASICs), Intel processor(s), Advanced RISC Machines (ARM) compatible processor(s), CPUs, GPUs).
Shah does not teach the system comprising wherein: the first persistent memory is part of a first semiconductor die; and the first semiconductor die and the second semiconductor die are part of a stack of dies.
Kwak teaches the system comprising wherein: the first persistent memory is part of a first semiconductor die (Fig. 2, Persistent memory die 110; Paragraph 0039, a non-volatile memory package 110); and the first semiconductor die and the second semiconductor die are part of a stack of dies (Fig. 2, Stack of dies; Paragraph 0039, the non-volatile memory package 110 is stacked on the controller package 120 in a package-on-package (PoP) manner in FIG. 2).
Shah and Kwak are analogous arts because they are in the same field of endeavor of creating integrated systems between random access memory, memory controllers, and persistent memory.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s system to incorporate the teachings of Kwak and include the persistent memory and random access memory as semiconductor chips within a stacked package with the control and inference chip of Shah.
One of ordinary skill in the art would be motivated to make the modifications in order to improve thermal characteristics, increase reliability, and reduce footprint in semiconductor systems (See Kwak: Paragraphs 0003 and 0036).
Regarding claim 11, Shah teaches the system of claim 1. Shah teaches the system comprising a random-access memory (Fig. 5, Memory 518), wherein: the control and inference circuit is part of a second semiconductor die (Fig. 5, Inference engine 510 is an integrated circuit die; Paragraph 0040, Inference engine 510 can include application specific integrated circuits (ASICs), Intel processor(s), Advanced RISC Machines (ARM) compatible processor(s), CPUs, GPUs).
Shah does not teach the system comprising wherein: the first persistent memory is part of a first semiconductor die; the random access memory is part of a third semiconductor die; and the first semiconductor die, the second semiconductor die, and the third semiconductor die are part of a stack of dies.
Kwak teaches the system comprising wherein: the first persistent memory is part of a first semiconductor die (Fig. 2, Persistent memory die 110; Paragraph 0039, a non-volatile memory package 110); the random access memory is part of a third semiconductor die (Fig. 2, DRAM die 126; Paragraph 0044, auxiliary memory unit 126 may be a dynamic random access memory (DRAM)); and the first semiconductor die (Fig. 2, Die 110), the second semiconductor die (Fig. 2, Die 124), and the third semiconductor die (Fig. 2, Die 126) are part of a stack of dies (Fig. 2, Stack of dies; Paragraph 0039, the non-volatile memory package 110 is stacked on the controller package 120 in a package-on-package (PoP) manner in FIG. 2).
Shah and Kwak are analogous arts because they are in the same field of endeavor of creating integrated systems between random access memory, memory controllers, and persistent memory.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s system to incorporate the teachings of Kwak and include the persistent memory and random access memory as semiconductor chips within a stacked package with the control and inference chip of Shah.
One of ordinary skill in the art would be motivated to make the modifications in order to improve thermal characteristics, increase reliability, and reduce footprint in semiconductor systems (See Kwak: Paragraphs 0003 and 0036).
Regarding claim 12, Shah teaches the system of claim 1. Shah teaches the system comprising a random-access memory (Fig. 5, Memory 518), wherein: the control and inference circuit is part of a second semiconductor die (Fig. 5, Inference engine 510 is an integrated circuit die; Paragraph 0040, Inference engine 510 can include application specific integrated circuits (ASICs), Intel processor(s), Advanced RISC Machines (ARM) compatible processor(s), CPUs, GPUs).
Shah does not teach the system comprising wherein: the first persistent memory is part of a first semiconductor die; the random access memory is part of a third semiconductor die; and the first semiconductor die is stacked on: the second semiconductor die, and the third semiconductor die.
Kwak teaches the system comprising wherein: the first persistent memory is part of a first semiconductor die (Fig. 2, Persistent memory die 110; Paragraph 0039, a non-volatile memory package 110); the random access memory is part of a third semiconductor die (Fig. 2, DRAM die 126; Paragraph 0044, auxiliary memory unit 126 may be a dynamic random access memory (DRAM)); and the first semiconductor die (Fig. 2, Die 110 is stacked on top of dies 124 and 126; Paragraph 0039, the non-volatile memory package 110 is stacked on the controller package 120 in a package-on-package (PoP) manner in FIG. 2) is stacked on: the second semiconductor die (Fig. 2, Die 124), and the third semiconductor die (Fig. 2, Die 126) are part of a stack of dies (Fig. 2, Stack of dies).
Shah and Kwak are analogous arts because they are in the same field of endeavor of creating integrated systems between random access memory, memory controllers, and persistent memory.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s system to incorporate the teachings of Kwak and include the persistent memory and random access memory as semiconductor chips within a stacked package with the control and inference chip of Shah.
One of ordinary skill in the art would be motivated to make the modifications in order to improve thermal characteristics, increase reliability, and reduce footprint in semiconductor systems (See Kwak: Paragraphs 0003 and 0036).
Regarding claim 20, Shah teaches the device of claim 17. Shah teaches the device comprising wherein: the control and inference circuit is part of a second semiconductor die (Fig. 5, Inference engine 510 is an integrated circuit die; Paragraph 0040, Inference engine 510 can include application specific integrated circuits (ASICs), Intel processor(s), Advanced RISC Machines (ARM) compatible processor(s), CPUs, GPUs).
Shah does not teach the device comprising wherein: the first persistent memory is part of a first semiconductor die.
Kwak teaches the device comprising wherein: the first persistent memory is part of a first semiconductor die (Fig. 2, Persistent memory die 110; Paragraph 0039, a non-volatile memory package 110).
Shah and Kwak are analogous arts because they are in the same field of endeavor of creating integrated systems between random access memory, memory controllers, and persistent memory.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s device to incorporate the teachings of Kwak and include the persistent memory and random access memory as semiconductor chips within a stacked package with the control and inference chip of Shah.
One of ordinary skill in the art would be motivated to make the modifications in order to improve thermal characteristics, increase reliability, and reduce footprint in semiconductor systems (See Kwak: Paragraphs 0003 and 0036).
Claims 13 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Shah (US 2020/0167098) in view of Fujiwara (US 2023/0361081).
Regarding claim 13, Shah teaches the system of claim 1. Shah teaches the system comprising a random-access memory (Fig. 5, Memory 518), wherein: the control and inference circuit comprises: a persistent memory controller (Fig. 5, State machine 516 is a microprocessor that controls storage operations for persistent media 550; Paragraph 0020, State machine 516 can control the overall flow of the inference engine 510. State machine 516 can be implemented as a microprocessor… Paragraph 0066, state machine issues a Read command for an inference model (and optionally, weights) and decoded data from the media).
Shah does not teach the system comprising a multiply-accumulate circuit; the first persistent memory is part of a first semiconductor die; the persistent memory controller is part of a second semiconductor die; the multiply-accumulate circuit is part of a third semiconductor die; the random-access memory is part of a fourth semiconductor die; and the first semiconductor die is stacked on: the second semiconductor die, the third semiconductor die, and the fourth semiconductor die.
Fujiwara teaches the system (Fig. 1A, Semiconductor stack 100) comprising a multiply-accumulate circuit (Fig. 1A, Memory dies MD includes CIM dies that implement multiply-accumulate functions; Paragraph 0017, An example of applications of CIM is multiply-accumulate (“MAC”) operations… Paragraph 0033, memory dies MD may be or include CIM dies); the first persistent memory is part of a first semiconductor die (Fig. 1A, Memory dies MD further include flash memory die which is persistent memory; Paragraph 0033, memory dies MD may be or include CIM dies that can be implemented with a variety of memory devices, including… Flash); the persistent memory controller is part of a second semiconductor die (Fig. 1A, Core die CD is a flash memory controller; Paragraph 0032, the memory controller is formed on a surface of a semiconductor substrate (e.g., core die)); the multiply-accumulate circuit is part of a third semiconductor die (Fig. 1A, Multiply-accumulate circuit is a CIM die which is part of the MD stack); the random-access memory is part of a fourth semiconductor die (Fig. 1A, MD stack includes SRAM; Paragraph 0033, memory dies MD may be or include CIM dies that can be implemented with a variety of memory devices, including charge based memory such as static random-access memory (“SRAM”)); and the first semiconductor die is stacked on: the second semiconductor die, the third semiconductor die, and the fourth semiconductor die (Fig. 1A, Upper most die UMD can be flash die (i.e. first semiconductor die that is persistent memory) which is stacked on core die CD (i.e. second semiconductor die that is memory controller), CIM die (i.e. third semiconductor die that is multiply-accumulator), and SRAM die (i.e. fourth semiconductor die that is random-access memory); Paragraph 0024, plurality of memory dies MD is stacked layer-by-layer to form a 3D architecture and is coupled to the core die CD… Paragraph 0034, FIG. 1A, the first conductive pillar TSVI also connects the upmost memory die UMD).
Shah and Fujiwara are analogous arts because they are in the same field of endeavor of creating integrated systems for machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s system to incorporate the teachings of Fujiwara and include a multiply-accumulator die, and have the persistent memory of Shah in a first semiconductor die stacked on top of multiply-accumulator die, memory controller die, and random-access memory die.
One of ordinary skill in the art would be motivated to make the modifications in order to implement complex neural network operations while enabling faster data transfer and power efficiency (See Fujiwara: Paragraphs 0003 and 0016-0021).
Regarding claim 16, Shah teaches the method of claim 14. Shah teaches the method comprising wherein: the performing of the inference operation comprises performing the inference operation in a system (Fig. 5, System; Paragraph 0019, IG. 5 depicts an example inference and storage system) comprising: the persistent memory (Fig. 5, Media 550 includes persistent memory such as SSD; Paragraph 0046, media 550 includes SSDs 1-8); and a control and inference circuit (Fig. 5, Inference engine 510), wherein: the persistent memory is connected to the control and inference circuit by the wideband data connection (Fig. 5, Media 550 (i.e. first persistent memory) is connected to inference engine 510 via PCIe interface 545 which is a wideband data connection; Paragraph 0041, Media interface 545 can be compatible with one or more of: DDR4, DDR5, PCIe… Paragraph 0094, High speed interconnects can be… Peripheral Component Interconnect express (PCIe)… Paragraph 0097, high-speed link 1018, such as a 40 Gigabit/second (Gb/s)); and the control and inference circuit comprises: a persistent memory controller (Fig. 5, State machine 516 is a microprocessor that controls storage operations for persistent media 550; Paragraph 0020, State machine 516 can control the overall flow of the inference engine 510. State machine 516 can be implemented as a microprocessor… Paragraph 0066, state machine issues a Read command for an inference model (and optionally, weights) and decoded data from the media).
Shah does not teach the method comprising the system a multiply-accumulate circuit.
Fujiwara teaches the method comprising the system (Fig. 1A, Semiconductor stack 100) comprising a multiply-accumulate circuit (Fig. 1A, Memory dies MD includes CIM dies that implement multiply-accumulate functions; Paragraph 0017, An example of applications of CIM is multiply-accumulate (“MAC”) operations… Paragraph 0033, memory dies MD may be or include CIM dies).
Shah and Fujiwara are analogous arts because they are in the same field of endeavor of creating integrated systems for machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shah’s method to incorporate the teachings of Fujiwara and include a multiply-accumulator die.
One of ordinary skill in the art would be motivated to make the modifications in order to implement complex neural network operations while enabling faster data transfer and power efficiency (See Fujiwara: Paragraphs 0003 and 0016-0021).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US PGPUB 2019/0213166 to Petkov discloses that PCIe is a well-known serial protocol (Paragraph 0006, Component Interconnect Express (“PCIe”); see, e.g., PCI Express Base Specification Revision 4.0 dated Oct. 5, 2017 (“PCIe Specification”), which is incorporated herein by reference in its entirety. PCIe is a high-speed serial computer expansion bus standard).
US PGPUB 2014/0281240 to Willhalm discloses that a conventional computer system uses flash memory for persistent storage (Paragraph 0005, a conventional computer system typically uses flash memory devices to store persistent system information).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HARRY Z WANG whose telephone number is (571)270-1716. The examiner can normally be reached 9 am - 3 pm (Monday-Friday).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henry Tsai can be reached at 571-272-4176. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.Z.W./Examiner, Art Unit 2184
/HENRY TSAI/Supervisory Patent Examiner, Art Unit 2184
Prosecution Insights