CTNF 18/527,627 CTNF 99031 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claims 1-20 are presented for examination. Claim Objections Claims 15 and 20 are objected to because of the following informalities, Claim 15 [line 5]: “the second segment of memory hardware is implemented the second processing unit” should be “the second segment of memory hardware is implemented on the second processing unit” Claim 20 [line 1]: “the non-transitory computer-readable medium storing instructions of claims 15 ” should be “the non-transitory computer-readable medium storing instructions of claims 17 ” Appropriate corrections are required. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sudarsanan et al (US 20230351144 A1) hereafter Sudarsanan, and further in view of Kim et al (US 20220066660 A1) hereafter Kim . With respect to claim 1 , Sudarsanan teaches a method comprising: first categorizing, according to a heuristic , weights of a neural network model into a first memory block and a second memory block (electronic device includes a memory storing one or more programs and plurality of registers, wherein a set of M-bit elements are stored in those registers, that then implement the neural network operation on the set of M-bit to generate a set of P-bit elements. One or more inputs are combined based on their corresponding weights and according to the propagation function. In an example of a corresponding neural network receives an input feature vector, M and P are equal to 32 and 8, respectively [par. 0004, 0057, 0071]) ; first storing individual weights of the weights first-categorized to the first memory block using a first number of memory bits (each set of elements is loaded from the memory. Electronic device 104 includes a mobile device, and the weight matrix W and bias vector b have quantized P-bit elements and are stored in the memory in association with the neural network layer. Electronic device implements neural network (NN) at low precision level using 8-bit fixed. The weights are quantized and stored at the low precision level, while the NN computation is implemented at high precision level [par. 0054, 0069, 0071, 0077]) ; first storing individual weights of the weights first-categorized to the second memory block using a second number of memory bits (when the propagation function is implemented at each artificial neuron, the weights are dequantized and stored at high precision level [par. 0077]) , wherein the first number of memory bits is smaller than the second number of memory bits ( basically, dequantization is the one of the instructions given when the weights with smaller bits are quantized to the weights with larger bits, for example, 8-bit to 16-bit, and vice versa . The process can also be quantizing weights from lower-precision level to higher-precision level [par. 0077]) ; second categorizing, according to the heuristic , the first-stored weights of the neural network model into the first memory block and the second memory block (each artificial neuron receives one or more node inputs those are combined based on their corresponding weights according to the propagation function. After the dequantization of first weights, the first weights may be stored in the memory devices to be quantized or to be dequantized in the next layer [par. 0004, 0057, 0063, 0064, 0071, 0080-0081, 0096]) ; second storing individual weights of the weights second-categorized to the first memory block using the first number of memory bits (the processor includes a load/store unit configured to execute load/store instructions related to the active instruction that generate virtual addresses and store operations and load data from memory or store it back to memory from the registers. The load/store unit may access the higher levels of caches or DRAM to extract data to be used in implementation [par. 0054, 0063, 0064]) ; and second storing individual weights of the weights second-categorized to the second memory block using the second number of memory bits (one or more vector registers store a set of first elements and a subset of the weighted combinations of node inputs of a neural network layer. After receiving the instructions, the linear activation operation is implemented on the set of first elements stored in and extracted from the one or more input vector registers to generate a set of output elements [par. 0054, 0063, 0064]) . However, Sudarsanan does not explicitly disclose categorizing, according to a heuristic. In the same field of endeavor, Kim teaches categorizing, according to a heuristic ( weights having relatively higher influence can be represented using higher precision . Each node of a layer is a calculation unit having one or more inputs and an output. A weight is a set for a connection between nodes that the weight may be adjusted or changed. The weight amplifies, reduces, or maintains a relevant data value that determines a degree of influence of the data value on a final result. That means higher influence equals to higher precision, and vice versa . The parameters of the NN including the weights may be learned in advance that may be performed in a high-precision format to secure the accuracy of the NN [par. 0053, 0054]) . It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated the concept of converting the input data of the first format into the second format by applying any one or any combination of any two or more of type converting, quantization and dequantization as suggested by Kim into the concept of loading each set of M-bit elements from the memory of the electronic device to respective input vector registers or storing each set of M-bit elements to the respective input vector registers in response to one or more preceding instructions as suggested by Sudarsanan because both of these systems addressing the process of performing dequantization to convert low-precision weights to high-precision weights. Doing so would be desirable because the system of Sudarsanan would be more efficient by determining a degree of influence of the data value on a final result, such that weighted inputs of nodes included in a previous layer may be input into each node included in the output layer (Kim, [par. 0053, 0054]). With respect to claim 2 , the combination of Sudarsanan and Kim teaches wherein: the first categorizing is responsive to completion of a first set of training iterations for the neural network model; and the second categorizing is responsive to completion of a second set of training iterations for the neural network model (Sudarsanan, the method includes receiving a single instruction to apply a linear activation operation to a set of first elements stored in the vector registers, in order to generate a set of output elements. The method further includes segmenting a first feature vector to a plurality of sets of first elements including the first set of the remaining sets of first elements, loading each set of first elements from the memory to respective input vector registers or storing each set of first elements to respective registers in response to one or more preceding instructions and repeating the single instruction to implement the linear activation operation [par. 0007-0009]) . With respect to claim 3 , the combination of Sudarsanan and Kim teaches wherein the heuristic comprises at least one of: a measurement quantifying a magnitude of a respective weight's influence on output of the neural network model during a most recent set of training iterations; and a measurement quantifying a magnitude of a respective weight's fluctuations in value during the most recent set of training iterations (Sudarsanan, weights having relatively higher influence can be represented using higher precision . Each node of a layer is a calculation unit having one or more inputs and an output. A weight is a set for a connection between nodes that the weight may be adjusted or changed. The weight amplifies, reduces, or maintains a relevant data value that determines a degree of influence of the data value on a final result. That means higher influence equals to higher precision, and vice versa . The parameters of the NN including the weights may be learned in advance that may be performed in a high-precision format to secure the accuracy of the NN [par. 0053, 0054]) . With respect to claim 4 , the combination of Sudarsanan and Kim teaches further comprising, during the second set of training iterations: performing intra-memory block matrix operations between the weights first- categorized into the first memory block (Sudarsanan, intra-memory matrix operations involving weights stored within the same memory segment . The vector processor multiplies the input feature vector with a weight matrix W to obtain an intermediate feature vector and adds a bias vector b to generate the first feature vector including the set of first elements. Each element includes a fixed-point P-bit number [par. 0071]) ; performing intra-memory block matrix operations between the weights first- categorized into the second memory block (Sudarsanan, the vector processor multiplies the input feature vector with a weight matrix W to obtain an intermediate feature vector and adds a bias vector b to generate the first feature vector including the set of first elements. Each element includes a fixed-point P-bit number [par. 0071]) ; and responsive to completion of the intra-memory block matrix operations, performing inter-memory block matrix operations between the weights first-categorized into the first memory block and the weights first-categorized into the second memory block (Sudarsanan, inter-memory matrix operations between weights of the first and the second blocks . The linear activation function is applied. The vector processor dequantizes each element of the input feature vector, weight matrix and bias vector to a fixed-point M-bit number. For example, M and P are equal to 32 and 8, respectively. The weight matrix and bias vector have quantized P-bit elements and are stored in the memory [par. 0071]) . With respect to claim 5 , the combination of Sudarsanan and Kim teaches wherein: the first storing the individual weights first-categorized to the first memory block using the first number of memory bits comprises first storing the individual weights first- categorized to the first memory block in individual sub-segments of a first segment of memory hardware, wherein a respective sub-segment of the first segment of memory hardware comprises the first number of memory bits (Sudarsanan, one or more input vector registers store a set of first elements or a subset of the weighted combinations of node inputs. The linear activation operation is implemented on the set of first elements stored in and extracted from the registers to generate the set of output elements. For each element, the processor detects a sign value of the respective element of the set of first elements and selects the respective scalar [par. 0054, 0064]) ; and the first storing the individual weights first-categorized to the second memory block using the second number of memory bits comprises first storing the individual weights first-categorized to the second memory block in individual sub-segments of a second segment of memory hardware, wherein a respective sub-segment of the second segment of memory hardware comprises the second number of memory bits (Sudarsanan, the processor temporarily stores the set of output elements in the output vector register, and then the processor extracts the set of output elements from the output vector register and continues to quantize the set of output elements [par. 0054, 0064]) . With respect to claim 6 , the combination of Sudarsanan and Kim teaches wherein: the first segment of memory hardware is implemented on a first streaming multiprocessor (SM) of a processing unit; and the second segment of memory hardware is implemented on a second SM of the processing unit (Kim, hardware components implemented by processor may execute instructions or software. A hardware component may have one or more different processing configurations, including parallel processors, wherein parallel processors are designed for massively parallel workloads like the SMs [par. 0088]) . With respect to claim 7 , the combination of Sudarsanan and Kim teaches further comprising: prior to the first set of training iterations for the neural network model, initially categorizing all weights of the neural network model into the first memory block; and initially storing individual weights of the weights initially-categorized to the first memory block using the first number of memory bits such that the neural network model has the initially-stored weights during the first set of training iterations (Kim, the storage device may first store the input data in high-precision format, and the operator may perform pre-processing on the input data prior to transmitting the input data to an accelerator for inference. The operator may perform quantization to convert the input data of the high-precision format into a low-precision format. The operator then receives the instruction to dequantize to convert the result data of low-precision format into high-precision format [par. 0072-0074]) . With respect to claim 8 , the combination of Sudarsanan and Kim teaches wherein: the first categorizing further comprises, first categorizing, according to the heuristic, weights of the neural network model into a third memory block (Kim, each node of a layer is a calculation unit having one or more inputs and an output. A weight is a set for a connection between nodes that the weight may be adjusted or changed. The weight amplifies, reduces, or maintains a relevant data value that determines a degree of influence of the data value on a final result. That means higher influence equals to higher precision, and vice versa . The parameters of the NN including the weights may be learned in advance that may be performed in a high-precision format to secure the accuracy of the NN [par. 0053, 0054]) .; the first storing further comprises, first storing individual weights of the weights first-categorized to the third memory block using a third number of memory bits, wherein the second number of memory bits is smaller than the third number of memory bits (Sudarsanan, when the propagation function is implemented at each artificial neuron, the weights are dequantized and stored at high precision level. Dequantization is the one of the instructions given when the weights with smaller bits are quantized to the weights with larger bits, for example, 8-bit to 16-bit, and vice versa . The process can also be quantizing weights from lower-precision level to higher-precision level [par. 0077]) ; the second categorizing further comprises, second categorizing, according to the heuristic, weights of the neural network model into the third memory block (the processor includes a load/store unit configured to execute load/store instructions related to the active instruction that generate virtual addresses and store operations and load data from memory or store it back to memory from the registers. The load/store unit may access the higher levels of caches or DRAM to extract data to be used in implementation [par. 0054, 0063, 0064]) ; and the second storing further comprises, second storing individual weights of the weights second-categorized to the third memory block using the third number of memory bits (one or more vector registers store a set of first elements and a subset of the weighted combinations of node inputs of a neural network layer. After receiving the instructions, the linear activation operation is implemented on the set of first elements stored in and extracted from the one or more input vector registers to generate a set of output elements [par. 0054, 0063, 0064]) . With respect to claim 9 , it is a system claim that is corresponding to the method of claim 1. Therefore, it is rejected for the same reason as claim in claim 1 above. With respect to claim 10 , it is a system claim that is corresponding to the method of claim 2. Therefore, it is rejected for the same reason as claim in claim 2 above. With respect to claim 11 , it is a system claim that is corresponding to the method of claim 3. Therefore, it is rejected for the same reason as claim in claim 3 above. With respect to claim 12 , it is a system claim that is corresponding to the method of claim 4. Therefore, it is rejected for the same reason as claim in claim 4 above. With respect to claim 13 , it is a system claim that is corresponding to the method of claim 5. Therefore, it is rejected for the same reason as claim in claim 5 above. With respect to claim 14 , it is a system claim that is corresponding to the method of claim 6. Therefore, it is rejected for the same reason as claim in claim 6 above. With respect to claim 15 , the combination of Sudarsanan and Kim teaches further comprising a first processing unit and a second processing unit, wherein: the first segment of memory hardware is implemented on the first processing unit; and the second segment of memory hardware is implemented the second processing unit (Sudarsanan, memory includes one or more storage devices remotely located from one or more processing units [par. 0035, 0036]) . With respect to claim 16 , it is a system claim that is corresponding to the method of claim 7. Therefore, it is rejected for the same reason as claim in claim 7 above. With respect to claim 17 , it is a non-transitory computer-readable medium claim that is corresponding to the method of claim 1. Therefore, it is rejected for the same reason as claim in claim 1 above. With respect to claim 18 , it is a non-transitory computer-readable medium claim that is corresponding to the method of claim 3. Therefore, it is rejected for the same reason as claim in claim 3 above. With respect to claim 19 , it is a non-transitory computer-readable medium claim that is corresponding to the method of claim 4. Therefore, it is rejected for the same reason as claim in claim 4 above. With respect to claim 20 , it is a non-transitory computer-readable medium claim that is corresponding to the method of claim 5. Therefore, it is rejected for the same reason as claim in claim 5 above . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Gong et al (US 20230118802 A1) disclosed systems, apparatuses and methods may provide technology for optimizing an inference neural network model that performs asymmetric quantization by generating a quantized neural network, wherein model weights of the neural network are quantized as signed integer values, and wherein an input layer of the neural network is configured to quantize input values as unsigned integer values, generating a weights accumulation table based on the quantized model weights and a kernel size for the neural network, and generating an output restoration function for an output layer of the neural network based on the weights accumulation table and the kernel size. The technology may also perform per-input channel quantization. The technology may also perform mixed-precision auto-tuning. Diamant et al (US 12499353 B1) disclosed systems and methods for performing hardware approximation of functions are provided. In one example, a system comprises a controller, a plurality of multiplexors, configurable arithmetic circuits, and a mapping table that stores a set of function parameters. According to a mode of operations, the controller may configure the plurality of multiplexors to forward the set of function parameters or a subset of the function parameters to the arithmetic circuits to compute an approximation result. In a case where the subset of the function parameters is forwarded to the arithmetic circuits, the controller may configure the arithmetic circuits to perform post-processing, such as quantization, of the approximation result. Chen et al (US 20250111213 A1) disclosed systems, methods, and computer program products are provided for saving memory during training of knowledge graph neural networks. The method includes receiving a training dataset including a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph, inputting the training dataset into a knowledge graph neural network to generate at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge graph, quantizing the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings, storing the at least one set of quantized knowledge graph embeddings in a memory, and dequantizing the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings. Chai et al (US 20210241108 A1) disclosed a system that generates and executes a deep neural network (DNN) based on target runtime parameters. During operation, the system receives a trained original model and a set of target runtime parameters for the DNN, wherein the target runtime parameters are associated with one or more of the following for the DNN: desired operating conditions, desired resource utilization, and desired accuracy of results. Next, the system generates a context-specific model based on the original model and the set of target runtime parameters. The system also generates an operational plan for executing both the original model and the context-specific model to meet requirements of the target runtime parameters. Finally, the system controls execution of the original model and the context-specific model based on the operational plan . 07-101 Any inquiry concerning this communication or earlier communications from the examiner should be directed to Quoc Phung whose telephone number is (703) 756 1330 . The examiner can normally be reached on Monday through Friday from 9am to 5pm PT. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.usp to. gov/interviewpractice . If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on 571-272-7212 . The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000 . /Q.L.P./Examiner, Art Unit 2143 /JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143 Application/Control Number: 18/527,627 Page 2 Art Unit: 2143 Application/Control Number: 18/527,627 Page 3 Art Unit: 2143 Application/Control Number: 18/527,627 Page 4 Art Unit: 2143 Application/Control Number: 18/527,627 Page 5 Art Unit: 2143 Application/Control Number: 18/527,627 Page 6 Art Unit: 2143 Application/Control Number: 18/527,627 Page 7 Art Unit: 2143 Application/Control Number: 18/527,627 Page 8 Art Unit: 2143 Application/Control Number: 18/527,627 Page 9 Art Unit: 2143 Application/Control Number: 18/527,627 Page 10 Art Unit: 2143 Application/Control Number: 18/527,627 Page 11 Art Unit: 2143 Application/Control Number: 18/527,627 Page 12 Art Unit: 2143 Application/Control Number: 18/527,627 Page 13 Art Unit: 2143 Application/Control Number: 18/527,627 Page 14 Art Unit: 2143 Application/Control Number: 18/527,627 Page 15 Art Unit: 2143 Application/Control Number: 18/527,627 Page 16 Art Unit: 2143
Prosecution Insights