DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the communication filed 11/28/2023.
Claims 1-14 are presented for examination.
Examiner Notes
Examiner cites particular columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in entirely as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d) or (f).
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/28/2023 and 2/24/2026. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Collier (US 20210374208 A1) in view of Shah et al. (US 20210342673 A1, hereafter Shah) and Lo et al. (US 20200210839 A1, hereafter Lo).
Regarding to claim 1, Collier discloses: A method for machine-learning parallelization using host CPUs of a multi-socket structure, performed by an apparatus for machine-learning parallelization using host CPUs of a multi-socket structure (see Fig. 1, [0025]; “the computer system 100 may perform processing to compute the product of two matrixes … the matrix-matrix multiplication may be used for a number of different applications, such as machine learning … As part of this execution, the computations are performed in a parallel fashion by the processing threads”. Also see [0021], [0023]; “a NUMA architecture-based computer system 100 includes processor sockets 110, NUMA nodes 114, and processing threads. More specifically, in accordance with some implementations, the computer system 100 may include multiple processor sockets 110, where, in this context, a “socket” refers to a CPU semiconductor package (or “chip”)” and “a given CPU core 120 may execute a processing thread in parallel with other CPU cores 120 that are executing other processing threads”), comprising:
performing a compile phase and (see [0001], [0025]; “a specialized math library that provides programs to support arithmetic operations in a wide variety of engineering, data mining, numeric processing, data analytics and machine learning applications. One such program may implement a version of a generalized matrix-matrix multiplication (GEMM) algorithm for purposes of performing matrix-matrix multiplication” and “the computer system 100 may perform processing to compute the product of two matrixes … the matrix-matrix multiplication may be used for a number of different applications, such as machine learning”. It is understood that in order to actually execute or run a software program described by [0001] and [0025] at computing fields, there will be at least performing a compile phase),
a learning model is split at [a layer level] for respective pipeline stages and allocated to Non-Uniform Memory Access (NUMA) nodes for respective CPU sockets (see [0017], [0019]-[0020]; “a matrix-matrix multiplication are partitioned in a way to divide up the processing workload of the matrix-matrix multiplication for better performance … The first phase involves considering different ways in which matrices involved in the matrix-matrix multiplication may be partitioned, with each way being considered a potential, or candidate, decomposition of matrices (also called a “candidate matrix decomposition” herein). The candidate matrix decompositions correspond to different workload distributions among the sockets”, “to determine how to distribute the workload among the NUMA nodes of each socket” and “to determine how to distribute the processing workload among processing threads”. Note: based on the feature of matrix-matrix multiplication can be used for machine learning application discussed at [0025], the splitting of matrix-matrix multiplication discussed at [0017] can be splitting a learning model of the machine-learning model); and
performing a runtime phase in which multiple threads generated in consideration of a policy of each parallelism algorithm are executed by being allocated to multiple cores included in the NUMA node (see [0020]; “to determine how to distribute the processing workload among processing threads … involve evaluating candidate thread decompositions and selecting the candidate thread decomposition that has the lowest processing cost according to a fitness function”. Also see [0016] and [0025]; “For matrix-matrix computations, the computer system may employ a generalized matrix-matrix multiplication (GEMM) algorithm that relies on different processing threads (on corresponding NUMA nodes and sockets)” and “As part of this execution, the computations are performed in a parallel fashion by the processing threads”).
Collier does not disclose: the compile phase in which the learning model is split at a layer level and allocated to NUMA nodes (note: again, Collier actually does teach features of a compile phase, and splitting or partitioned learning model and allocating the partitioned learning model to NUMA nodes for CPU sockets, but Collier does not specify such splitting and allocating is performed within the compile phase);
performing a runtime phase in which parameters required for learning are initialized.
However, Shah discloses: performing a compile phase in which a learning model is split at a layer level for respective pipeline stages and allocated to worker nodes for respective CPU resources (see [0022]-[0023]; “A compiler receives 110 a description of a machine learning network and generates the computer program that implements the machine learning network. The compiler allocates 112 computations to different processing elements in the MLA for execution such that different groups of processing elements implement different layers of the machine learning network”, “the compiler can determine which instructions are executed by which processing elements at what times”. Also see [0058]; “The compiler 320 may also optimize 324 the computer program 350 … Partitioning 328 concerns mapping the computations in the MLN to an implementation on the MLA. This includes determining which computations are allocated to which Tiles and how data flows through the mesh of Tiles during computation”)
It would have been obvious to one with ordinary skill, in the art before the effective filing date of the claim invention, to modify the compiling phase, task decomposition and allocation performed from Collier by including compiling phase to including allocating different layers of a machine learning workload to different processor resources from Shah, since it would provide a specific compilating phase to ensure data transfer instruction are not conflicting (see [0023] from Shah; “For each data transfer, the compiler analyzes all possible data transfer paths and generates a non-colliding data transfer path. Because the compiler can determine these at compile-time rather than at run-time, it can ensure that the data transfer instructions are not conflicting”).
In addition, Lo discloses: performing a runtime phase in which parameters required for learning are initialized (see [0091], “parameters, such as weights and biases, of the neural network can be initialized. As one example, the weights and biases can be initialized to random normal-precision floating-point values. As another example, the weights and biases can be initialized to normal-precision floating-point values that were calculated from an earlier training set”. Note: although the description of [0091] does not actually state that [0091] is performed during the runtime phase, according to the context of [0090]-[0100], it is understood that method 600 including the parameter initialization step is performed during the runtime phase of the neural network, i.e., the phase of training a neural network, instead of compiling phase of the neural network. Actually, “the weights and biases can be initialized to normal-precision floating-point values that were calculated from an earlier training set” from [0091] also implies that the parameter initialization step is performed during runtime phase).
It would have been obvious to one with ordinary skill, in the art before the effective filing date of the claim invention, to modify the runtime phase performed from the combination of Collier and Shah by including runtime phase to initialize different parameter required for the neural network model from Lo, and thus the combination of Collier, Shah and Lo disclose the missing limitations from Collier, since it would provide a flexible system that is able initializing operation parameter via different initialization mechanisms (see [0091] from Lo; “As one example, the weights and biases can be initialized to random normal-precision floating-point values. As another example, the weights and biases can be initialized to normal-precision floating-point values that were calculated from an earlier training set”).
Regarding to Claim 2, the rejection of Claim 1 is incorporated and further the combination of Collier, Shah and Lo discloses:
the NUMA node for each of the CPU sockets includes a CPU, including multiple cores, and memory (see Fig. 1, [0014]-[0015] and [0021] from Collier; “a NUMA architecture recognizes that processing nodes have faster access times to local memories than to non-local memory” and “a “socket” refers to a CPU semiconductor package (or “chip”) … The socket 110, or CPU semiconductor package, contains one or multiple CPU cores 120 (also called “processing cores” herein), as well as additional circuits, such as caches, interconnects, and a local memory 116”),
the multiple cores share the memory via an interconnect between the cores (see Fig. 1, [0014]-[0015] and [0021] from Collier; “The socket 110, or CPU semiconductor package, contains one or multiple CPU cores 120 (also called “processing cores” herein), as well as additional circuits, such as caches, interconnects, and a local memory 116”), and
the NUMA node for each of the CPU sockets shares memory of each NUMA node via an interconnect between the sockets (see [0044], [0051] from Collier; “the engine 140 assigns NUMA node masks for shared buffer allocations”, “the GEMM engine 134 uses shared data structures that are explicitly local to a given NUMA node 114, if possible, to help further improve data locality and reduce contention”).
Regarding to Claim 8, Claim 8 is a system claim corresponds to method Claim 1 and is rejected for the same reason set forth in the rejection of Claim 1 above (note: also see Fig. 11 and [0054] of Collier for claimed “a processor” and “memory”).
Regarding to Claim 9, Claim 9 is a system claim corresponds to method Claim 2 and is rejected for the same reason set forth in the rejection of Claim 2 above.
Claims 3 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Collier (US 20210374208 A1) in view of Shah et al. (US 20210342673 A1, hereafter Shah) and Lo et al. (US 20200210839 A1, hereafter Lo) and further in view of Singh et al. (US 20190102419 A1, hereafter Singh) and Meng et al. (CN110704542A -English translation provided by Google Patents).
Regarding to Claim 3, the rejection of Claim 1 is incorporated, the combination of Collier, Shah and Lo does not disclose: a default value for a number of pipeline stages is set to correspond to a number of NUMA nodes, and an equal number of model operations is distributed to each of the NUMA nodes.
However, Singh discloses: a default value for a number of parallelism stages is set to correspond to a number of [NUMA] nodes (see [0024]; “By applying a forced high degree of parallelism at steps 114 and 116, an embodiment of the present invention achieves increased and improved performance. The forced parallelism enables the system to distribute the processing to a full number of data nodes, as opposed to a small subset”).
It would have been obvious to one with ordinary skill, in the art before the effective filing date of the claim invention, to modify the parallelism level from the combination of Collier, Shah and Lo by including executing under a forced high degree of parallelism from Singh, since it would provide a mechanism of utilizing all of worker nodes for achieve highest parallelism (see [0024] from Singh).
In addition, Meng discloses: an equal number of model operations is distributed to each of the [NUMA] nodes (see [0105]; “In the data pre-partitioning stage, the default partitioning mode of the MemSql is that the number of partitions of each node is the same”).
It would have been obvious to one with ordinary skill, in the art before the effective filing date of the claim invention, to modify the initial workload distribution from the combination of Collier, Shah, Lo and Singh by including evenly distributing same amount of operations to each worker node from Meng, and thus the combination of Collier, Shah, Lo, Singh and Meng would disclose the missing limitations of Collier, Shah and Lo, since it would provide an default initial workload distribution method (see [0104]-[0105] from Meng).
Regarding to Claim 10, Claim 10 is a system claim corresponds to method Claim 3 and is rejected for the same reason set forth in the rejection of Claim 3 above.
Claims 4, 6-7 and 11 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Collier (US 20210374208 A1) in view of Shah et al. (US 20210342673 A1, hereafter Shah) and Lo et al. (US 20200210839 A1, hereafter Lo) and further in view of Zheng et al. (US 20220044112 A1, hereafter Zheng).
Regarding to Claim 4, the rejection of Claim 1 is incorporated, the combination of Collier, Shah and Lo does not disclose: wherein the parameters include global parameters for sharing data between the multiple threads and local parameters used individually by each of the multiple threads.
However, Zheng discloses: wherein the parameters include global parameters for sharing data between the multiple threads and local parameters used individually by each of the multiple threads (see [0005]; “the computing system may use the worker threads to perform training operations. The training operations may comprise generating, for each of the trainers, an updated local version of the parameters using the worker thread associated with that trainer. While the worker threads are performing training operations, the computing system may use the synchronization threads to perform synchronization operations. In particular embodiments, the synchronization operations may comprise generating a global version of the parameters based on the updated local versions of the parameters and generating, for each of the trainers, a synchronized local version of the parameters based on the global version of the parameters”. Note: the claim 4 now only requires there are global parameters that are shared among multiple threads and local parameters used by each individual thread. The current claimed language from claim 4 does not require the claimed global parameters and the claimed local parameters are different types of parameters indicating different meanings. Thereby, feature related to global version of parameters and local version of parameters from Zheng is still reasonable to teach what is claimed now under BRI. Also see [0002], [0044]; “In parallel computing, all processors may have access to a shared memory to exchange information between processors. In distributed computing, each processor has its own private memory (distributed memory)” and “The trainers are the workers who control the training loop. In particular embodiments, the trainers may be associated with a shared reader service. The shared reader service may convert a training example to a feature representation used for training the machine-learning model. Each trainer connects to a shared reader service. It has a local queue that fetches new batch of examples from the reader service. The reader service is a distributed system which consumes the raw data in the distributed storage, and then converts the raw data to feature tensors after the feature engineering step so that the trainers can focus on training without being bottlenecked on the data reading”, “In the model parallelism regime, the worker threads may access the shared parameters in the embedding PSs”).
It would have been obvious to one with ordinary skill, in the art before the effective filing date of the claim invention, to modify the runtime phase performed from the combination of Collier, Shah and Lo by including synchronizations of threads for different training stages/portions to update data or parameter among different training stages/portions from Zheng, and thus the combination of Collier, Shah, Lo and Zheng would disclose the missing limitations from the combination of Collier, Shah and Lo, since it would provide a mechanism of training complicated models to shorten the training time (see [0004] from Zheng).
Regarding to Claim 6, the rejection of Claim 4 is incorporated and further the combination of Collier, Shah, Lo and Zheng discloses:
synchronizing execution of the threads allocated to each of the NUMA nodes (see [0005] from Zheng; “While the worker threads are performing training operations, the computing system may use the synchronization threads to perform synchronization operations. In particular embodiments, the synchronization operations may comprise generating a global version of the parameters based on the updated local versions of the parameters and generating, for each of the trainers, a synchronized local version of the parameters based on the global version of the parameters”. Also see [0017], [0019]-[0020] from Collier; “The candidate matrix decompositions correspond to different workload distributions among the sockets”, “to determine how to distribute the workload among the NUMA nodes of each socket” and “to determine how to distribute the processing workload among processing threads”. At the combination system, the different training operations are distributed into different NUMA nodes, and thus after combining feature of thread synchronizations among trainers from Zheng into the combination of Collier, Shah and Lo, the thread synchronization among trainers is actually thread synchronization among NUMA nodes); and
updating the parameters for each of the NUMA nodes based on the global parameters (see [0005] from Zheng; “generating, for each of the trainers, a synchronized local version of the parameters based on the global version of the parameters”).
Regarding to Claim 7, the rejection of Claim 6 is incorporated and further the combination of Collier, Shah, Lo and Zheng discloses: wherein updating the parameters comprises updating the parameters using any one of a method in which the multiple threads synchronously update the parameters and a method in which the multiple threads asynchronously update the parameters (see [0005] from Zheng; “While the worker threads are performing training operations, the computing system may use the synchronization threads to perform synchronization operations … generating, for each of the trainers, a synchronized local version of the parameters based on the global version of the parameters”).
Regarding to Claim 11, Claim 11 is a system claim corresponds to method Claim 4 and is rejected for the same reason set forth in the rejection of Claim 4 above.
Regarding to Claim 13, Claim 13 is a system claim corresponds to method Claim 6 and is rejected for the same reason set forth in the rejection of Claim 6 above.
Regarding to Claim 14, Claim 14 is a system claim corresponds to method Claim 7 and is rejected for the same reason set forth in the rejection of Claim 7 above.
Claims 5 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Collier (US 20210374208 A1) in view of Shah et al. (US 20210342673 A1, hereafter Shah), Lo et al. (US 20200210839 A1, hereafter Lo) and Zheng et al. (US 20220044112 A1, hereafter Zheng) and further in view of Cha et al. (US 20230153624 A1, hereafter Cha).
Regarding to Claim 5, the rejection of Claim 4 is incorporated, the combination of Collier, Shah, Lo and Zheng does not disclose: wherein the local parameters store a gradient for loss and a state of an optimizer for determining whether to apply the gradient, which are used in a backpropagation process of the learning model.
However, Cha discloses: wherein the local parameters store a gradient for loss (see [0097] and [0109]; “the backpropagation performing unit 184 may update the first parameter PAR1 and the second parameter PAR2 for each layer of the neural network by using a gradient (a differential value) of each loss function”) and a state of an optimizer for determining whether to apply the gradient, which are used in a backpropagation process of the learning model (see [0095]; “adjust a loss LS for the prediction value PVL so as to be a minimum value. When the loss LS becomes the minimum value, the prediction value PVL may be output as a tracking result for the object”. Also see [0097] and [0106]-[0109]; “perform backpropagation based on the loss LS adjusted by the loss adjusting unit 182. For example, the backpropagation performing unit 184 may update the first parameter PAR1 and the second parameter PAR2 for each layer of the neural network by using a gradient (a differential value) of the loss function”. The backpropagation operation discussed at [0097] and [0109] to apply the gradient for loss function is performed based on the adjusted loss LS, and thus the “minimum value” for the adjusted LS is reasonable to be considered as local parameter that indicates state of an optimizer for determining whether to apply the gradient used in a backpropagation process).
It would have been obvious to one with ordinary skill, in the art before the effective filing date of the claim invention, to modify the local parameters of each trainer or layers and loss parameter determination from the combination of Collier, Shah, Lo and Zheng (note: see [0066]-[0078] from Zheng about determining loss values for the trainers) by including backpropagation operation based on gradient of loss function from Cha, and thus the combination of Collier, Shah, Lo, Zheng and Cha would disclose the missing limitations from the combination of Collier, Shah, Lo and Zheng, since it would provide a mechanism of continuing reduce loss function to achieve minimal loss (see [0095]-[0097] and [0106]-[0109] from Cha).
Regarding to Claim 12, Claim 12 is a system claim corresponds to method Claim 5 and is rejected for the same reason set forth in the rejection of Claim 5 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Mukundan et al. (US 20240069969 A1) discloses: launch deep learning jobs with a NUMA binding of their choice. In at least one embodiment, a number of NUMA nodes per socket is variable based on architecture of a particular cluster (see [0068]), scheduler can allocate work to clusters of processing array using various scheduling and/or work distribution algorithms, which may vary depending on a workload arising for each type of program or computation. In at least one embodiment, scheduling can be handled dynamically by scheduler, or can be assisted in part by compiler logic during compilation of program logic configured for execution by processing array (see [0335]), scheduler can be configured to divide a processing workload into approximately equal sized tasks, to better enable distribution of graphics processing operations to multiple clusters of processing array (see [0385]), a compiler is also included in OpenCL frame-work. Source code may be compiled offline prior to executing an application or online during execution of an application (see [0377]).
Wang et al. (US 11544113 B2) discloses: distributing and executing a receptive task from a set of tasks forming a machine learning workload on a host having NUMA sockets (see claim 1).
Khaitan et al. (US 20220083844 A1) discloses: A compiler analyzes a workload to be performed by the neural network and determines respective coarse-grained tensor instructions to be sent to each tensor processor cluster using a SIMD and/or single-program-multiple-data (SPMD) approach to distribute the workload (see [0029]).
Prabhakar et al. (US 11126574 B1) discloses: compile time logic configured to partition execution of the dataflow graph into two or more asynchronous stages, wherein each of the stages includes one or more compute nodes in the plurality of compute nodes; runtime logic configured with the compile time logic to determine a stage latency for each of the stages by calculating elapsed time between input stage buffers of a particular stage receiving a unit of data and output stage buffers of the particular stage receiving results of processing the unit of data through one or more compute nodes of the particular stage (see claim 1. Also see lines 19-33 of col. 3).
Gu et al. (US 20200151088 A1) discloses: compile the DNN to create algorithmic code including instructions for running the DNN on the processor cores (see [0035]), The compiling of the DNN 116 may include several stages. For example, it may include memory allocation, static scheduling, and convolution implementation (see [0102]), The scheduler 205 may map the layers of the DNN 116 to either the Conv processor 124 or the FC processor 126 for execution (see [0107]).
Ren (US 12462147 B2) discloses: a firing neural network computing system includes the parameter database is configured to store all parameter data of a network including initialization parameters and runtime parameters (see lines 4-22 of col. 3 and lines 60-62 of col. 15).
Nama et al. (US 20220309028 A1) discloses: Before the tiles 924a, . . . , 924d are written to the memory 140, a 20×20 space 921 in the memory 140 is initialized to zero, and reserved for or allocated to the tensor 920 (see [0172]).
Savic et al. (US 20190325302 A1) discloses: allocate memory for the global parameters assigned for handing by the PS service, and initialize the values (may be zero or randomized) (see [0055]).
Du (CN 113033784 A-English translation provided by Google Patents) discloses: performing a runtime phase in which parameters required for learning are initialized (see [0015]; “201) selecting the number of layers of the fixed super-net and the number of neurons of each layer … and randomly initializing parameters of the super-net”. Also see [0020], [0025]-[0026]; “using a recurrent neural network as a predictor … the structure is randomly initialized before searching”, “In step 6), initializing the parameters of the optimal network structure obtained in step 5)” and “selecting the optimal network structure obtained in the step 5), carrying out random initialization on the optimal network structure”).
Strimel et al. (US 11887583 B1) discloses: whether to further iterate training based on, for example, achieving a target loss value, detecting local minimum using a gradient descent algorithm, and/or until a predetermined number of iterations have been performed (see lines 48-53 of col. 15).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHI CHEN whose telephone number is (571)272-0805. The examiner can normally be reached on M-F from 9:30AM to 5:30PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, April Y Blair can be reached on 571-270-1014. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/Zhi Chen/
Patent Examiner, AU2196
/APRIL Y BLAIR/Supervisory Patent Examiner, Art Unit 2196