DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to pending claims 1-20 filed 4/3/2024.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Prakash (US 20240202521 A1).
For claim 1, Prakash discloses: a computer implemented method for training a large model using edge computing devices (fig.1 gives network and hardware overview), the method comprising:
generating, at the training controller, a training code chunk (0022-24) comprising
a component of the large model, wherein the component is individually trainable (0022: layers of a large model are divided by central server training controller and provided to edge devices 103-1, 103-N for individual training), and
a tuning code for the large model or the component (ibid: code for forward propagation and updating weights during backpropagation on edge devices, see 0024, 0041, 0052-53 (backpropagation phase), 0028: code is provided by the central server to coordinate between edge devices, hence, tuning code for tuning or training the large mode via respective components, see also 0052: providing loss function by the server to adjust the weights, hence, tuning code for adjusting the component);
generate, at the training controller, a plurality of training data chunks from training data, at least one training data chunk from the plurality of training data chunks capable of being processed by the training code chunk to train the component (0022-23: provision of training data chunks to edge devices);
generating, at the training controller, a chunk pair comprising the training code chunk and the training data chunk (0022-24, 0028, 0052: chunk code described above and chunk training data are provided by the training controller at the server, hence, chunk pair is generated and provided for each edge device);
sending, from the training controller to an edge node remote to the training controller, the chunk pair (ibid: chunk pair is provided to the edge nodes);
receiving, from the edge node at the training controller, a first processed training code chunk (0023-24, 0052-53: generating and sending activations as processed training code chunk); and
aggregating the first processed training code chunk with at least one second processed training code chunk to generate an updated large model (0024-25).
For claim 2, Prakash discloses the method of claim 1, as described above. Prakash further discloses: further comprising splitting the large model into a plurality of trainable components (0022).
For claim 3, Prakash discloses the method of claim 2, as described above. Prakash further discloses: wherein the component comprises at least one of a convolutional layer, a fully-connected layer (0012, fig.3: neural networks, ANN’s comprise fully connected layers), a pipeline stage unit (ibid: neural networks comprise pipeline stages), a tensor model unit (ibid: neural network layers comprise tensor model units), a data specific sub-unit (ibid: neural network layers are data specific unit), a distilled model, a sub-unit of the large model capable of being trained individually using training data (ibid: neural network layers), or the large model (ibid: components comprise portions of the large model).
For claim 4, Prakash discloses the method of claim 3, as described above. Prakash further discloses: wherein the aggregating comprises at least one of tensor model parallelism, pipeline parallelism (fig.3, 0022-24, 0017 contemplates layer-wise or pipeline parallelism), or model distillation.
For claim 5, Prakash discloses the method of claim 1, as described above. Prakash further discloses: wherein the aggregating comprises aggregating the first trained component and the second trained component in series or in parallel (0024: layers are aggregated, hence, series aggregation).
For claim 6, Prakash discloses the method of claim 1, as described above. Prakash further discloses: wherein the generating the chunk pair comprises generating the chunk pair according to at least one of the capability of the edge node or the availability of the edge node (0045, 0047: grouping edge nodes based on characteristics, computing capabilities; 0045: grouping based on geographic proximity, hence, availability with regards to an area, hence, generating instructions and layers ,etc. based on capabilities and availabilities of edge nodes).
For claim 7, Prakash discloses the method of claim 1, as described above. Prakash further discloses: wherein the tuning code utilizes at least one of tensor model parallelism, pipeline parallelism, transfer learning, tensor decomposition or discriminative fine-tuning (0052-53: pipeline parallelism).
For claim 8, Prakash discloses the method of claim 1, as described above. Prakash further discloses: at least one of
receiving a copy of the large model deployed on a large model server (0024: an updated large model is received for utilization or provision to a different computing device for utilization, hence, control unit for serving or providing large model receives the updated model), or
sending the updated large model to a large model server for deployment thereon (ibid: updated large model is sent to another device for utilization or deployment on that device, said device hence providing large model services).
For claim 9, Prakash discloses an apparatus corresponding to the method of claim 1, and is hence rejected for the same reasons. Prakash further discloses: a computing apparatus comprising:
a processor (fig.5:591); and
a memory storing instructions that, when executed by the processor, configure the apparatus (fig.5:597, 598, 593).
For claim 16, Prakash discloses a computer-readable medium corresponding to the method of claim 1, and is hence rejected for the same reasons. Prakash further discloses: a non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer perform the recited steps (fig.5, 0073).
Remaining claims 10-15, 17-20 recite apparatuses and computer media corresponding to the above methods and are hence likewise rejected.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Chen-Accept ("AccEPT: An Acceleration Scheme for Speeding Up Edge Pipeline-parallel Training", published 11/10/2023) discloses an edge-based training algorithm.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LIANG LI whose telephone number is (303)297-4263. The examiner can normally be reached Mon-Fri 9-12p, 3-11p MT (11-2p, 5-1a ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. The examiner is available for interviews Mon-Fri 6-11a, 2-7p MT (8-1p, 4-9p ET).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Jennifer Welch can be reached on (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/LIANG LI/
Primary examiner AU 2143