Notice of Pre-AIA or AIA Status
The present application 18/247,447, filed on 3/31/2023 (or after March 16, 2013), is being examined under the first inventor to file provisions of the AIA (First Inventor to File).
In the event the determination of the status of the application as subject to AIA 35
U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application is a 371 of PCT/US2022/053601 filed on 12/21/2022
PCT/US2022/053601 has PRO 63/386,959 filed on 12/12/2022
DETAILED ACTION
Response to Amendment
Claims 1-20 are pending in this application.
Examiner acknowledges applicant’s amendment filed on 12/21/2025
Drawings
The Drawings filed on 3/14/2024 are acceptable for examination purpose.
Priority
Acknowledgment is made of applicant’s claim for domestic priority application
U.S. Provisional Patent application serial number # 63/386,959 filed on 12/12/2022
under 35 U.S.C. 119 (e)
35 USC § 112
In view of applicant’s remarks, the rejection under 35 USC § 112 as set forth in the previous office action is hereby withdrawn.
Response to Arguments
Applicant's arguments filed 12/21/2025 with respect to claims 1-20 have been fully considered but they are not persuasive, for examiner’s response, see discussion below:
Double Patenting
At page 9-10, examiner acknowledges applicant’s remarks on double patent rejection, however, applicant may consider filing terminal disclaimer to overcome the double patent rejection, subject to approval
35 USC § 101
At page 8-9, applicant argues:
Claim 1 and 15, which capture the server side of the training process, recite applying the server partition to the activations to produce output, then computing loss and gradient vectors to update the parameters.
According to MPEP 2106.04(a)(1), training a neural network is not an abstract idea. More specifically, example vii recites, among other limitations, "training the neural network in a first stage using the first training set" and "training the neural network in a second stage using the second training set". This example is presented as a claim that does "not recite (set forth or describe) an abstract idea". This point is reinforced in Example 39 of the 2019 Patent Eligibility Guidance (issued January 7, 2019), which includes a claim nearly identical to example vii of MPEP 2106.04(a)(1). In the Analysis, the Guidance states that "the claim does not recite a mental process because the steps are not practically performed in the human mind" (See page 9). This point is further reinforced in Example 47 of the 2024 Patent Eligibility Guidance (effective July 17, 2024), which includes claim 2 that recites "(c) training, by the computer, the ANN based on the input data and a selected training algorithm to generate a trained ANN, wherein the selected training algorithm includes a backpropagation algorithm and a gradient descent algorithm". The Guidance identifies certain mental processes recited in claim 2, but (c) is not among them (See page 7). The Guidance identifies (c) as encompassing a mathematical concept, but only because "backpropagation algorithm" and "gradient descent algorithm" are recited (See page 7). On the other hand, with respect to claim 1 of Example 47, the Guidance states "While ANNs may be trained using mathematics, there is no mathematical concept recited in the claim", before concluding that claim 1 does not recite any abstract ideas. In summary, MPEP 2106.04(a)(1), the 2019 Patent Eligibility Guidance, and the 2024 Patent Eligibility Guidance each either expressly teach or inherently support the assertion that training of a neural network is not, by itself, an abstract idea. Should the Office maintain that this recitation of claim 1 is an abstract idea, Applicant respectfully requests a detailed explanation of how this particular recitation of claim 1 differs from the examples cited above in MPEP 2106.04(a)(1), the 2019 Patent Eligibility Guidance, and the 2024 Patent Eligibility Guidance.
Therefore, the training operations cited above cannot be considered part of the abstract idea
Examiner’s response:
As best understood by the examiner, the additional element of “training, neural network model, computing a set of gradient vectors” may improve the accuracy and/or automation of data processing, the addition of neural network model”, in claim 1 herein does not make the claim rooted in computer technology or improve the functioning of a computer. The “training of data” improves the accuracy of computing a set of gradient vectors” of each layer of the partition and is NOT an improvement to “neural network model” or the functioning of a computing device(s), and hence does not result in a practical application. It is further noted that additional elements of claim 1, 15 are either performing basic computer functions such as applying server partition, receiving, reading, performing loss functions “known” in the art and hence are well-understood, routine and conventional activities and do not amount or add significantly more. The limitations when taken individually or as an ordered combination do not offer an inventive concept that may amount to add significantly more. As discussed above, the broadest reasonable interpretation of claim 1,15 limitations may grouping of abstract idea(s) because they cover concepts performed including observation, evaluation, judgment, and opinion, furthermore, mere data gathering and output recited at a high level of generality, and are insignificant extract solution activity, See MPEP 2106.05(g), Thus these arguments are not persuasive.
At page 11, claim 1, applicant argues:
Zheng, Hu prior art references of record, alone or in combination disclose or suggest the “partitioning a plurality of layers of a neural network model into a device partition and a server partition”
Examiner’s response:
As best understood by the examiner, the prior art of Zheng is directed to distributed training of machine learning models, particularly with respective to plurality of training computing devices (Zheng: Abstract). The prior art of Zheng teaches each server of the resource comprises training computing devices or TCDs invoking function(s) of device partitioning (Zheng: col 6, line 8-12). The prior art of Zheng teaches partition groups associated with training computing devices for example as detailed in fig 4, line 37-60 is identical to instant specification para 0027
PNG
media_image1.png
232
323
media_image1.png
Greyscale
The prior art of Zheng teaches multi-stage training state parameters through computations/propagations in a distributed training model(s) thereby allows gradient synchronization and parameter updating to each of the training computing devices or TCDs associated with partitioning of layers (Zheng: col 6, line 30-39)
The prior art of Hu is directed to device, neutral network and training , particularly network forms a neural network (Hu: Abstract). Hu teaches segmentation process identifying sub-network sampling layers, particularly training segmented data and/or image (Hu: 0016)
It is however, noted that Zheng does not disclose “applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values”, although Zheng teaches defining the learning model to incorporate loss function with respect to neural network used for the model, sizes of training batches (Zheng: col 8, line 11-15, col 22, line 39-50). On the other hand, Hu disclosed “applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values” (Hu: fig 7, 0147-0148, 0157-0158,0163 – Hu teaches training process of neural network in calculating loss value of the respective loss function of the train[ing] segment image
PNG
media_image2.png
281
455
media_image2.png
Greyscale
It would have been obvious to a person of ordinary skill in the art at the time of filing the claimed invention performing segment process in training method particularly using neural network of Hu into distributed training of machine learning models particularly training partition groups of Zheng et al., because both Zheng, Hu teaches training model in a neural network environment (Zheng: Abstract, fig 3-4; Hu: Abstract, fig 2-3) and they both are from the same field of endeavor. Because both Zheng, Hu teaches training model in a neural network environment, it would have been obvious to one skill ed in the art to substitute and/or modify one method for the other particularly training algorithm to define parameters of partition(s) of segment loss values with respect to loss function configured to process each sample output, while optimization function may calculate error values of the parameters of the neural network, further satisfying predetermined training of the neural network (Hu:0162-0163), further satisfying predetermined training of the neural network, thereby improves overall performance of the training segments. The exemplary rationales that may support prima facie conclusion of obviousness includes (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention- -KSR, 550 US at 398.
Examiner applies above argument to claims 2-8,16-20 depend from claim 1,15
At page 12, claim 9, applicant argues:
As stated above, Zheng does not teach or fairly describe the example subject matter cited above.
independent claim 9 includes subject matter substantially similar to the example subject matter of claim 1 discussed above, and therefore also is not disclosed by the cited references.
Dependent claims 10-14 necessarily include subject matter similar to that of claim 1 recited above by way of dependency, and therefore also are not disclosed by Zheng
Examiner’s response:
As best understood by the examiner, the prior art of Zheng is directed to distributed training of machine learning models, particularly with respective to plurality of training computing devices (Zheng: Abstract). The prior art of Zheng teaches each server of the resource comprises training computing devices or TCDs invoking function(s) of device partitioning (Zheng: col 6, line 8-12). Zheng teaches training state information, more specifically, partitioning training server(s) and accordingly partition groups such as PGO, PG1……., fig 4, line 37-60. Zheng teaches variety of different devices and servers in collaborate to training the data, further Zheng teaches distributed training of the DNN model including distributed training of model(s)), and prior art of Zheng teaches partitioning of devices as detailed in fig 4, fig 1, col 12, line 5-8, line 36-45, fig 11, line 43-47. It is also noted that Zheng teaches neural network model specifically teaches set of data samples in the partitioning of training data as detailed in fig 4.
PNG
media_image3.png
191
285
media_image3.png
Greyscale
Hence,the prior art of Zheng fairly disclosed claim 9 subject matter, claim 9 is rejected under 35 USC 102(a)(2), examiner applies above arguments to claims 10-14 depend from claim 9.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The judicial exception is not integrated into a practical application.
Claim 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The judicial exception is not integrated into a practical application. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The eligibility analysis in support of these findings is provided below, in accordance with the 2019 Revised Patent Subject Matter Eligibility Guidance, Federal Register (84 FR 50) on January 7, 2019 hereinafter 2019 PEG
Step 1. In accordance with Step 1 of the eligibility inquiry (as explained in MPEP 2106), it is noted that the non-transitory computer-readable medium of claim 1,9, and claim 15, directed to method, is/are one of the eligible categories of subject matter and therefore satisfy Step 1.
Step 2A. In accordance with Step 2A prong one of the 2019 PEG, the limitations reciting the abstract idea are highlighted, and the limitations directed to additional elements are highlighted, as set forth in exemplary claim 1
Claim 1,15. A non-transitory computer-readable medium including instructions executable by a processor to cause the processor to perform operations comprising:
partitioning a plurality of layers of a neural network model into a device partition and a server partition;
transmitting, to a computation device, the device partition, training, collaboratively with the computation device through a network, the neural network model by applying the server partition to a set of activations to obtain a set of output instances, the set of activations obtained by one of receiving, from the computation device,
the set of activations as output from the device partition, or reading, from an activation buffer, the set of activations as previously recorded,
applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values, and
computing a set of gradient vectors for each layer of the server partition, including a set of gradient vectors of a layer bordering the device partition, based on the set of loss values”, The limitations of claim 1,15 above, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. For example, in the context of this claim, this limitation encompasses the user thinking of partition, reading applying loss function, computing set of values based on the loss in obtaining and receiving previously recorded
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas set forth in the 2019 PEG. Accordingly, the claim recites an abstract idea.
With respect to Step 2A prong two of the 2019 PEG, the judicial exception is not integrated into a practical application. The additional elements are directed to claim 15 method steps however, these elements fail to integrate the abstract idea into a practical application because they fail to provide an improvement to the functioning of a computer or to any other technology or technical field, fail to apply the exception with a particular machine, fail to apply the judicial exception to effect a particular data structure of partitioning, output instances, applying a loss function to effect a transformation of a particular article to a different state or thing, and fail to apply/use the abstract idea in a meaningful way beyond generally linking the use of the judicial exception to a particular technological environment.
Furthermore, although these elements have been fully considered, they are directed to the use of generic computing elements (para 0024-0026, 0075-0082,0085 of the instant specification make it clear that the disclosed functionality is implemented on well-known computing systems and general purpose computing devices) to perform the abstract idea, which is not sufficient to amount to a practical application (as noted in the 2019 PEG) and is amount to simply saying "apply it" using a general purpose computer, which merely serves to tie the abstract idea to a particular technological environment computer based operating environment) by using the computer as a tool to perform the abstract idea.
Since the analysis of Step 2A prong one and prong two results in the conclusion that the claims are directed to an abstract idea, additional analysis under Step 2B of the eligibility inquiry must be conducted in order to determine whether any claim element or combination of elements amount to significantly more than the judicial exception.
Claim 9. “A non-transitory computer-readable medium including instructions executable by a processor to cause the processor to perform operations comprising:
receiving, from a server, a device partition of a neural network model, the neural network model including a plurality of layers partitioned into the device partition and a server partition,
training, collaboratively with the server through a network, the neural network model by applying the device partition to a set of data samples to obtain a set of activations, and transmitting, to the server, the set of activations in response to determining to transmit the set of activations”, The limitations of claim 9 above, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. For example, in the context of this claim, this limitation encompasses the user thinking of partition, set of samples, set of activations, transmit the set of activations, If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas set forth in the 2019 PEG. Accordingly, the claim recites an abstract idea.
With respect to Step 2A prong two of the 2019 PEG, the judicial exception is not integrated into a practical application. The additional elements are directed to claim 15 method steps however, these elements fail to integrate the abstract idea into a practical application because they fail to provide an improvement to the functioning of a computer or to any other technology or technical field, fail to apply the exception with a particular machine, fail to apply the judicial exception to effect a particular data structure of partitioning, output instances, applying a loss function to effect a transformation of a particular article to a different state or thing, and fail to apply/use the abstract idea in a meaningful way beyond generally linking the use of the judicial exception to a particular technological environment.
Furthermore, although these elements have been fully considered, they are directed to the use of generic computing elements (para 0024-0026, 0075-0082,0085 of the instant specification make it clear that the disclosed functionality is implemented on well-known computing systems and general purpose computing devices) to perform the abstract idea, which is not sufficient to amount to a practical application (as noted in the 2019 PEG) and is amount to simply saying "apply it" using a general purpose computer, which merely serves to tie the abstract idea to a particular technological environment computer based operating environment) by using the computer as a tool to perform the abstract idea.
Since the analysis of Step 2A prong one and prong two results in the conclusion that the claims are directed to an abstract idea, additional analysis under Step 2B of the eligibility inquiry must be conducted in order to determine whether any claim element or combination of elements amount to significantly more than the judicial exception.
Step 2B. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional method limitations are directed to a generic computer, at a very high level of generality and without imposing meaningful limitations on the scope of the claim. In addition para 0024-0026, 0075-0082,0085 of the instant specification of the instant specification describe generic off-the-shelf computer-based elements for implementing the claimed invention which does not amount to significantly more than the abstract idea and is not enough to transform an abstract idea into eligible subject matter. Such generic, high-level, and nominal involvement of a computer or computer-based elements for carrying out the invention merely serves to tie the abstract idea to a particular technological environment, which is not enough to render the claims patent-eligible, as noted at pg. 74624 of Federal Register/Vol. 79, No. 241, citing Alice, which in turn cites Mayo. Further, See, e.g., Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 134 S. Ct. 2347, 2359-60, 110 USPQ2d 1976, 1984 (2014). See also OIP Techs. v. Amazon.com, 788 F.3d 1359, 1364, 115 USPQ2d 1090, 1093-94 (Fed. Cir. 2015) ("Just as Diehr could not save the claims in Alice, which were directed to 'implement[ing] the abstract idea of intermediated settlement on a generic computer', it cannot save O/P's claims directed to implementing the abstract idea of price optimization on a generic computer.") (citations omitted). See also, Affinity Labs of Texas LLC v. DirecTV LLC, 838 F.3d 1253, 1257-1258 (Fed. Cir. 2016) (mere recitation of a GUI does not make a claim patent-eligible); Intellectual Ventures I LLC v. Capital One Bank, 792 F.3d 1363, 1370 (Fed. Cir. 2015) ("the interactive interface limitation is a generic computer element".)
The additional elements are broadly applied to the abstract idea at a high level of generality ("similar to how the recitation of the computer in the claims in Alice amounted to mere instructions to apply the abstract idea of intermediated settlement on a generic computer,") as explained in MPEP § 2106.05(f)) and they operate in a well-understood, routine, and conventional manner.
MPEP § 2106.05 (d)(II) sets forth the following:
The courts have recognized the following computer functions as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g. at a high level of generality) as insignificant extra-solution activity.
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec...; TLI Communications LLC v. AV Auto. LLC...; OIP Techs., Inc., v. Amazon.com, Inc... ; buySAFE, Inc. v. Google, Inc...;
Performing repetitive calculations, Flook ... ; Bancorp Services v. Sun Life...;
Electronic recordkeeping, Alice Corp...; Ultramercial... ;
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc...;
Electronically scanning or extracting data from a physical document, Content Extraction and Transmission, LLC v. Wells Fargo Bank...; and
A web browser's back and forward button functionality, Internet Patent Corp. v. Active Network, Inc...
Courts have held computer-implemented processes not to be significantly more than an abstract idea (and thus ineligible) where the claim as a whole amounts to nothing more than generic computer functions merely used to implement an abstract idea, such as an idea that could be done by a human analog (i.e., by hand or by merely thinking).
As to Claim 2,16, further elaborates, ”wherein the operations further comprise: training, before the partitioning, the neural network model”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 3,17, further elaborates, “wherein the operations further comprise: transmitting, to the computation device, the set of gradient vectors of the layer bordering the device partition in response to determining to transmit the set of gradient vectors”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 4,18, further elaborates, “wherein the training the neural network model further includes:
dequantizing the set of activations by increasing the bit-width of each activation among the set of activations”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 5,19, further elaborates, “wherein the training the neural network model further includes:
updating weight values of the server partition based on the set of gradient vectors for each layer of the server partition”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 6,20, further elaborates, “wherein the operations further comprise:
performing a plurality of iterations of the training, wherein at least a first iteration among the plurality of iterations includes receiving the set of activations and at least a second iteration among the plurality of iterations includes reading the set of activations;
receiving the device partition from the computation device; and
combining the device partition with the server partition to obtain an updated neural network model”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 7, further elaborates, “wherein the receiving the set of activations includes receiving a set of labels from the computation device”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 8, further elaborates, “wherein the applying further includes recording the set of activations to the activation buffer in response to receiving the set of activations”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 10, further elaborates, “wherein the training further includes:
computing a set of gradient vectors for each layer of the device partition, based on a set of gradient vectors of a layer of the server partition bordering the device partition,
the set of gradient vectors obtained by one of receiving, from the server, the set of gradient vectors as computed by the server, or reading, from a gradient buffer, the set of gradient vectors as previously recorded”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 11, further elaborates, “wherein the training the neural network model further includes:
quantizing the set of activations by decreasing the bit-width of each activation among the set of activations”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 12, further elaborates, “wherein the training the neural network model further includes:
updating weight values of the device partition based on the set of gradient vectors for each layer of the device partition”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 13, further elaborates, “wherein the operations further comprise:
performing a plurality of iterations of the training, wherein at least a first iteration among the plurality of iterations includes determining to transmit the set of activations and at least a second iteration among the plurality of iterations includes determining not to transmit the set of activations”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
As to claim 14, further elaborates, “wherein the transmitting the set of
compressed activations includes transmitting a set of labels to the server”, which have been determined to be extra-solution activity that does not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(b)(I). Even in combination, the additional details recited in these claims do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea..
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-20 (18/247,447) are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of copending Application No. 17/770,049 (as filed on 6/25/2025-reference application). Although the claims at issue are not identical, they are not patentable distinct from each other because they are substantially similar in scope and they use the similar limitations to produce output relation diagram of plurality of attributes items.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
instant application 18/247,447
co-pending Appl No 17/770,049 (as filed on 6/25/2025)
Claim 1,15. A non-transitory computer-readable medium including instructions executable by a processor to cause the processor to perform operations comprising:
partitioning a plurality of layers of a neural network model into a device partition and a server partition;
transmitting, to a computation device, the device partition, training, collaboratively with the computation device through a network, the neural network model by applying the server partition to a set of activations to obtain a set of output instances, the set of activations obtained by one of receiving, from the computation device,
the set of activations as output from the device partition, or reading, from an activation buffer, the set of activations as previously recorded,
applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values, and
computing a set of gradient vectors for each layer of the server partition, including a set of gradient vectors of a layer bordering the device partition, based on the set of loss values.
Claim 1, A computer-readable storage medium including instructions executable by a first server to cause the first server to perform operations comprising:
training, cooperatively with a computational device through a network, the neural network model by performing iterations of receiving, from the computational device, a feature map output from a device partition of a neural network model, the neural network model including a plurality of layers partitioned into the device partition and a server partition, applying the server partition to the feature map, updating gradient values and weight values of the layers of the server partition based on a loss function relating feature maps to output of the server partition, and transmitting, to the computational device, gradient values of a layer bordering the device partition and a loss value of the loss function, creating, during the iterations of training, a data checkpoint,
the data checkpoint including the gradient values and the weight values of the server partition, the loss value, and an optimizer state; receiving, during the iterations of training, a migration notice, the migration notice including an identifier of a second edge server; and transferring, during the iterations of training, the data checkpoint to the second edge server; wherein the second edge server receives the data checkpoint, and resumes the iterations of training, cooperatively with the computational device through the network, the neural network model by utilizing the weight values of the server partition of the data checkpoint during the applying and the updating of a first iteration by the second edge server
It would have been obvious to a person of ordinary skill was made to modify and/or to omit the additional elements of claims 1-20 of the instant application 18/247,447 to arrive at the claims1-20 of U.S. copending application 17/770,049 (as filed on 6/25/2025) because the obvious limitation, the difference between applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values, and
computing a set of gradient vectors for each layer of the server partition, including a set of gradient vectors of a layer bordering the device partition, based on the set of loss values (instant application 18/247,447) and gradient values of a layer bordering the device partition and a loss value of the loss function, creating, during the iterations of training, a data checkpoint, (copending application 17/770,049 ( as filed on 6/25/2025) the ordinary skilled person would have realized that the remaining element(s) would perform the same function as before. Omission and/or addition of elements and its function in combination is obvious expedient if the remaining elements perform same functions as before
Claims 1-20 (18/247,447) are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of copending Application No. 18028765 -reference application). Although the claims at issue are not identical, they are not patentable distinct from each other because they are substantially similar in scope and they use the similar limitations to produce output relation diagram of plurality of attributes items.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
instant application 18/247,447
co-pending Appl No 18/028,765
Claim 1,15. A non-transitory computer-readable medium including instructions executable by a processor to cause the processor to perform operations comprising:
partitioning a plurality of layers of a neural network model into a device partition and a server partition;
transmitting, to a computation device, the device partition, training, collaboratively with the computation device through a network, the neural network model by applying the server partition to a set of activations to obtain a set of output instances, the set of activations obtained by one of receiving, from the computation device,
the set of activations as output from the device partition, or reading, from an activation buffer, the set of activations as previously recorded,
applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values, and
computing a set of gradient vectors for each layer of the server partition, including a set of gradient vectors of a layer bordering the device partition, based on the set of loss values.
A non-transitory computer-readable medium including instructions executable by a processor to cause the processor to perform operations comprising:
partitioning a plurality of layers of a neural network model into a device partition and a server partition;
combining a plurality of encoding layers of an auto-encoder neural network with the device partition, wherein a largest encoding layer among the plurality of encoding layers is adjacent a layer of the device partition bordering the server partition;
combining a plurality of decoding layers of the auto-encoder neural network with the server partition, wherein a largest decoding layer among the plurality of decoding layers is adjacent a layer of the server partition bordering the device partition;
transmitting, to a computation device, the device partition combined with the plurality of encoding layers; and training, collaboratively with the computation device through a network, the neural network model by receiving, from the computation device, a set of compressed activations output from the plurality of encoding layers, applying the plurality of decoding layers to the set of compressed activations to obtain a set of activations, applying the server partition to the set of activations to obtain a set of output instances, applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values, computing a set of gradient vectors for each layer of the server partition, including a set of gradient vectors of a layer bordering the device partition, based on the set of loss
It would have been obvious to a person of ordinary skill was made to modify and/or to omit the additional elements of claims 1-20 of the instant application 18/247,447 to arrive at the claims1-20 of U.S. copending application 18/028,765 because the obvious limitation, the difference between applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values, and computing a set of gradient vectors for each layer of the server partition, including a set of gradient vectors of a layer bordering the device partition, based on the set of loss values (instant application 18/247,447) applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values, computing a set of gradient vectors for each layer of the server partition, including a set of gradient vectors of a layer bordering the device partition, based on the set of loss, (copending application 18/028,765 the ordinary skilled person would have realized that the remaining element(s) would perform the same function as before. Omission and/or addition of elements and its function in combination is obvious expedient if the remaining elements perform same functions as before
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-8,15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zheng et al., (hereafter Zheng), US Patent No. 12,423,578 filed on Jan 2022 based on provisional application 63/303,379 in view of Hu, US Pub. No. 2022/0398783 filed on Aug, 2020
As to claim 1,15, Zheng teaches a system which including “A non-transitory computer-readable medium including instructions executable by a processor to cause the processor to perform operations comprising” (Zheng: fig 1, and 11, col 28, line 16-53)
PNG
media_image4.png
470
301
media_image4.png
Greyscale
“partitioning a plurality of layers of a neural network model into a device partition and a server partition” (Zheng: fig 4, line 37-60 - partitioning training server(s) and accordingly partition groups such as PGO, PG1…….) ,
PNG
media_image1.png
232
323
media_image1.png
Greyscale
transmitting, to a computation device, the device partition, training, collaboratively with the computation device through a network, the neural network model (Zheng: col 6, line 30, fig 1, col 12, line 5-8, line 36-45, fig 11, line 43-47 – Zheng teaches variety of different devices and servers in collaborate to training the data, further Zheng teaches distributed training of the DNN model including distributed training of model(s)), and prior art of Zheng teaches partitioning of devices as detailed in fig 4
PNG
media_image3.png
191
285
media_image3.png
Greyscale
“by applying the server partition to a set of activations to obtain a set of output instances, the set of activations obtained by one of receiving, from the computation device” (Zheng: fig 4, col 15, line 45-49,col 16, line 37-56, col 17, line 19-33 - Zheng teaches server partition groups, and training the model(s) and each partition server associated with training resource group(s) as detailed in fig 4 and specifying the parameters such as bandwidths between partitioned servers)
"the set of activations as output from the device partition, or reading, from an activation buffer, the set of activations as previously recorded” (Zheng: col 14, line 26-32, col 17, line 44-49 - Zheng teaches partition groups identifies computing resources across all the devices accordingly output and corresponding model parameters previously associated with the training computations of a machine learning model);
“computing a set of gradient vectors for each layer of the server partition, including a set of gradient vectors of a layer bordering the device partition” (Zheng: fig 7, col 20, line 20-30, line 34-45, 46-52, col 21, line 1-26 – Zheng teaches gradient synchronization, and gradient accumulation boundary with respect to each partition groups as detailed in fig 7
PNG
media_image5.png
226
310
media_image5.png
Greyscale
It is however, noted that Zheng does not disclose “applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values”, although Zheng teaches defining the learning model to incorporate loss function with respect to neural network used for the model, sizes of training batches (Zheng: col 8, line 11-15, col 22, line 39-50). On the other hand, Hu disclosed “applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values” (Hu: fig 7, 0147-0148, 0157-0158,0163 – Hu teaches training process of neural network in calculating loss value of the respective loss function of the train[ing] segment image
PNG
media_image2.png
281
455
media_image2.png
Greyscale
It would have been obvious to a person of ordinary skill in the art at the time of filing the claimed invention performing segment process in training method particularly using neural network of Hu into distributed training of machine learning models particularly training partition groups of Zheng et al., because both Zheng, Hu teaches training model in a neural network environment (Zheng: Abstract, fig 3-4; Hu: Abstract, fig 2-3) and they both are from the same field of endeavor. Because both Zheng, Hu teaches training model in a neural network environment, it would have been obvious to one skill ed in the art to substitute and/or modify one method for the other particularly training algorithm to define parameters of partition(s) of segment loss values with respect to loss function configured to process each sample output, while optimization function may calculate error values of the parameters of the neural network, further satisfying predetermined training of the neural network (Hu:0162-0163), further satisfying predetermined training of the neural network, thereby improves overall performance of the training segments
As to claim 2,16, the combination of Zheng, Hu disclosed “ The computer-readable medium of claim 1, wherein the operations further comprise: training, before the partitioning, the neural network model” (Zheng: Abstract, col 6, line 30-38, col 6, line 50-55).
As to claim 3,17, the combination of Zheng, Hu disclosed “The computer-readable medium of claim 1, wherein the operations further comprise: transmitting, to the computation device, the set of gradient vectors of the layer bordering the device partition in response to determining to transmit the set of gradient vectors” (Zheng: col 13, line 50-63, col 26, line 24-45).
As to claim 4,18, the combination of Zheng, Hu disclosed The computer-readable medium of claim 1, wherein the training the neural network model further includes:
dequantizing the set of activations by increasing the bit-width of each activation among the set of activations” (Zheng: col 17, line 5-12, line 25-37).
As to claim 5,19, the combination of Zheng, Hu disclosed “The computer-readable medium of claim 1, wherein the training the neural network model further includes:
updating weight values of the server partition based on the set of gradient vectors for each layer of the server partition” (Zheng: col 19, line 1-20, fig 7, col 20, line 64-67, col 21, line 1-15)
As to claim 6,20, the combination of Zheng, Hu disclosed The computer-readable medium of claim 5, wherein the operations further comprise:
“performing a plurality of iterations of the training, wherein at least a first iteration among the plurality of iterations includes receiving the set of activations and at least a second iteration among the plurality of iterations includes reading the set of activations”(Zheng: col 6, line 30-36, col 14, line 26-38, col 26, line 24-43);
“receiving the device partition from the computation device” (Zheng: fig 4, col 17, line 19-35, col 19, line 1-19); and
“combining the device partition with the server partition to obtain an updated neural network model” (Zheng: col 20, line 20-44, line 45-59).
As to claim 7, the combination of Zheng, Hu disclosed” The computer-readable medium of claim 1, wherein the receiving the set of activations includes receiving a set of labels from the computation device” (Zheng: col 26, line 24-35) .
As to claim 8, the combination of Zheng, Hu disclosed “The computer-readable medium of claim 1, wherein the applying further includes recording the set of activations to the activation buffer in response to receiving the set of activations” (Zheng: col 7, line 9-26, col 14, line 15-25, col 20, line 29-40).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 9-14 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Zheng et al., (hereafter Zheng), US Patent No. 12,423,578 filed on Jan 2022 based on provisional application 63/303,379
As to claim 9., Zheng teaches a system which including “A non-transitory computer-readable medium including instructions executable by a processor to cause the processor to perform operations comprising (Zheng: fig 1, and 11, col 28, line 16-53)
PNG
media_image4.png
470
301
media_image4.png
Greyscale
“receiving, from a server, a device partition of a neural network model, the neural network model including a plurality of layers partitioned into the device partition and a server partition” (Zheng: fig 4, line 37-60 – Zheng teaches training state information, more specifically, partitioning training server(s) and accordingly partition groups such as PGO, PG1…….) ,
PNG
media_image1.png
232
323
media_image1.png
Greyscale
“training, collaboratively with the server through a network (Zheng: col 6, line 30, fig 1, col 12, line 5-8, line 36-45, fig 11, line 43-47 – Zheng teaches variety of different devices and servers in collaborate to training the data, further Zheng teaches distributed training of the DNN model including distributed training of model(s)), and prior art of Zheng teaches partitioning of devices as detailed in fig 4), “the neural network model by applying the device partition to a set of data samples to obtain a set of activations, and transmitting, to the server, the set of activations in response to determining to transmit the set of activations” (Zheng: fig 4, col 6-7, line 30-34, col 26, line 24-30, line 40-42, line 49-54 – Zheng teaches neural network model specifically teaches set of data samples in the partitioning of training data)
PNG
media_image3.png
191
285
media_image3.png
Greyscale
PNG
media_image6.png
126
466
media_image6.png
Greyscale
PNG
media_image7.png
252
158
media_image7.png
Greyscale
As to claim 10, Zheng disclosed:
“computing a set of gradient vectors for each layer of the device partition (Zheng: col , 22, line 45-50) based on a set of gradient vectors of a layer of the server partition bordering the device partition” (Zheng: fig 4, Abstract, col 16, line 37-56, fig 7, col 13, line 50-55, col 20. Line 20-28, col 20, line 46-52),
“the set of gradient vectors obtained by one of receiving, from the server, the set of gradient vectors as computed by the server, or reading, from a gradient,, the set of gradient vectors as previously recorded” (Zheng: col 14, line 6-25, col 20, line 25-34,col 21, line 46-67)
As to claim 11, Zheng disclosed:
quantizing the set of activations by decreasing the bit-width of each activation among the set of activations” (Zheng: col 7, line 9-16)
As to claim 12, Zheng disclosed:
“updating weight values of the device partition based on the set of gradient vectors for each layer of the device partition” (Zheng: col 19, line 1-20, fig 7, col 20, line 64-67, col 21, line 1-15)
As to claim 13, Zheng disclosed:
“performing a plurality of iterations of the training, wherein at least a first iteration among the plurality of iterations includes determining to transmit the set of activations and at least a second iteration among the plurality of iterations includes determining not to transmit the set of activations” (Zheng: fig 10, col 6, line 30-37, col 14, line 26-43, col 26, line 24-40)
As to 14, Zheng disclosed:” wherein the transmitting the set of compressed activations include transmitting a set of labels to the server” (Zheng: col 24, line 34-50, col 25, line 26-33)
Conclusion
The prior art made of record
a. US Patent. No. 12,423,578
b. US Pub. No. 2022/0398783
Examiner's Note: Examiner has cited particular columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
SEE MPEP 2141.02 [R-5] VI. PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, INCLUDING DISCLOSURES THAT TEACH AWAY FROM THE CLAIMS: A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert. denied, 469 U.S. 851 (1984) In re Fulton, 391 F.3d 1195, 1201,73 USPQ2d 1141, 1146 (Fed. Cir. 2004). >See also MPEP §2123.
In the case of amending the Claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
The prior art made of record, listed on form PTO-892, and not relied upon, if any, is considered pertinent to applicant's disclosure
Authorization for Internet Communications
The examiner encourages Applicant to submit an authorization to communicate with the examiner via the Internet by making the following statement (from MPEP 502.03):
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with the undersigned and practitioners in accordance with 37 CFR 1.33 and 37 CFR 1.34 concerning any subject matter of this application by video conferencing, instant messaging, or electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Please note that the above statement can only be submitted via Central Fax (not Examiner's Fax), Regular postal mail, or EFS Web using PTO/SB/439.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Srirama Channavajjala whose telephone number is 571-272-4108. The examiner can normally be reached on Monday-Friday from 8:00 AM to 5:30 PM Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gorney, Boris, can be reached on (571) 270- 5626. The fax phone numbers for the organization where the application or proceeding is assigned is 571-273-8300 Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free)
/Srirama Channavajjala/Primary Examiner, Art Unit 2154