DETAILED ACTION
This final office action is responsive to application 17/493,947 with applicant’s amendments and request for reconsideration as submitted 09 Dec 2025.
Claim status is currently pending and under examination for claims 1-20 of which independent claims are 1, 11 and 20 further corresponding to the amended claims; no claims are in condition for allowance.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Remarks
Applicant’s responsive remarks filed 12/09/25 regarding the prior art have been considered, but they are moot in view of the new grounds of rejection as necessitated by the applicant’s amendments. Updated search and consideration is given whereby additional art is identified to meet the new language of amendment. Particularly, reference Gu is newly relied upon in the updated rejection under 35 U.S.C. 103 detailed below. See optionally Kim in conclusion for alternative, not relied upon but is considered among pertinent prior arts.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 7, 10-13, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over: Shukla et al., US PG Pub No 2022/0311832A1 hereinafter Shukla (Microsoft) in view of Gu et al., US PG Pub No 2022/0004933A1 hereinafter Gu, in view of Deng et al., “SHARE: Shaping Data Distribution at Edge for Communication-Efficient Hierarchical Federated Learning” hereinafter Deng, and further in view of Oluyemi et al., US PG Pub No 2022/0308929A1 hereinafter Oluyemi (IBM).
With respect to claim 1, Shukla teaches:
A method {Shukla [0025,29] “techniques, such as, for example, but without limitation” describes DLT jobs and AI workloads, see Figs 4 and 6} comprising:
receiving, at a device and via a user interface, definition data for a machine learning workload, wherein the user interface represents the machine learning workload as a set of roles and channels, each role representing a task to be performed by a node in a network and each channel representing a communication channel between roles {Shukla Fig 4:404 “Receive AI Workloads …Training Workload” particularly as Fig 6 “DLT job” defines Deep Learning Training job introduced [0025], so as for [0049-52] “execution of a training workload” is a task representative of a role performed by a node device e.g. GPU or CPU Figs 6, 7:709. Further “API” [0063, 0129-30] is interface detailing socket protocol with addressing (addr) for network connections representative of communication channels, noting e.g. [0074] “scale across hundreds of datacenters… cross-geographical” hence Title or Fig 5 global/regional AI workloads};
identifying, by the device and based on the definition data, groups of training nodes in the network that store training datasets, to perform training roles for the machine learning workload by training machine learning models on their respective training datasets {Shukla [0050] “identifying a subset of infrastructure resources… e.g., an AI workload that requires the use of four GPUs in parallel may be assigned to a node of the system that has at least four GPUs” e.g. [0024-27] “GPUs in a cluster” describes DLT job processing performed by GPU cluster among “node pools” where “Worker Nodes are where machine learning workloads run” with [0032] “cluster of infrastructure resources within a region” Illustratively, Figs 6, 7:702 show DLT in system for the subset of infrastructure resources Fig 4:406 that perform role of [0052] “execution of a training workload… on an infrastructure resource, such as a GPU” and [0051] “saving a state checkpoint of an AI workload that is currently being executed” similar [0083,81] saving DLT job state is storing, container-based files such as images are described for underlying data of a “training loop” shown Fig 9:906 and TensorFlow/PyTorch (TF/PT) are disclosed [0154] to provide the skilled artisan with tools for custom training datasets, again at [0023,69], [0149]};
Shukla further suggests [0152] “sum is computed across all of the workers/GPUs” (aggregation) with [0077] “control over topology” and uses [0075] “distributed snapshot of all workers in the DLT job” as well as Resource Allocation/Assigning [0058] Fig 4 and content hashing [0125].
However, Shukla does not expressly disclose “index of training datasets” which is met by Gu:
wherein identifying comprises querying an index of training datasets using a type of training data specified in the definition data to locate training nodes that store training datasets of that type {Gu [0118] “sampling… i is the index of the training samples. Each training sample includes the instance (xi)G1 and its corresponding label yi …every worker or computing device has the label yi” distributing the xi instances to workers of a federated learning is disclosed [0046-47] and the labels are distributed to active learners who perform gradient-based training [0061] Fig 1. The data may be partitioned into feature groups or categorical features are disclosed [0085,186]. Worker storage is shown Fig 4B:456};
Gu is directed to distributed training of machine learning models thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to perform sampling on indexed training data per Gu in combination for a motivation “because the workers pick their own i-th training instances independently. In other words, the different worker are likely training on different instances at the same time” [0148] and/or requesting coordinator of workers in communication to collect certain data for a distributed learning [0100,120].
However, the combination of Shukla and Gu does not expressly disclose the following limitation which is met by Deng:
selecting, by the device, a set of intermediate aggregator nodes for the groups of training nodes, each intermediate aggregator node performing an aggregation role to aggregate those machine learning models trained by an assigned group of training nodes into intermediate models {Deng [P.27 Rt.Col] “A subset of distributed computing nodes in N can be selected as edge aggregators to conduct model aggregation …denoted by Ne ⊂ N. As shown in Fig. 4” aggregated model updates over “local training iterations… global model aggregations” and shown again per Fig 5 bottom-right, detailed [P.29 Last¶ - P.30 ¶2] “edge aggregator selection problem to find the optimal edge aggregator set” and [P.26 Last¶] “assigning each edge aggregator 10 nodes with different classes of training data” Algs.1-2 provide implementation}; and
Deng is directed to distributed training of machine learning models thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to select aggregator nodes per Deng in combination for the motivation “To deal with the highly-complex coupling of edge aggregator selection and computing node association, we devise two light-weight algorithms to optimize them accordingly” [P.29 ¶4] in other words to reducing complexity is simplifying the distributed learning in a manner of communication-efficiency, so-titled. This may further entail Alg.2 “repeat… until No operation can reduce the total communication cost” [P.30 Alg.2 Lines2,29] as well as finding the optimal set of edge aggregators [P.29 Last¶].
However, the combination Shukla, Gu and Deng does not disclose the following limitation which is met by Oluyemi:
provisioning, by the device, the machine learning workload by configuring the groups of training nodes, the set of intermediate aggregator nodes, and a global aggregator node for the set of intermediate aggregator nodes and by configuring channels between training nodes in a group, between the groups of training nodes and the set of intermediate aggregator nodes, and between the set of intermediate aggregator nodes and the global aggregator node {Oluyemi discloses [0006-07] “provisioning cloud resources… API is used to provision the cloud computing resources” where the provisioned cloud resources perform ML model training [0050-51] and algorithmic tasks as workloads [0058-56, 32-31]. Notably, Fig 6:40-43 shows Azure/AWS/GCP known machine learning cloud platforms, described [0065-67] “agnostic to the cloud provider… convert the cloud deployment pattern, topology, and/or representation into manifest/configuration files that can be recognized for provisioning by the infrastructure as a code tool (e.g., terraform)” Terraform (or CloudFormation [0024]) are software tools from Hashicorp now IBM and AWS respectively that are used for provisioning Infrastructure-as-Code IaC across different cloud providers. Furthermore [0026-27] describes “interconnected nodes Nodes 10 may communicate with each other. They may be grouped” grouped nodes interconnected is the topology - Fig 8:805 “topology which represent cloud resources and their connections” this topology being at least part of the “configuration files” and/or “manifest” described e.g. [0067-66] is used for configuring of the nodes with connections/channels therebetween. Exemplary servers, computers, and machines of resource pool are disclosed [0040] and shown Figs 4-8}.
Oluyemi is directed to distributed cloud computing and communication for machine learning thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to provision per Oluyemi in combination to arrive at the invention as claimed for the motivation “To simplify and ease the transition for customers, also referred to as deployers, to change to and/or use cloud computing resources… it would be advantageous if a customer could specify the cloud computing resources and specifications, e.g., the architecture, and/or topology” cont’d “users without an in-depth understanding on cloud infrastructure deployment tools could provision” [0023].
With respect to claim 2, the combination of Shukla, Gu, Deng and Oluyemi teaches the method as in claim 1, wherein
the device configures the channels by configuring application programming interfaces (APIs) on the groups of training nodes, the set of intermediate aggregator nodes, and the global aggregator node, to form communication channels in the network {Oluyemi [0067] “configuration files are converted into input data which can be used by cloud provider APIs” shown Figs 4:495, 5:495, 9:940. The API is configured to convert topology for cloud provisioning [0067,72] where topology corresponds to the network nodes with connections/channels [0026]. This may entail metadata collected from a pattern recognition/detected sketch of cloud network [0072,51]. API configured to make program calls for running trained ML models [0051-50]}.
With respect to claim 3, the combination of Shukla, Gu, Deng and Oluyemi teaches the method as in claim 1, wherein
the device selects the set of intermediate aggregator nodes for the groups of training nodes based in part on their distances to the groups of training nodes {Deng [P.29 Last¶] aggregator selection based on [P.30 Last¶] “distance between node n and edge aggregator e, dec (km) is the shortest path distance between the edge aggregator e and the cloud aggregator” for federated learning/training Fig 1 [P.27 ¶4,1] so-Titled. As a computed function, distance comprises delta Δ [Alg.1 Lines7-8,12] and/or subtractive difference [Alg.2 Lines20-24]}.
With respect to claim 7, the combination of Shukla, Gu, Deng and Oluyemi teaches the method as in claim 1, wherein
the global aggregator node performs a global aggregation of the intermediate models into a global machine learning model {Deng [P.27 ¶3] “global model aggregations” over “training iterations …round of model training. The process repeats” Fig 4 shows aggregation using server and edge devices, the server is commonly global, local being edge devices. See also [P.26 ¶1-2] “global aggregations” similarly at [P.25 ¶]}.
With respect to claim 10, the combination of Shukla, Gu, Deng and Oluyemi teaches the method as in claim 1, wherein
at least one of the set of intermediate aggregator nodes is cloud-based {Deng Fig 4 illustrates “cloud aggregation” with “nodes” as sets and/or subsets described throughout e.g. [P.26 Last¶], [P.27 Sect.III]}.
With respect to claim 11, the rejection of claim 1 is incorporated. The difference in scope being an apparatus comprising network interface coupled to processor and memory to store process and execute the process limitations of method claim 1. Shukla discloses [0167] “computing device 1200 may operate in a networked environment… includes a network interface” with processor and memory described per [0166-65] and shown Figs 12 and/or 7. The remainder of the claim is rejected for the same rationale as claim 1.
With respect to claim 12, the apparatus of claim 11 is taught by the combination of Shukla, Gu, Deng and Oluyemi whom further teach the limitation of claim 2. Therefore, the rejection of claim 2 is applied to claim 12.
With respect to claim 13, the apparatus of claim 11 is taught by the combination of Shukla, Gu, Deng and Oluyemi whom further teach the limitation of claim 3. Therefore, the rejection of claim 3 is applied to claim 13.
With respect to claim 17, the apparatus of claim 11 is taught by the combination of Shukla, Gu, Deng and Oluyemi whom further teach the limitation of claim 7. Therefore, the rejection of claim 7 is applied to claim 17.
With respect to claim 20, the rejection of claim 1 is incorporated. The difference in scope being a tangible, non-transitory computer-readable medium storing instructions causing a device to execute the process limitations of method claim 1. Shukla discloses [0188] “computer readable media… tangible” e.g. [0169] “software, firmware, hardware, or a combination thereof… executable instructions” with memory and processor described [0166-65] and shown Figs 12 and/or 7. The remainder of the claim is rejected for the same rationale as claim 1.
Claims 4, 8, 14 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Shukla, Gu, Deng and Oluyemi in view of:
Radhakrishnan et al., US PG Pub No 2022/0374762A1 hereinafter Radhakrishnan.
With respect to claim 4, the combination of Shukla, Gu, Deng and Oluyemi teaches the method as in claim 1. Radhakrishnan teaches wherein
the channels between the groups of training nodes and the set of intermediate aggregator nodes are parameter channels via which the groups of training nodes send parameters of the machine learning models that they train to their intermediate aggregator nodes {Radhakrishnan [0084] “communication channels are maintained between aggregators for training… nodes to pull their corresponding model updates, aggregate them together, and distribute” shown Fig 6 arrows indicate communication channels among plurality of aggregators for FL is federated learning and W[P] are weights/parameters, [0067] “sends the aggregated model parameters… parameter uploads” similar at [0064] “sends the resulting parameters to the aggregation server” also [0054], Figs 5 and 7}.
Radhakrishnan is directed to distributed machine learning model training thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify communication channel for parameters of a model per Radhakrishnan in combination to arrive at the invention as claimed for the motivation “synchronizes model parameters” [0067] where “aggregators 602 need to communicate with each other for training synchronization” [0075,84]. Moreover, it is conventional in distributed training that a parameter server is to serve parameters by communicating over a channel.
With respect to claim 8, the combination of Shukla, Gu, Deng and Oluyemi teaches the method as in claim 1. Radhakrishnan teaches wherein
the training datasets comprise private data not shared externally by the training nodes {Radhakrishnan [0066] “private training data” introduces “a key driver behind the emergence of federated learning has been the need to address privacy” where [0064] “Each agent has its own dataset that it wishes to protect and cannot share with other agents or the aggregation server” similar at [0070] “do not tend to share training data”}.
A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to hold training data private and not share per Radhakrishnan in combination to arrive at the invention as claimed for the motivation “federated learning training setting presents a unique advantage for preserving training data privacy… where sharing data is prohibited by law or regulations” [0002]. For example, HIPAA regulation compliance by hospital where private medical information cannot be shared.
With respect to claim 14, the apparatus of claim 11 is taught by the combination of Shukla, Gu, Deng and Oluyemi with whom the combination with Radhakrishan further teach the limitation of claim 4. Therefore, the rejection of claim 4 with equal motivation is applied to claim 14.
With respect to claim 18, the apparatus of claim 11 is taught by the combination of Shukla, Gu, Deng and Oluyemi with whom the combination with Radhakrishan further teach the limitation of claim 8. Therefore, the rejection of claim 8 with equal motivation is applied to claim 18.
Claims 5-6 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Shukla, Gu, Deng and Oluyemi in view of:
Dunsmore et al., US Patent No 11,425,054B1 hereinafter Dunsmore.
With respect to claim 5, the combination of Shukla, Gu, Deng and Oluyemi teaches the method as in claim 1. Dunsmore teaches wherein
the definition data indicates a grouping parameter that specifies how the groups of training nodes should be formed {Dunsmore Fig 12:1275 GroupName: us-east, us-west-2, eu-west, eu-central-1 with “weight” being a parameter which may further correspond to variable GroupName as defined by the pseudocode, see [Col38 Lines12-23] describes weighting for the respective groups, being for a [Col28 Line37] “defined geographic area” whereby [Col3 Line36-39] “geographical area in which the cloud provider clusters data centers” data centers are nodes shown e.g. Figs 4 or 6, implementation uses [Col25 Line48] “AWS CloudFormation”}.
Dunsmore is directed to provisioning cloud resources with data centers thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify grouping per Dunsmore in combination to arrive at the invention as claimed for a motivation “benefit provided by logical distribution groups-such as ‘United States’ or ‘United States East’-is that they can automatically incorporate new regions (that logically belong to those groups) as they are created, and thus no re-configuration is necessary” [Col26 Lines15-19] and/or “enable users to more easily build highly available and/or latency-sensitive applications that will run seamlessly across multiple deployment zones… abstracting away all the complexities that arise with managing applications across many locations” [Col23 Lines47-53].
With respect to claim 6, the combination of Shukla, Gu, Deng, Oluyemi and Dunsmore teaches the method as in claim 5, wherein
the grouping parameter specifies that training nodes should be grouped by geographic area {Dunsmore [Col28 Line37] “defined geographic area” shown Fig 12:1275 us/eu - east/west with weighting and pseudocode for parameter and variable, illustratively Figs 4 and 6 see also [Col38 Lines12-23], [Col3 Line36-39]}. Motivation for combination is applied equally as in claim 5.
With respect to claim 15, the apparatus of claim 11 is taught by the combination of Shukla, Gu, Deng and Oluyemi with whom the combination with Dunsmore teaches the limitation of claim 5. Therefore, the rejection of claim 5 with equal motivation is applied to claim 15.
With respect to claim 16, the apparatus of claim 15 is taught by the combination of Shukla, Gu, Deng, Oluyemi and Dunsmore whom further teach the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 16.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Shukla, Gu, Deng and Oluyemi in view of:
Wang et al., “Characterizing Deep Learning Training Workloads on Alibaba-PAI” hereinafter Wang (arXiv: 1910.05930v1).
With respect to claim 9, the combination of Shukla, Gu, Deng and Oluyemi teaches the method as in claim 1. Wang teaches wherein
the definition data specifies a type of training data and does not specify locations of the training datasets {Wang [P.2-3 Pg.Brk] “training replicas” of [P.7 Last¶] “selected training workloads” specifies type of training data Fig 4, Wang is silent as to ‘location’ thus unspecified however teaches synchronization via Pearl architecture Fig 14, [P.9] describes a distributed training strategy over GPU workers in communication using TensorFlow and XLA compiler}.
Wang is directed to training workloads for distributed machine learning thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify types of training data as replicas and use the distributed learning teachings of Wang in combination to arrive at the invention as claimed for the motivation “we explore different optimization techniques upon different types of workloads… improving practical deep learning training workloads” [P.2 ¶2], [P.12 Last¶].
With respect to claim 19, the apparatus of claim 11 is taught by the combination of Shukla, Gu, Deng and Oluyemi with whom the combination with Wang teaches the limitation of claim 9. Therefore, the rejection of claim 9 with equal motivation is applied to claim 19.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Kim et al., US PG Pub No 2023/0059674A1 (Samsung) discloses amendment, see below
wherein identifying comprises querying an index of training datasets using a type of training data specified in the definition data to locate training nodes that store training datasets of that type {Kim [0043] “index training data in the training dataset and allow the data subsets to be distinguished based on index numbers” and [0091] “load training data of index” index shown Fig 11 in a system Fig 1 showing nodes as workers of compute cluster, and process Fig 7:S730-S770. The type of training data can be batches and/or subsets of training data Figs 4,12. Allocating the data to nodes is per [0098-97]. See also [0055-56], [0162]};
Kim is directed to distributed training of machine learning models thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to index training data per Kim in combination for a motivation “identify nodes available for distributed learning… update the list of available nodes” [0162] and/or “improving a distributed learning” [0004].
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chase P Hinckley whose telephone number is (571)272-7935. The examiner can normally be reached M-F 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda M. Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHASE P. HINCKLEY/Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124