Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4 are rejected under 35 U.S.C. 103 as being unpatentable over Zheng et.al (NPL: DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs) in view of Zhao et.al (NPL: Distributed Hierarchical GPU parameter server for massive scale Deep Learning Ads systems)
Regarding claim 1,
Zheng teaches or at least suggest the limitation “obtaining a subgraph structure, the subgraph structure being configured to reflect a graph structure of a corresponding subgraph, and the subgraph comprising a plurality of nodes and edges between the plurality of nodes” (Page 2 section I col.1 “In this work, we develop DistDGL on top of DGL to perform efficient and scalable mini-batch GNN training on a cluster of machines ... It distributes graph data (both graph structure and the associated data, such as node and edge features) across all machines and run trainers, sampling servers (for sampling subgraphs to generate mini-batches) and in-memory KVStore servers (for serving node data and edge data) all on the same set of machines”, Page 2 section II-B col.2 “Therefore, we need to carefully sample subgraphs that capture the data dependencies in the original graph to train GNN models”, and Page 3 section III-A col.1 “A number of samplers in charge of sampling the mini batch graph structures from the input graph ...”, and Figure 1-a) and 1-b). Zheng discloses a distributed graph neural network training framework that generates mini-batch subgraphs from an input graph for scalable GNN training. Zheng teaches sampling mini-batch graph structures that preserve the graph dependencies of the original graph, whereby the sampled mini-batch graph structures reflect the graph structures of their corresponding subgraphs comprising a plurality of nodes and edges between the plurality of nodes (figure 1-a, and 1-b)), corresponding to the claimed subgraph structures, as claimed.)
Zheng teaches or at least suggest the limitation “obtaining, based on the subgraph structure ..., node features of the plurality of nodes and edge features of the edges from a plurality of memories” (Page 2 section III-A col.1 “A number of samplers in charge of sampling the mini batch graph structures from the input graph ... After mini-batch graphs are generated, they are sent back to the trainers. A KVStore that stores all vertex data and edge data distributedly ... A number of trainers ... At each iteration, they first fetch the mini-batch graphs from the samplers and the corresponding vertex/edge features from the KVStore”, Page 3 section III-A col.2 “DistDGL launches the sampler and KVStore servers on each machine to serve the local partition data”, and Page 4 section III-C col.2 “The features of vertices and edges are partitioned and stored in multiple machines ... DistDGL develops a distributed in memory key-value store (KVStore) to manage the vertex and edge features as well as vertex embeddings”. Zheng discloses a distributed graph training architecture in which sampled subgraph structures are used to retrieve the corresponding graph features for GNN training. Specifically, after the mini-batch graph structures are generated, trainers fetch the corresponding vertex and edge features from the distributed KVStore. Zheng further teaches that KVStore servers are launched on each machine to serve local partition data and that the graph features are partitioned and stored across multiple machines. Thus, a person of ordinary skill in the art would understand that the distributed in-memory KVStore is implemented across the memories of multiple machines, such that obtaining the vertex and edge features from the distributed in-memory KVStore of multiple machines corresponds to obtaining, based on the subgraph structure, node features of the plurality of nodes and edge features of the edges from a plurality of memories, as claimed.)
Zheng teaches or at least suggest the limitation “fusing, based on the subgraph structure, the node features of the plurality of nodes and the edge features of the edges to obtain subgraph data” (Page 2 section II-B col.2 “The sampled graph and together with the extracted features are called a mini-batch in GNN training”, and Page 3 section III-A col.1 “A number of samplers in charge of sampling the mini batch graph structures from the input graph ... After mini-batch graphs are generated, they are sent back to the trainers. A KVStore that stores all vertex data and edge data distributedly ... A number of trainers ... At each iteration, they first fetch the mini-batch graphs from the samplers and the corresponding vertex/edge features from the KVStore. They then run the forward and backward computation on their own mini-batches in parallel to compute the gradients”, and Figure 2. Zheng discloses generating mini-batch graph data by combining sampled graph structures with their corresponding graph features for distributed GNN training. Specifically, Zheng teaches that the sampled graph together with the extracted vertex and edge features forms the mini-batch used for GNN training, corresponding to fusing the subgraph structures with the corresponding node and edge features to obtain the claimed subgraph data, as claimed.)
Zheng teaches or at least suggest the limitation “training, based on the subgraph data, the graph neural network” (Page 3 section III-A col.1 “DistDGL distributes the mini-batch training process of GNN models to a cluster of machines. It follows the synchronous stochastic gradient descent (SGD) training; each machine computes model gradients with respect to its own mini-batch, synchronizes gradients with others and updates the local model replica”. Zheng discloses distributed graph neural network training using the generated mini-batch graph data. Specifically, Zheng teaches performing forward and backward propagation on the mini-batches, computing gradients, synchronizing the gradients across machines, and updating the local model replicas, corresponding to training the graph neural network based on the subgraph data, as claimed.)
Zheng does not teach a part of the limitation “obtaining ... according to a predetermined priority ... features”. However, Zhao teaches or at least suggest this part of the limitation (Page 4 section 3 col.2 “each node identifies the union of the referenced parameters in the current received batch and pulls these parameters from the local MEM-PS/SSD-PS (line3) and the remote MEM PS ... all the referenced parameters are loaded in the memory ...”, Page 5 section 3 col.1 “HBM-PS, MEM-PS, and SSD-PS communicate in a hierarchical storage fashion. The upper-level module acts as a high-speed cache of the lower-level module”, and Page 6 section 5 col. 2 “The MEM-PS identifies the referenced parameters from the input and communicates with the local SSD-PS and remote MEM-PS to gather the required parameters ... For the local parameters, the MEM-PS reads the SSDs to fetch the parameters”. Zhao discloses a hierarchical parameter server architecture that retrieves required parameters from multiple memory levels organized in a hierarchical storage system. Specifically, Zhao teaches identifying the referenced parameters, gathering the required parameters from HBM-PS, MEM-PS, and SSD-PS, and retrieving the parameters from the appropriate memory level, where the upper-level module operates as a cache for the lower-level module, corresponding to obtaining the required data according to a predetermined priority, as claimed.)
Zheng does not teach “the predetermined priority being obtained by sorting the plurality of memories in accordance with memory size in an ascending order” However, Zhao teaches or at least suggest this limitation (Page 5 section 3 col.1 “HBM-PS, MEM-PS, and SSD-PS communicate in a hierarchical storage fashion. The upper-level module acts as a high-speed cache of the lower-level module... the HBM capacity is limited and pricey. In order to fit all the terabyte-scale parameters into HBMs ... Therefore, our proposed hierarchical architecture leverages memory and SSD to store the massive model parameters” Zhao discloses a hierarchical storage architecture organized according to the characteristics of different memory resources to efficiently store and retrieve parameters. Specifically, Zhao teaches that HBM-PS, MEM-PS, and SSD-PS are organized hierarchically, that the upper-level module acts as a high-speed cache for the lower-level module, and that HBM has limited capacity while lower memory levels provide progressively larger storage for massive model parameters. Thus, although Zhao does not expressly disclose sorting the memories according to memory size, Zhao teaches or at least suggests a memory hierarchy arranged according to progressively increasing storage capacities, which a person of ordinary skill in the art would have understood as suggesting establishing the predetermined priority by sorting the plurality of memories according to memory size in ascending order, as claimed.)
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine the teaching of DistDGL for training large-scale graph neural network through subgraph and in-memory KVStore servers by Zheng with the teaching of distributed hierarchical parameter server utilizing multiple memories for training massive-scale neural networks by Zhao. The motivation to do so is referred to in Zhao’s disclosure (Page 5 section 3 col.1 “HBM-PS, MEM-PS, and SSD-PS communicate in a hierarchical storage fashion. The upper-level module acts as a high-speed cache of the lower-level module... the HBM capacity is limited and pricey. In order to fit all the terabyte-scale parameters into HBMs, we have to maintain a distributed computing cluster with hundreds of GPUs—it is not only expensive but also inefficient ... Therefore, our proposed hierarchical architecture leverages memory and SSD to store the massive model parameters” Zheng discloses that GPU HBM has limited capacity and that leveraging main memory and SSD in a hierarchical storage architecture enables storage and retrieval of massive model parameters that exceed GPU memory capacity. Zhao further teaches that the hierarchical architecture allows the upper-level memory to operate as a cache of the lower-level memory and maintains an in-memory cache to avoid excessive SSD I/Os, thereby improving storage efficiency and reducing memory access overhead. A person of ordinary skill in the art would have been motivated to apply Zhao’s hierarchical memory architecture to Zheng’s distributed graph feature retrieval to efficiently retrieve graph features while accommodating data exceeding GPU memory capacity, thereby improve the overall large-scale graph neural network training by Zheng.)
Regarding claim 4 depends on claim 1, thus the rejection of claim 1 is incorporated.
Zheng teaches or at least suggest the limitation “obtaining the subgraph structure from a second graph storage server memory based on a sampler” (Page 3 section III-A col.1-2 “A number of samplers in charge of sampling the mini batch graph structures from the input graph. Users invoke DistDGL samplers in the trainer process via the same interface in DGL for neighbor sampling ... DistDGL launches the sampler and KVStore servers on each machine to serve the local partition data” Zheng discloses that a number of samplers generate mini-batch graph structures from the input graph and that DistDGL launches sampler and KVStore servers on each machine to serve the local partition data. A person of ordinary skill in the art would understand that the sampled mini-batch graph structures are obtained from the distributed KVStore servers storing the graph partitions. Accordingly, Zheng teaches obtaining the subgraph structure from a second graph storage server memory based on a sampler, as claimed.)
Zheng teaches or at least suggest the limitation “a plurality of subgraph structures being stored in the second graph storage server memory” (Page 3 section III-A col.1-2 “A number of samplers in charge of sampling the mini batch graph structures from the input graph. Users invoke DistDGL samplers in the trainer process via the same interface in DGL for neighbor sampling ... DistDGL launches the sampler and KVStore servers on each machine to serve the local partition data” Zheng discloses that multiple samplers generate mini-batch graph structures from the input graph and that DistDGL launches KVStore servers on each machine to serve local graph partition data. Since the graph is partitioned across multiple machines and each partition is sampled to generate mini-batch graph structures, a person of ordinary skill in the art would understand that a plurality of subgraph structures is stored within the distributed graph storage server memories, thereby corresponding to the claimed plurality of subgraph structures being stored in the second graph storage server memory.)
Zheng in view of Zhao teaches or at least suggest the limitation “node features and edge features of the subgraph structure being pre-extracted and cached into the plurality of memories” (Page 3 section III-A-B col. 2 “DistDGL first par titions the input graph with a light-weight min-cut graph par titioning algorithm. It then partitions the vertex/edge features and co-locates them with graph partitions. DistDGL launches the sampler and KVStore servers on each machine to serve the local partition data... Graph partitioning is a preprocessing step before distributed training” Zheng discloses that DistDGL partitions the input graph, partitions the corresponding vertex and edge features, co-locates the features with the graph partitions, and performs graph partitioning as a preprocessing step before distributed training. Zheng further teaches launching KVStore servers on each machine to serve the local partition data. In view of Zhao’s hierarchical plurality of memories (HBM-PS, MEM-PS, and SSD-PS), a person of ordinary skill in the art would have understood the preprocessed node and edge features associated with the graph partitions to be pre-extracted and stored within the hierarchical memory architecture for efficient retrieval during training, thereby corresponding to the claimed node features and edge features of the subgraph structure being pre-extracted and cached into the plurality of memories.)
Claims 2, 3, 11-18 are rejected under 35 U.S.C. 103 as being unpatentable over Zheng et.al (NPL: DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs) in view of Zhao et.al (NPL: Distributed Hierarchical GPU parameter server for massive scale Deep Learning Ads systems), further in view of Guan et.al (US 20230064080 A1)
Regarding claim 2 depends on claim 1, thus the rejection of claim 1 is incorporated.
Zhao teaches or at least suggest the limitation “... the plurality of memories comprises a graphics processing unit memory, a central processing unit memory and a first graph storage server memory” (Page 3 section 1.3 col.1 “We show a CPU main memory parameter server”, and Page 5 section 3 col.1 “HBM-PS, MEM-PS, and SSD-PS communicate in a hierarchical storage fashion. The upper-level module acts as a high-speed cache of the lower-level module... the HBM capacity is limited and pricey. In order to fit all the terabyte-scale parameters into HBMs ... Therefore, our proposed hierarchical architecture leverages memory and SSD to store the massive model parameters” Zhao discloses a hierarchical storage architecture organized according to the characteristics of different memory resources to efficiently store and retrieve parameters. Specifically, Zhao teaches that HBM-PS,CPU main memory parameter server (MEM-PS), and SSD-PS respectively utilize GPU memory (HBM), CPU main memory, and SSD storage server memory within the disclosed hierarchical storage architecture. Accordingly, Zhao’s disclosed HBM-Ps, MEM-PS, and SSD-PS respectively corresponds to the graphics processing unit memory, central processing unit memory and first graph storage server memory, as claimed.)
Zhao in view of Zheng teaches or at least suggest the limitation “obtaining, based on the subgraph structure and according to the predetermined priority, the node features of the plurality of nodes and the edge features of the edges from the plurality of memories comprises: determining first node identifications associated with the graphics processing unit memory from identifications of the plurality of nodes, and obtaining the node features of first nodes and the edge features of edges connected to the first nodes from the graphics processing unit memory according to the first node identifications” (Page 2 section III-A col.1 “A number of samplers in charge of sampling the mini batch graph structures from the input graph ... After mini-batch graphs are generated, they are sent back to the trainers. A KVStore that stores all vertex data and edge data distributedly ... A number of trainers ... At each iteration, they first fetch the mini-batch graphs from the samplers and the corresponding vertex/edge features from the KVStore”, and Page 4 section III-C col.2 “The features of vertices and edges are partitioned and stored in multiple machines ... DistDGL develops a distributed in memory key-value store (KVStore) to manage the vertex and edge features as well as vertex embeddings”. Zheng discloses that after the mini-batch graph structures are generated, trainers fetch the corresponding vertex and edge features from the distributed KVStore. In view of Zhao, a person of ordinary skill in the art would further understand that the distributed KVStore data to be implemented using the the hierarchical memory architecture including GPU memory (HBM-Ps), CPU main memory (MEM-PS), and storage server memory (SSD-PS), such that the retrieved graph features from the distributed KVStore implemented using the hierarchical memory architecture corresponds to determining and obtaining the node features of first nodes and the edge features of edges connected to the first nodes from the graphics processing unit memory according to the first node identifications, as claimed.)
Zhao/Zheng does not teach the aspect of determining and obtaining from the plurality of memories within the limitation “the first nodes do not include all of the plurality of nodes, determining second node identifications associated with the central processing unit memory from identifications of first remaining nodes, and obtaining the node features of second nodes and the edge features of edges connected to the second nodes from the central processing unit memory according to the second node identifications; the first remaining nodes being nodes obtained after removing the first nodes from the plurality of the nodes”. However, Guan teaches or at least suggest the determining and obtaining from the plurality of memories aspect within the limitation (paragraph 14 “determine whether the attribute data is cached in the second-level buffer of the first memory based on the received graph node identifier; in response to determining that the attribute data is not cached in the second-level buffer, determine whether the graph node identifier table comprises an entry corresponding to the received graph node identifier; and in response to determining that the graph node identifier table does not comprise the entry corresponding to the received graph node identifier, determine that the attribute data is not cached in the first memory”, and paragraph 20 “a computer-implemented method for accelerating Graph Neural Network (GNN) attribute data access is described. The method includes receiving, by a processor within a GNN accelerator, a graph node identifier of a graph node in a graph; determining, by the processor, a target memory address within the first memory; determining, by the processor, whether attribute data corresponding to the received graph node identifier is cached in a first memory at the target memory address; and in response to determining that the attribute data is not cached in the first memory: fetching, by the processor, the attribute data from a second memory, and writing, by the processor, the fetched attribute data into the first memory at the target memory address.” Guan discloses determining, based on a received graph node identifier, whether the requested graph attribute data is stored in a first memory and, when the requested graph attribute data is determined not to be stored in the first memory, determining that the requested graph attribute data is to be retrieved from a second memory and retrieving the requested graph attribute data therefrom. Thus, Guan teaches determining, based on graph node identifications, the memory location from which the requested graph data is to be retrieved and obtaining the corresponding graph data from the determined memory within a plurality of memories. In view of Zhao’s disclosure that MEM-PS provides the CPU main memory tier within the disclosed hierarchical memory architecture, a person of ordinary skill in the art would have understood Guan’s determination and retrieval from the second memory to correspond to determining second node identifications associated with the claimed central processing unit memory and obtaining the node features of the second nodes and the edge features of edges connected to the second nodes from the central processing unit memory according to the determined second node identifications, as claimed.)
Zhao/Zheng does not teach the aspect of determining and obtaining from the plurality of memories within the limitation “if the first nodes and the second nodes do not include all of the plurality of nodes, obtaining the node features of second remaining nodes and the edge features of edges connected to the second remaining nodes from the first graph storage server memory according to identifications of the second remaining nodes, and the second remaining nodes being nodes obtained after removing the first nodes and the second nodes from the plurality of nodes” However, Guan teaches or at least suggest the determining and obtaining from the plurality of memories aspect within the limitation (paragraph 20 “..., a graph node identifier of a graph node in a graph; determining, by the processor, a target memory address within the first memory; determining, by the processor, whether attribute data corresponding to the received graph node identifier is cached in a first memory at the target memory address; and in response to determining that the attribute data is not cached in the first memory: fetching, by the processor, the attribute data from a second memory, and writing, by the processor, the fetched attribute data into the first memory at the target memory address”, and paragraph 43 “the attribute data of each node, commonly represented as a feature vector, is embedded via aggregate-combine functions iteratively in the GNN computation phase to incorporate the interdependence or relationship between the node and its neighboring nodes. During this process, the GNN accelerator 230 needs to retrieve the attribute data of graph nodes/edges of interest and feed the retrieved data to attribute processing units (e.g., processors like central processing units (CPU), graphic processing units (GPU), tensor processing units (TPU), neural processing units (NPU), etc.) for computation ... the GNN accelerator 230 may provide an attribute cache to hold the attribute data that are retrieved from the external memory 210 and for the consumption of the attribute processing units”. Guan further discloses retrieving requested graph attribute data from an external memory when the requested graph attribute data is unavailable in the first memory, including retrieving the attribute data of graph nodes and edges from the external memory for GNN computation. Although Guan expressly discloses retrieval from a second (external) memory, a person of ordinary skill in the art would have understood Guan’s memory-selection and retrieval mechanism to be equally applicable to additional memory tiers when implemented within Zhao’s expressly disclosed hierarchical memory architecture, because Guan’s retrieval mechanism is based on determining the memory storing the requested graph data rather than being limited to only two memory levels. Accordingly, in view of Zhao’s disclosure that SSD-PS provides the storage server memory tier of the hierarchical memory architecture, a POSITA would have understood Guan’s retrieval from the external memory to correspond to obtaining the node features of the second remaining nodes and the edge features of edges connected to the second remaining nodes from the claimed first graph storage server memory according to identifications of the second remaining nodes, as claimed.)
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine the teaching of DistDGL for training large-scale graph neural network through subgraph and in-memory KVStore servers, and the teaching of distributed hierarchical parameter server utilizing multiple memories for training massive-scale neural networks by Zheng/Zhao with the teaching of multi-level attribute caching for accelerating Graph Neural Network by Guan. The motivation to do so is referred to in Guan’s disclosure (paragraph 21 “a multi-level attribute data caching architecture is designed to improve the memory access efficiency for retrieving graph node attribute data and thus the overall performance of GNN computations ... the multi-level attribute data caching architecture described herein introduces another layer of cache as a staging area to store the to-be-evicted attribute data. This additional layer of cache allows the to-be-evicted attribute data to have another chance to be hit again in the RAM and thereby avoiding the expensive external data retrieval” Guan discloses that its multi-level attribute caching architecture is designed to improve memory access efficiency for retrieving graph node attribute data, improve the overall performance of GNN computations, increase cache-hit opportunities by introducing additional cache levels, and avoid expensive external memory retrievals. Accordingly, a person of ordinary skill in the art would have been motivated to apply Guan’s multi-level memory retrieval mechanism to Zheng’s distributed graph feature retrieval within Zhao’s hierarchical memory architecture because doing so would have predictably improved the efficiency of retrieving distributed graph features across the GPU memory, CPU main memory, and storage server memory while reducing memory access latency and improving the overall performance of large-scale graph neural network training.)
Regarding claim 3 depends on claim 2, thus the rejection of claim 2 is incorporated.
Zhao teaches the limitation “The method for generating a graph neural network of claim 2, wherein the plurality of memories comprises a plurality of the graphics processing unit memories” (page 5 section 3 col. 1 “HBM-PS is distributed in the High-Bandwidth Memory (HBM) across multiple GPUs.” Zhao discloses High-Bandwidth Memory (HBM) across multiple GPUs, thereby corresponding to the plurality of the graphics processing unit memories, as claimed.)
Zhao teaches the limitation “obtaining the node features of the first nodes and the edge features of the edges connected to the first nodes from the graphics processing unit memory according to the first node identifications comprises: determining the first node identifications associated with respective graphics processing unit memories in the plurality of graphics processing unit memories, and obtaining the node features of corresponding first nodes and the edge features of the edges connected to the first nodes from the respective graphics processing unit memories using a plurality of threads through a thread queue” (page 5 section 3 col. 2 “We build a 4-stage pipeline to hide the latency of those tasks by maintaining a prefetch queue for each stage. A worker thread is created for each stage—it extracts jobs from the prefetch queue and feeds the corresponding hardware re source. After that, the worker thread pushes the processed results into the prefetch queue of the next stage. Especially, the worker thread stalls when the prefetch queue of the next stage is full—the next stage has already obtained too many unprocessed jobs.” Zhao discloses that the HBM parameter server is distributed across multiple GPUs, thereby corresponding to the claimed plurality of graphics processing unit memories. Zhao further teaches a plurality of worker threads operating through respective prefetch queues, wherein each worker thread extracts jobs from a thread queue and feeds the corresponding hardware resource before pushing processed results to the next queue. A person of ordinary skill in the art would understand that, in Zhao’s distributed GPU architecture, the worker threads are responsible for retrieving and processing the parameters stored in the respective GPU memories. Accordingly, Zhao’s disclosed worker threads operating through the prefetch queues correspond to determining first node identifications associated with the respective graphics processing unit memories and obtaining the corresponding node and edge features from the respective graphics processing unit memories using a plurality of threads through a thread queue, as claimed.)
Regarding claim 11,
Guan teaches a part of the limitation “A non-transitory computer readable medium having stored thereon a computer program that, when executed by a processing device, the program performs acts” (paragraph 75 “The computer system apparatus 800 may include one or more processors and one or more non-transitory computer-readable storage media (e.g., one or more memories) coupled to the one or more processors and configured with instructions executable by the one or more processors to cause the system or device (e.g., the processor) to perform the above-described embodiments”. A person of ordinary skill in the art would have found it obvious to implement the graph processing methods and system of Zheng/Zhao as processor executable instructions stored on Guan’s non-transitory computer readable storage media and executed by Guan’s processing device, thereby provide a practical and predictable software-based implementation of the disclosed operation.)
Claim 11 is further rejected under the same rationale as claim 1. The applicant is further directed to the rejection of claim 1 above, because the claim recites similar limitations and processing steps.
Regarding claim 12,
Guan teaches a part of the limitation “An electronic device, comprising: a storage device having stored thereon one or more computer programs; and one or more processing devices configured to execute the one or more computer programs in the storage device to implement” (paragraph 79 “When the functions disclosed herein are implemented in the form of software functional units and sold or used as independent products, they can be stored in a processor executable non-volatile computer-readable storage medium ... The software product may be stored in a storage medium, including a number of instructions to cause a computing device (which may be a personal computer, a server, a network device, and the like) to execute all or some steps of the methods of the embodiments of the present application” A person of ordinary skill in the art would have found it obvious to implement the graph processing methods and system of Zheng/Zhao as processor executable instructions stored on Guan’s non-transitory computer readable storage media and executed by Guan’s processing device, thereby provide a practical and predictable software-based implementation of the disclosed operation.)
Claim 12 is further rejected under the same rationale as claim 1. The applicant is further directed to the rejection of claim 1 above, because the claim recites similar limitations and processing steps.
Regarding claim 13 depends on claim 11, thus the rejection of claim 11 is incorporated. The claim is further rejected under the same rationale as claim 2. The applicant is further directed to the rejection of claim 2 above, because the claim recites similar limitations and processing steps.
Regarding claim 14 depends on claim 13, thus the rejection of claim 13 is incorporated. The claim is further rejected under the same rationale as claim 3. The applicant is further directed to the rejection of claim 3 above, because the claim recites similar limitations and processing steps.
Regarding claim 15 depends on claim 14, thus the rejection of claim 13 is incorporated. The claim is further rejected under the same rationale as claim 4. The applicant is further directed to the rejection of claim 4 above, because the claim recites similar limitations and processing steps.
Regarding claim 16 depends on claim 12, thus the rejection of claim 12 is incorporated. The claim is further rejected under the same rationale as claim 2. The applicant is further directed to the rejection of claim 2 above, because the claim recites similar limitations and processing steps.
Regarding claim 17 depends on claim 16, thus the rejection of claim 16 is incorporated. The claim is further rejected under the same rationale as claim 3. The applicant is further directed to the rejection of claim 3 above, because the claim recites similar limitations and processing steps.
Regarding claim 18 depends on claim 12, thus the rejection of claim 12 is incorporated. The claim is further rejected under the same rationale as claim 4. The applicant is further directed to the rejection of claim 4 above, because the claim recites similar limitations and processing steps.
Claims 5, 6 are rejected under 35 U.S.C. 103 as being unpatentable over Zheng et.al (NPL: DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs) in view of Zhao et.al (NPL: Distributed Hierarchical GPU parameter server for massive scale Deep Learning Ads systems), further in view of Neelam et.al (US 20230169115 A1), further in view of Elyasi et.al (US 20200183604 A1)
Regarding claim 5 depends on claim 4, thus the rejection of claim 4 is incorporated.
Zheng /Zhao does not teach the limitation “segmenting a full graph into a plurality of graph blocks according to a breadth-first search algorithm, each of the graph blocks comprising the corresponding number of the nodes and edges connected to the nodes”. However, Neelam teaches or at least suggest this limitation (Paragraph 19 “the in-memory graph 104 is processed, in conjunction with constraint data 106, by graph analysis and partitioning component 108. For example, in at least one embodiment, the in-memory graph 104 is partitioned, by graph analysis and partitioning component 108, into N disjoint vertex sets for N degrees of parallelism ... Such partitioning can be carried out, for example, using one or more polynomial time partition algorithms (e.g., one or more breadth-first search (BFS) algorithms, ...)” Neelam discloses segmenting an in-memory graph into a plurality of graph portions using a breadth-first search algorithm. Neelam teaches partitioning the in-memory graph into N disjoint vertex sets for N degrees of parallelism and identifies one or more breadth-first search algorithms as suitable polynomial-time partitioning algorithms. Neelam further describes graph adjacency information identifying incident edges of the vertices. Accordingly, a person of ordinary skill in the art would have understood Neelam’s in-memory graph as corresponding to the claimed full graph, and each resulting disjoint vertex set, together with the incident edges identified by the graph’s adjacency information, as corresponding to a graph block comprising nodes and edges connected to the nodes. Thus, Neelam teaches or at least suggests segmenting a full graph into a plurality of graph blocks according to a breadth-first search algorithm, each graph block comprising a corresponding number of nodes and edges connected to the nodes, as claimed.)
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine the teaching of DistDGL for training large-scale graph neural network through subgraph and in-memory KVStore servers, and the teaching of distributed hierarchical parameter server utilizing multiple memories for training massive-scale neural networks by Zheng/Zhao with the teaching of segmenting an in-memory graph into a plurality of graph portions using a breadth-first search algorithm by Neelam. The motivation to do so is referred to in Neelam’s disclosure (paragraph 2 “graph ingestion can be very slow due to its storage model, and if ingestion is attempted to be hastened through multiple threads, it can result in an inconsistent graph which may not fulfill constraint requirements”, and paragraph 17 “one or more embodiments include partitioning and parallel loading of property graphs with constraints in an incremental fashion. Such an embodiment includes portioning an in-memory graph to support loading a partial graph with one or more source constraints, as well as performing thread assignment that considers and/or satisfies source constraints while loading data into a property graph” Neelam discloses that graph ingestion can be slow because of the graph storage model and that attempting parallel ingestion through multiple threads without an appropriate partitioning mechanism can produce an inconsistent graph that fails to satisfy applicable constraints. Neelam addresses this problem by partitioning an in-memory graph into disjoint subgraphs, identifying breadth-first search as a suitable polynomial-time partitioning algorithm, and assigning the resulting subgraphs for parallel loading while accounting for graph-related constraints. Although Neelam discusses property graphs, both Neelam and Zheng process graph structures comprising vertices and edges and divide large graphs into smaller graph portions for parallel or distributed processing. A person of ordinary skill in the art therefore would have been motivated to employ Neelam’s known BFS-based partitioning technique in the Zheng/Zhao system to generate discrete, connected graph blocks suitable for subsequent distribution and processing, thereby facilitating parallel handling of the large graph while maintaining graph consistency and obtaining the predictable result of efficiently generated graph portions for distributed GNN training.)
Zheng /Zhao/Neelam does not teach the limitation “allocating the plurality of graph blocks to a plurality of graph slices”. However, Elyasi teaches or at least suggest the limitation (Paragraph 50 “Embodiments may include a computer-implemented method for partitioning graph data for large-scale graph processing. The method may include, in a partitioning pre-processing step, assigning a plurality of destination vertices to a plurality of partitions such that each destination vertex of the plurality of destination vertices is uniquely assigned to only one partition from among the plurality of partitions” Elyasi discloses allocating the plurality of graph blocks to a plurality of graph slices. Specifically, Elyasi teaches, during a graph-partitioning preprocessing step, assigning a plurality of destination vertices to a plurality of partitions such that each destination vertex is uniquely assigned to one of the partitions. A person of ordinary skill in the art would have understood that a destination vertex together with its associated source vertices, neighboring information, and edge data constitutes a portion of the graph corresponding to a graph block, and that Elyasi’s partitions correspond to the claimed graph slices. Accordingly, assigning the respective graph portions to the plurality of partitions corresponds to allocating the plurality of graph blocks to the plurality of graph slices. When applied to the distributed graph-processing environment of Zheng, as modified by Zhao and Neelam, Elyasi’s partition assignment provides the placement of the graph blocks generated using Neelam’s BFS-based segmentation into respective partitions for distributed GNN processing, as claimed.)
Zheng /Zhao/Neelam does not teach the limitation “performing, based on the full graph, structural restoration on the graph blocks included in each of the plurality of graph slices, to determine the plurality of subgraph structures” However, Elyasi teaches or at least suggest the limitation (Paragraph 50 “The method may include, in a main execution of external graph processing step, (i) loading a given partition of destination vertices from among the plurality of partitions from a solid state drive (SSD) into a main memory of a computing machine, (ii) streaming one or more chunks of source vertex data from the SSD into the main memory of the computing machine, and (iii) performing graph processing based at least on the loaded given partition of destination vertices and the streamed one or more chunks of source vertex data”, and Paragraph 51 “updating the loaded given partition of destination vertices based at least on the streamed one or more chunks of source vertex data ... The method may further include generating mirror updates to mirrors associated with destination vertices for a plurality of other partitions associated with the given partition.” Elyasi discloses loading a given partition of destination vertices into main memory, streaming associated source-vertex data, reading neighboring information from edge data, and performing graph processing based on the loaded destination-vertex partition and the streamed source-vertex data. Elyasi further teaches updating the loaded partition based on the streamed source-vertex data and generating mirror updates for corresponding vertices maintained in other associated partitions. A person of ordinary skill in the art would have understood that bringing together the destination vertices, associated source vertices, neighboring information, incident edges, and mirror updates reestablishes the graph connectivity represented in the full graph for the respective partition, thereby corresponding to performing structural restoration on the graph blocks included in each graph slice to determine the respective subgraph structures. In the Zheng/Zhao distributed GNN system, the resulting restored subgraph structures provide the connected node-and-edge information used for distributed graph-neural-network processing.)
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine the teaching of DistDGL for training large-scale graph neural network through subgraph and in-memory KVStore servers, the teaching of distributed hierarchical parameter server utilizing multiple memories for training massive-scale neural networks, and the teaching of segmenting an in-memory graph into a plurality of graph portions using a breadth-first search algorithm by Zheng/Zhao/Neelam with the teaching of assigning vertices to a plurality of partitions and updating the loaded partition based on the streamed source-vertex data by Elyasi. The motivation to do so is referred to in Elyasi’s disclosure (paragraph 20 “Embodiments disclosed herein include a greedy partitioning mechanism for vertex data such that the destination vertices in each partition from among a group of partitions readily fit in main memory DRAM space, hence eliminating the need for sorting the intermediate fine-grained data. The partitioning is done prior to the execution as a pre-processing phase”, paragraph 26 “Thus, given a limited amount of DRAM space to run the graph application, even vertex data needs be stored on the SSD and be read into the main memory in different time intervals”, and paragraph 45 “This disclosure devises a greedy partitioning technique for vertex data in external graph processing. Vertex data for each partition is sufficiently small to fit in the main memory, thereby only needing to pin destination vertex data for the current partition in the memory. The rest of the data may be transferred and processed in memory in a streaming fashion ... Accesses to source vertex data and edge data can be done in parallel (i.e., enabling high levels of parallelism) ... The partitioning may be performed prior to the main process execution, thereby resulting in a one-time cost since it only happens once in the pre-processing step” Elyasi discloses that, as graph datasets increase in size, limited DRAM capacity makes retaining all vertex and edge data in main memory inefficient or impractical. Elyasi addresses this problem by assigning graph data to partitions whose destination-vertex data is sufficiently small to fit in main memory, while transferring and processing the associated source-vertex and edge data in a streaming manner. Elyasi further discloses that source-vertex and edge-data accesses may be performed in parallel, intermediate fine-grained sorting may be avoided, and the partitioning may be performed once as a preprocessing operation. Accordingly, a person of ordinary skill in the art would have been motivated to further modify the distributed graph-neural-network system of Zheng, as modified by Zhao and Neelam, with Elyasi’s partition-assignment and streaming technique by assigning the graph portions generated through Neelam’s BFS-based segmentation, which correspond to the claimed graph blocks, to respective Elyasi partitions, which correspond to the claimed graph slices, and by loading, processing, and updating the associated destination-vertex, source-vertex, and edge information for each partition to determine the corresponding subgraph structure. Such a modification would have predictably accommodated limited main-memory capacity, reduced sorting and storage-access overhead, and enabled efficient parallel processing of large-scale graph data in the distributed GNN system.)
Regarding claim 6 depends on claim 5, thus the rejection of claim 5 is incorporated.
Elyasi teaches or at least suggest the limitation “allocating the plurality of graph blocks to the plurality of graph slices in accordance with a predetermined order” (paragraph 40 “Source vertices may be stored for each partition in a sorted order”, and paragraph 50 “Embodiments may include a computer-implemented method for partitioning graph data for large-scale graph processing. The method may include, in a partitioning pre-processing step, assigning a plurality of destination vertices to a plurality of partitions such that each destination vertex of the plurality of destination vertices is uniquely assigned to only one partition from among the plurality of partitions” Elyasi discloses generating mirror updates to mirrors associated with destination vertices for a plurality of other partitions associated with a given partition. Since Elyasi’s partitions correspond to the claimed graph slices, the generated mirror updates constitute information exchanged among the respective graph slices to maintain consistency of the partitioned graph data during graph processing and updating.)
Elyasi teaches or at least suggest the limitation “exchanging information of respective picture slices of the plurality of graph slices, and adjusting the graph blocks having association relationship into a same graph slice, to obtain a plurality of adjusted graph slices” (paragraph 51 “The method may further include generating mirror updates to mirrors associated with destination vertices for a plurality of other partitions associated with the given partition”, and paragraph 52 “... for each edge ... in which vertex “u” is associated with vertex “v” in a “u→v” relationship, determining whether the vertex “v” is already assigned to a partition “P.” In response to determining that the vertex “v” is already assigned to the partition “P,” the method may include determining whether the vertex “u” already exists on the partition “P.” In response to determining that the vertex “u” does not already exist on the partition “P,” the method may include adding the vertex “u” to the partition “P.” ...)”. Elyasi discloses that for an edge in which vertex u is associated with vertex v in a u→v relationship, determining whether vertex v has already been assigned to partition P, and, if vertex u does not already exist in partition P, adding vertex u to partition P. A person of ordinary skill in the art would have understood that Neelam’s BFS-generated graph blocks comprise respective vertices and their associated edges. Thus, when the exchanged information indicates that vertices belonging to different graph blocks are associated, the corresponding graph blocks likewise have an association relationship and would have been adjusted into the same Elyasi partition. Since Elyasi’s partitions correspond to the claimed graph slices, the resulting partition corresponds to the claimed adjusted graph slice.)
Claims 7, 8 are rejected under 35 U.S.C. 103 as being unpatentable over Zheng et.al (NPL: DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs) in view of Zhao et.al (NPL: Distributed Hierarchical GPU parameter server for massive scale Deep Learning Ads systems), further in view of Neelam et.al (US 20230169115 A1), further in view of Elyasi et.al (US 20200183604 A1), further in view of Aronovich et. al (US 20150019511 A1).
Regarding claim 7 depends on claim 5, thus the rejection of claim 5 is incorporated.
Neelam in view of Elyasi teaches the fusing aspect within the limitation “fusing a first target graph block having a node number less than a first predetermined threshold with other graph blocks to obtain a plurality of fused graph blocks, the other graph blocks being graph blocks obtained after removing the first target graph block from the plurality of graph blocks” (Neelam discloses at paragraph 19 “the in-memory graph 104 is partitioned, by graph analysis and partitioning component 108, into N disjoint vertex sets”, and Elyasi discloses at paragraph 52 “... for each edge ... in which vertex “u” is associated with vertex “v” in a “u→v” relationship, determining whether the vertex “v” is already assigned to a partition “P.” In response to determining that the vertex “v” is already assigned to the partition “P,” the method may include determining whether the vertex “u” already exists on the partition “P.” In response to determining that the vertex “u” does not already exist on the partition “P,” the method may include adding the vertex “u” to the partition “P.” ...” Neelam discloses partitioning an in-memory graph into N disjoint vertex sets, thereby providing a plurality of distinct graph portions. Elyasi discloses determining that vertices u and v are associated through an edge u -> v and, when vertex v is assigned to partition P but vertex u is not present therein, adding vertex u to the same partition P. In the combination, a selected one of Neelam’s disjoint vertex sets corresponds to the first target graph block, while the remaining disjoint vertex sets correspond to the other graph blocks obtained after removing the first target graph block from the plurality. Applying Elyasi’s association-based placement technique to graph data extending between the selected vertex set and the remaining vertex sets would combine the corresponding graph portions within common partitions. Thus, Neelam in view of Elyasi teaches or at least suggests fusing the first target graph block with the other graph blocks to obtain a plurality of fused graph blocks.)
Zheng/Zhao/Neelam/Elyasi does not teach the threshold-based comparison aspect “...a node number less than a first predetermined threshold...” within the limitation. However, Aronovich teaches or at least suggest this aspect within the limitation (paragraph 30 “the algorithm of the present invention applies a minimum bound on the size of the produced blocks, which is data dependent, and does not increase the sensitivity of the produced segmentation to the high level partition of the data ... In addition, applying a minimum size bound helps in reducing the variance of the block sizes, which increases the effectiveness of deduplication.” Aronovich discloses applying a minimum size bound to the size of produced blocks and explains that, without such a bound, a block may become excessively small, thereby increasing the storage required for maintaining block information; Aronovich further explains that applying the minimum size bound reduces variation among block sizes. A person of ordinary skill in the art would have understood that Neelam’s disjoint vertex sets correspond to respective graph blocks and that their respective vertex counts represent the sizes, or node numbers, of the corresponding graph blocks, and would have applied Aronovich’s minimum-size-bound technique to those vertex counts before performing the graph-block fusion suggested by Elyasi.)
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine the teaching of DistDGL for training large-scale graph neural network through subgraph and in-memory KVStore servers, the teaching of distributed hierarchical parameter server utilizing multiple memories for training massive-scale neural networks, the teaching of segmenting an in-memory graph into a plurality of graph portions using a breadth-first search algorithm, and the teaching of assigning vertices to a plurality of partitions and updating the loaded partition based on the streamed source-vertex data by Zheng/Zhao/Neelam/Elyasi, with the teaching of applying a minimum size bound to produced blocks by Aronovich. The motivation to do so is referred to in Aronovich’s disclosure (paragraph 30 “the algorithm of the present invention applies a minimum bound on the size of the produced blocks, which is data dependent, and does not increase the sensitivity of the produced segmentation to the high level partition of the data. A minimum size bound is required for facilitating efficient storage of blocks' information, because if there is no minimum size bound then a block size can be very small, thus entailing an increased amount of storage that should be allocated for storing the block's information. In addition, applying a minimum size bound helps in reducing the variance of the block sizes, which increases the effectiveness of deduplication.” Aronovich discloses applying a minimum size bound to produced blocks to prevent excessively small blocks, facilitate efficient storage of block information, and reduce variation among block sizes. A person of ordinary skill in the art would have been motivated to apply Aronovich’s known minimum-size-bound technique to the node numbers of Neelam’s disjoint vertex sets, which correspond to respective graph blocks, to identify an undersized graph block for fusion according to Elyasi’s graph-partition technique, thereby reducing the storage and management overhead associated with excessively small graph blocks.)
Regarding claim 8 depends on claim 5, thus the rejection of claim 5 is incorporated.
Neelam teaches or at least suggest a part of the limitation “determining, from the plurality of graph blocks ... a second target graph block, ... and a third target graph block ...” (paragraph 19 “the in-memory graph 104 is partitioned, by graph analysis and partitioning component 108, into N disjoint vertex sets” Neelam discloses partitioning the in-memory graph into N disjoint vertex sets, thereby producing a plurality of distinct graph portions. A person of ordinary skill in the art would have understood each disjoint vertex set to correspond to a respective graph block and the number of vertices contained in each vertex set to correspond to the node number of that graph block. Accordingly, two different vertex sets selected from the N disjoint vertex sets correspond to the claimed second target graph block and third target graph block.)
Aronovich teaches or at least suggest the threshold-based comparison technique within the limitation “determining, from the plurality of graph blocks, a second target graph block having a node number less than a second predetermined threshold, and a third target graph block having a node number greater than a third predetermined threshold; the third predetermined threshold being greater than the second predetermined threshold” (paragraph 28 “the algorithm of the present invention applies a maximum bound on the size of the produced blocks, such that the segmenting positions produced by applying the maximum size bound are data dependent. This is in contrast to existing methods that apply a maximum size bound by applying arbitrary segmenting positions, which are not data dependent, thus reducing the effectiveness of deduplication. By applying a maximum size bound, which is data dependent, by the algorithm of the present invention, the deduplication effectiveness is considerably increased ... applying a maximum size bound helps in reducing the variance of the block sizes, and reducing this variance increases the effectiveness of deduplication”, and paragraph 30 “the algorithm of the present invention applies a minimum bound on the size of the produced blocks, which is data dependent, and does not increase the sensitivity of the produced segmentation to the high level partition of the data. A minimum size bound is required for facilitating efficient storage of blocks' information, because if there is no minimum size bound then a block size can be very small, thus entailing an increased amount of storage that should be allocated for storing the block's information. In addition, applying a minimum size bound helps in reducing the variance of the block sizes, which increases the effectiveness of deduplication”. Aronovich discloses applying a maximum size bound and a minimum size bound to produced blocks to address excessively large and excessively small blocks, respectively, and to reduce variation among block sizes. A person of ordinary skill in the art would have understood that applying such maximum and minimum size bounds entails evaluating block sizes relative to an upper threshold and a lower threshold, with the maximum-size threshold being greater than the minimum-size threshold. In the combination, Neelam’s disjoint vertex sets correspond to respective graph blocks, and the respective numbers of vertices correspond to the node numbers of those graph blocks. Accordingly, applying Aronovich’s dual-bound technique to Neelam’s disjoint vertex sets would suggest determining one corresponding graph block having a node number below the minimum-size threshold and another corresponding graph block having a node number above the greater maximum-size threshold, thereby corresponding to the claimed second target graph block and third target graph block, respectively.)
Elyasi teaches or at least suggest the limitation “if the second target graph block and the third target graph block have association relationship with each other, fusing the second target graph block with the third target graph block to obtain a plurality of fused graph blocks” (paragraph 51 “The method may further include generating mirror updates to mirrors associated with destination vertices for a plurality of other partitions associated with the given partition”, and paragraph 52 “... for each edge ... in which vertex “u” is associated with vertex “v” in a “u→v” relationship, determining whether the vertex “v” is already assigned to a partition “P.” In response to determining that the vertex “v” is already assigned to the partition “P,” the method may include determining whether the vertex “u” already exists on the partition “P.” In response to determining that the vertex “u” does not already exist on the partition “P,” the method may include adding the vertex “u” to the partition “P.” ...” Elyasi discloses identifying an association relationship between graph data based on an edge u -> v. Elyasi further determines whether vertex v is assigned to partition P and, when vertex v is assigned to partition P but associated vertex u is not already present, adds vertex u to the same partition P. Thus, Elyasi teaches or at least suggests co-locating graph portions having an association relationship within the same partition. A person of ordinary skill in the art would have understood that, when vertices of Neelam’s second and third disjoint vertex sets are connected by such edge relationships, applying Elyasi’s association-based placement to the associated vertices of the respective sets would combine the associated graph portions within the same partition, corresponding to fusing the second target graph block with the third target graph block to obtain a plurality of fused graph blocks.)
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine the teaching of Zheng/Zhao/Neelam/Elyasi, with the teaching of applying maximum and minimum size bounds to produced blocks by Aronovich. The motivation to do so is referred to in Aronovich’s disclosure (paragraph 28 “the algorithm of the present invention applies a maximum bound on the size of the produced blocks, such that the segmenting positions produced by applying the maximum size bound are data dependent ... By applying a maximum size bound, which is data dependent, by the algorithm of the present invention, the deduplication effectiveness is considerably increased ... applying a maximum size bound helps in reducing the variance of the block sizes, and reducing this variance increases the effectiveness of deduplication”, and paragraph 30 “the algorithm of the present invention applies a minimum bound on the size of the produced blocks, which is data dependent, and does not increase the sensitivity of the produced segmentation to the high level partition of the data ... In addition, applying a minimum size bound helps in reducing the variance of the block sizes, which increases the effectiveness of deduplication.” Aronovich discloses applying maximum and minimum size bounds to produced blocks to address excessively large and excessively small blocks, respectively, and to reduce variation among block sizes. A person of ordinary skill in the art would therefore have been motivated to apply Aronovich’s dual-size-bound technique to the respective vertex counts of Neelam’s disjoint vertex sets, which correspond to the node numbers of the respective graph blocks, to provide size-aware identification of undersized and oversized graph blocks before applying Elyasi’s association-based fusion technique. Such a modification would predictably reduce block-size variation and the storage and management overhead associated with inadequately sized graph blocks while retaining the benefit of placing associated graph data together.)
Claims 19, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zheng et.al (NPL: DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs) in view of Zhao et.al (NPL: Distributed Hierarchical GPU parameter server for massive scale Deep Learning Ads systems) further in view of Guan et.al (US 20230064080 A1), further in view of Neelam et.al (US A1), further in view of Elyasi et.al (US A1)
Regarding claim 19 depends on claim 18, thus the rejection of claim 18 is incorporated.
The rejection of claim 18, including the teaching of Guans and the rationale for combining Guan with Zheng and Zhao, is incorporate herein. It would have been obvious to further modify the system of Zheng/Zhao/Guan with the teaching of graph partitioning and processing techniques by Neelam and Elyasi for the same reasons discussed above with respect to claim 5. Guan’s computer-readable medium and processing-device-implementation does not alter the graph-processing operations of Zheng, Zhao or Neelam and Elyasi. Accordingly, Neelam’s and Elyasi’s teachings would have performed their known functions in the Zheng/Zhao/Guan’s teaching combination. The applicant is further directed to the rejection of claim 5 above for the corresponding teachings and rationales.
Regarding claim 20 depends on claim 19, thus the rejection of claim 19 is incorporated.
The rejection of claim 20, including the teaching of Guans and the rationale for combining Guan with Zheng and Zhao, is incorporate herein. It would have been obvious to further modify the system of Zheng/Zhao/Guan with the teaching of graph partitioning and processing techniques by Neelam and Elyasi for the same reasons discussed above with respect to claim 6. Guan’s computer-readable medium and processing-device-implementation does not alter the graph-processing operations of Zheng, Zhao or Neelam and Elyasi. Accordingly, Neelam’s and Elyasi’s teachings would have performed their known functions in the Zheng/Zhao/Guan’s teaching combination. The applicant is further directed to the rejection of claim 6 above for the corresponding teachings and rationales.
Claims 21, 22 are rejected under 35 U.S.C. 103 as being unpatentable over Zheng et.al (NPL: DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs) in view of Zhao et.al (NPL: Distributed Hierarchical GPU parameter server for massive scale Deep Learning Ads systems) further in view of Guan et.al (US 20230064080 A1), further in view of Neelam et.al (US A1), further in view of Elyasi et.al (US A1), further in view of Aronovich et. al (US 20150019511 A1).
Regarding claim 21 depends on claim 19, thus the rejection of claim 19 is incorporated.
The rejection of claim 21, including the teaching of Guans and the rationale for combining Guan with Zheng and Zhao, is incorporate herein. It would have been obvious to further modify the system of Zheng/Zhao/Guan/Neelam/Elyasi with the teaching of Aronovich’s size bounds technique for the same reasons discussed above with respect to claim 7. Guan’s computer-readable medium and processing-device-implementation does not alter the graph-processing operations of Zheng, Zhao, Neelam, Elyasi or Aronovich. Accordingly, Aronovich’s teachings would have performed their known functions in the Zheng/Zhao/Guan/Neelam/Elyasi’s teaching combination. The applicant is further directed to the rejection of claim 7 above for the corresponding teachings and rationales.
Regarding claim 22 depends on claim 19, thus the rejection of claim 19 is incorporated.
The rejection of claim 22, including the teaching of Guans and the rationale for combining Guan with Zheng and Zhao, is incorporate herein. It would have been obvious to further modify the system of Zheng/Zhao/Guan/Neelam/Elyasi with the teaching of Aronovich’s size bounds technique for the same reasons discussed above with respect to claim 8. Guan’s computer-readable medium and processing-device-implementation does not alter the graph-processing operations of Zheng, Zhao, Neelam, Elyasi or Aronovich. Accordingly, Aronovich’s teachings would have performed their known functions in the Zheng/Zhao/Guan/Neelam/Elyasi’s teaching combination. The applicant is further directed to the rejection of claim 8 above for the corresponding teachings and rationales.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUY TU DIEP whose telephone number is (703)756-1738. The examiner can normally be reached M-F 8-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY T DIEP/ Examiner, Art Unit 2123
/ALEXEY SHMATOV/ Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights