DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/29/2024 and 05/22/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. KR10-2024-0027642, filed on 02/27/2024.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over
Zhong et al. (“Hierarchical Message-Passing Graph Neural Networks”) in view of You et al. (“Graph Structure of Neural Networks”).
Regarding claim 1
Zhong teaches a method performed by a neural network model learning apparatus operated by a processor, the method comprising the steps of: creating nodes and edges constituting a hierarchical tree-structured system and assigning local information to each node; (“Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.”)
arranging, in each node, a first neural network including parameters for generating upward information transferred from a first node to a parent node, a second neural network including parameters for the first node to generate control information to be executed by the first node, (Fig. 2 “(a)The architecture of Hierarchical Message-passing Graph Neural Networks: we first generate a hierarchical structure, in which each level is formed as a super graph, use the level t graph to update nodes of level t+1graph(bottom-up propagation)…”)
and a third neural network including parameters for generating downward information transferred from the first node to child nodes; (“Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.” Examiner notes that Level 3 message get pass down to level 1 first node)
…
and determining control information to be executed on its own local information by each node constituting the hierarchical tree structure. (Algorithm 1 shows input graph and output node representation “We summaries the process of Hierarchical Message-passing Graph Neural Networks in Algorithm 1. Given a graph G, we first generate the hierarchical structure and combine it with the original graph G, to obtain H = {Gt|t = 1, 2,...,T}, where G1 = G (line 2). For each node, including original and generated super nodes, in each NN layer, we perform three primary operations in order: (1) bottom-up propagation (line 6), (2) within-level propagation (line 7), and (3) top-down propagation (line 9−15). After getting the representation vector of each node that is enhanced with informative long-range interactions and multi-grained semantics, and we train the model with the loss function L in Eq. 7.”)
Zhong does not teach and learning the parameters of the first, second, and third neural networks to maximize an expected value of the system according to a result of receiving the upward information and the downward information on the basis of the first, second, and third neural networks.
You teaches and learning the parameters of the first, second, and third neural networks to maximize an expected value of the system according to a result of receiving the upward information and the downward information on the basis of the first, second, and third neural networks. (Section 3.1 “Given the complex nature of graph structure, graph measures are often used to characterize graphs. In this paper, we focus on one global graph measure, average path length, and one local graph measure, clustering coefficient. No tably, these two measures are widely used in network sci ence (Watts & Strogatz, 1998) and neuroscience (Sporns, 2003; Bassett & Bullmore, 2006). Specifically, average path length measures the average shortest path distance between any pair of nodes; clustering coefficient measures the pro portion of edges between the nodes within a given node’s neighborhood, divided by the number of edges that could possibly exist between them, averaged over all the nodes.”)
Zhong and You are analogous art because they are both directed to Machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for combining hierarchical message passing graph neural network by Zhong to include rational graph structure of neural networks of You in order to provide a “smooth function of the clustering coefficient and average path length of its relational graph;” as disclosed by You (abstract “We develop a novel graph-based representation of neural networks called relational graph, where layers of neural network computation correspond to rounds of message exchange along the graph structure. Using this representation we show that: (1) a “sweet spot” of relational graphs leads to neural networks with significantly improved predictive performance; (2) neural network’s performance is approximately a smooth function of the clustering coefficient and average path length of its relational graph;”).
Regarding claim 2
Zhong in view of You teaches the method according to claim 1.
Zhong further teaches wherein the first neural network is set to generate second upward information to be transferred to the parent node according to the parameters learned on the basis of first upward information transferred from a child node and local information of the first node. (“Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.”)
Regarding claim 3
Zhong in view of You teaches the method according to claim 2.
Zhong further teaches wherein the first node is set to derive, when the first node has a plurality of child nodes, a plurality of information according to the first neural network on the basis of the first upward information transferred from each child node and the local information of the first node, and generate one piece of second upward information by integrating the plurality of information on the basis of a predetermined integration function. (“Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.”)
Regarding claim 4
Zhong in view of You teaches the method according to claim 1.
Zhong further teaches wherein the second neural network is set to determine control information to be executed by the first node according to parameters learned on the basis of the first upward information transferred from the child node, first downward information transferred from the parent node, and the local information of the first node. (“Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.”)
Regarding claim 5
Zhong in view of You teaches the method according to claim 4.
Zhong further teaches wherein the first node is set to generate, when the first node has a plurality of child nodes, first upward information obtained by integrating upward information transferred from each child node according to a predetermined integration function, and the second neural network is set to determine control information to be executed by the first node according to the parameters learned on the basis of the integrated first upward information, first downward information transferred from the parent node, and its own local information. (“Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.” Examiner notes that FIG. 1 show different levels of neural network architecture where massages are passed between each different levels and see child nodes from figures)
Regarding claim 6
Zhong in view of You teaches the method according to claim 5.
Zhong further teaches wherein the third neural network is set to generate second downward information to be transferred to a child node according to parameters learned on the basis of the first upward information transferred from the child node, the first downward information transferred from the parent node, the local information of the first node, and the control information of the first node. (“Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.” Examiner notes that FIG. 1 show different levels of neural network architecture where massages are passed between each different levels and Fig 1(b) specifically show level 3 corresponds to third neural network generating message to downward to the child node level 1)
Regarding claim 7
Zhong in view of You teaches the method according to claim 6.
Zhong further teaches wherein the first node is set to generate, when the first node has a plurality of child nodes, a plurality of second downward information to be transferred to each child node according to the third neural network on the basis of the first upward information transferred from each child node, the first downward information transferred from the parent node, its own local information, and control information determined by itself. . (“Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.” Examiner notes that FIG. 1 show different levels of neural network architecture where massages are passed between each different levels and Fig 1(b) specifically show level 3 corresponds to third neural network generating message to downward to the child node level 1)
Regarding claim 8
Zhong in view of You teaches the method according to claim 1.
Zhong further teaches wherein the step of assigning local information includes the steps of: determining the number of tiers in the hierarchical tree structure; determining the number of nodes in each tier; and arranging nodes in each tier according to the number of tiers and the number of nodes in each tier, and connecting edges between nodes of upper and lower tiers. (see the different levels “Fig. 1 Elaboration of the proposed hierarchical message passing: (a) a collaboration network, (b) an illustration of hierarchical message-passing mechanism based on (a) and (c), and (c) an example of the identified hierarchical structure.”)
Regarding claim 9
Zhong in view of You teaches the method according to claim 8.
You further teaches wherein the step of determining the number of nodes in each tier includes the step of setting a minimum value and a maximum value of the number of nodes in at least any one tier, the step of connecting edges includes the step of creating a plurality of hierarchical tree structures according to each number of cases between the minimum value and the maximum value of the number of nodes in at least any one tier, and the step of learning the parameters of the first, second, and third neural networks includes the step of learning the parameters of the first, second, and third neural networks by applying the first, second, and third neural networks to the plurality of hierarchical tree structures. (pg. 7 “we discover that top-performing graphs tend to cluster into a sweet spot in the space defined by C and L (red rectangles in Figure 4(f)). We follow these steps to identify a sweet spot: (1) we downsample and aggregate the 3942 graphs in Figure 4(a) into a coarse resolution of 52 bins, where each bin records the performance of graphs that fall into the bin; (2) we identify the bin with best average performance (red cross in Figure 4(f)); (3) we conduct one tailed t-test over each bin against the best-performing bin, and record the bins that are not significantly worse than the best-performing bin (p-value 0.05 as threshold). The minimum area rectangle that covers these bins is visualized as a sweet spot. For 5-layer MLP on CIFAR-10, the sweet spot is C ∈ [0.10,0.50], L ∈ [1.82,2.75].”)
Regarding claim 10
Zhong in view of You teaches the method according to claim 1.
Zhong further teaches wherein the step of learning the parameters of the first, second, and third neural networks includes the step of setting the parameters of the first, second, and third neural networks embedded in all nodes of the hierarchical tree structure to be learned equally. (pg. 3 “. Taking the collaboration network in Fig. 1-(a) as an example; author nodes highlighted in light yellow come from the same institutes, and nodes filled with different colours indicate authors in various research areas. In order to generate the node representation of a given author, existing GNNs mainly capture the co-author level informa tion depending on the explicit graph structure. However, information hidden at meso and macro levels is neglected. In the example of Fig. 1, meso-level information means authors belong to the same institutes and their connections to adjacent institutes.”)
Regarding claim 11
Zhong in view of You teaches the method according to claim 1.
Zhong further teaches wherein the step of learning the parameters of the first, second, and third neural networks includes the step of setting the parameters of the first, second, and third neural networks embedded in the nodes of the same tier of the hierarchical tree structure to be learned equally, and setting the parameters of the first, second, and third neural networks embedded in the nodes of different tiers of the hierarchical tree structure to be learned differently. (pg. 3 “. Taking the collaboration network in Fig. 1-(a) as an example; author nodes highlighted in light yellow come from the same institutes, and nodes filled with different colours indicate authors in various research areas. In order to generate the node representation of a given author, existing GNNs mainly capture the co-author level informa tion depending on the explicit graph structure. However, information hidden at meso and macro levels is neglected. In the example of Fig. 1, meso-level information means authors belong to the same institutes and their connections to adjacent institutes.”)
Regarding claim 12
Claim 12 recites analogous limitations to independent claim 1 and therefore is rejected on the same ground as independent claim 1.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN C MANG whose telephone number is (571)270-7598. The examiner can normally be reached Mon - Fri 8:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at 5712707519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN C MANG/Primary Examiner, Art Unit 2126
Prosecution Insights