DETAILED ACTION
Response to Amendment
The amendment filed on 17 February 2026 has been entered.
Claims 1-12 are pending.
Claims 4-8, 11 are amended.
Applicant’s amendments to the Drawings and Claims have overcome each and every objection previously set forth in the Non-Final Office Action mailed 17 November 2025.
Response to Arguments
Applicant’s arguments, filed 17 February 2026, with respect to Claims 1-4, 6-12 under 35 USC 101 have been fully considered and are persuasive. The rejections of Claims 1-4, 6-12 under 35 USC 101 have been withdrawn.
Applicant’s remarks, regarding the rejections of claims under 35 USC 103, have been fully considered.
Applicant contends that Claim 1 is not obvious and unpatentable over Sztyler in view of
Marvaniya and Wang because the combination of cited references does not teach or render obvious each and every element of Claim 1. For example, the combination of cited references does not teach or render obvious, "assigning the blank node to a bin from a set of bins depending on the topology- based measure that has been computed for the current center node."
Applicant contends that while Wang's CaGCN approach is topology-aware in the sense that it employs another GCN as a calibration function to propagate confidence along the network topology, this is distinct from the claimed method. Wang's topology-awareness results in each node receiving its own unique temperature, whereas the claimed method assigns the blank node to a bin based on a topology-based measure computed specifically for the current center node, and then applies a scaling factor associated with that bin. The claimed method requires a specific two-step process: (1) computing a topology-based measure for the center node, and (2) assigning the blank node to a bin based on that measure. Depending on the same-class-neighbor ratio (or another topology-based measure) of the current center node, the classification vector of the blank node is calibrated using the scaling factor for the assigned bin. Moreover, Claim 1 recites a "blank node representing the new module, wherein the blank node has no technical attributes." None of the cited references teach or suggest this concept. The blank node is a placeholder for a module that does not yet exist in the engineering project and therefore has no attributes of its own. Because the blank node has no technical attributes, the claimed
method uses the topology-based measure of the center node to determine how to calibrate the blank node's confidence scores. This is a fundamentally different approach from calibrating a node based on its own properties, as taught by Wang.
Examiner respectfully disagrees. In response to Applicant's argument that the references fail to
show certain features of the invention, it is noted that the features upon which Applicant relies, under broadest reasonable interpretation (BRI), are given their plain meaning, unless such meaning is
inconsistent with the Specification, see MPEP § 2111.01(I).
As outlined in the Non-Final Office Action mailed 17 November 2025, pg. 16-18, and henceforth further elaborated, Wang teaches topology-aware post-hoc calibration method for GNNs, propagating confidence, naturally enabling that the confidence of topologically adjacent nodes becomes similar and a calibrated self-training model CaGCN-st in which the confidence is firstly calibrated then used to generate pseudo labels with high confidence (cf. Wang, [1 Introduction, pg. 2]), in a similar manner to the claimed invention. Wang teaches a classic temperature scaling method as a calibration function, summing the difference of confidence between neighbored nodes, using total variation of said confidence to evaluate calibration of the model to determine the confidence, associated with the predicted class label (cf. Wang, [3.1 CaGCN:GCNs as Calibration Function, pg. 4]), similar to the calculating a topology-based measure step of Claim 1 in light of the Specification [0029], outlining the topology-based measure as a proportion of the neighbors of the current center node for which the class with the highest determined preliminary confidence score is identical to the class with the highest determined preliminary confidence score for the current center node. The nodes are grouped into bins according to their confidence score (cf. Wang, [2 Notation and Preliminary Study, pg. 3]), similar to the assigning the blank node to a bin from a set of bins depending on the topology-based measure step of Claim 1. Examiner notes, under broadest reasonable interpretation (BRI), the nodes of Wang encompass labeled and unlabeled nodes. Further, the self-training model CaGCN-st of Wang trains and finds the calibrated confidence of each node, with the most confident predictions adopted as pseudo labels for blank node unlabeled nodes. Wang employs the temperature scaling method to calibrate and propagate node features along network topology, smoothing similar information between neighboured nodes, to generate labels for unlabeled nodes, similar to the calculating and assigning steps of Claim 1.
The rejection of Claim 1 under 35 USC 103 has been maintained. Rejections of Claims 2-12, which depends directly or indirectly from Claim 1 under 35 USC 103 have been maintained.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 7 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 7 recites the limitation "the classes" in line 7. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the classes" has been construed to be “classes”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 7-8, 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Sztyler et al. (U.S. Pre-Grant Publication No. 20240095805, hereinafter ‘Sztyler'), in view of Marvaniya et al. (U.S. Pre-Grant Publication No. 20220318831, hereinafter 'Marvaniya') and Wang et al. (NPL: "Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration", hereinafter 'Wang').
Regarding claim 1, Sztyler teaches A computer implemented method for recommending modules for an engineering project, comprising operations, wherein the operations are performed by components, and wherein the components are software components executed by one or more processors and/or hardware components, the method comprising: storing, in a database, a set of classes, with each class representing a type of module, and with each module including a hardware module and/or software module that is used in an engineering project ([0043] The graph analyzer module 110 communicates with the FLE prediction module 108 as well as with a diachronic analyzer module 112. Further, the graph analyzer module 110 enables interaction with a user 102 of the recommender system by means of a corresponding interface. The graph analyzer module 110 receives as input the input and output of the FLE prediction module 108, i.e. KGt and KGt+x, as well as a predefined with each class representing a type of module list of interest. This storing, in a database list of interest, which may be user-defined, may be provided in form of a list C={c1 . . . cn}, where ci denote a set of classes different classification labels.; [0078] According to a further application scenario, a recommender system according to embodiments of the invention may be applied in connection with product placement in smart retail scenarios. For instance, in a smart retail store that is equipped with cameras to track the customers, a recommender system according to embodiments of the invention can use the sensor network of the shop to observe the behavior of the customers. Thus, the system can analyze the buying behavior of the customer in respect of product placement but also analyze when products run out. Taking the each module including a hardware module and/or software module that is used in an engineering project state of interest into account (e.g. which products should be sold with priority, or when to actually restock certain products) the system can compute recommendations how to arrange products or when to restock them (e.g. considering perishable products). This also includes time-dependent product arrangement recommendations. Based on the output, an employee could adjust the product placement robot in terms of work instructions. Alternatively, the robot can be directly adjusted without any human in the loop (e.g. in an autonomous retail store).), and
with a current center node representing one of the previously selected modules to which a new module should be connected, wherein nodes connected to the current center node are neighbors of the current center node, with a blank node representing the new module, wherein the blank node has no technical attributes, and with an edge connecting the current center node to the blank node, computing, by a graph neural network, an embedding in latent space for each node or at least for the current center node and neighbors ([0040] Everything which is representing one of the previously selected modules to which a new module should be connected logged/observed (e.g. through a sensor network) by the AI unit 104 is transformed into a Knowledge Graph (KG), i.e., a set of triples. In this regard, the AI unit 104 may be seen as an information extraction pipeline that (continuously or periodically) extracts new triples. This set of triples represents with a current center node entities and objects, relation between them, and corresponding attributes. According to some embodiments, the AI unit 104 may be configured to store the information found in a Temporal Knowledge Graph (TKG), i.e. a KG that also contains temporal facts indicating relationships among entities and objects at different times. Embodiments of the invention wherein nodes connected to the current center node are neighbors of the current center node, with a blank node representing the new module, wherein the blank node has no technical attributes combine (i.e. take into account) the time-dependent triples with a list of actions and a scope of desired outcomes (i.e. States of Interest) to compute recommendations, as will be described in more detail below.; [0042] The FLE prediction module 108 may be implemented in form of a neural network whose weights may be trained with stochastic gradient descent (SGD) using the known knowledge. The FLE prediction module 108 produces a new Knowledge Graph (KGt+x) which represents the future of the input KG (KGt). In this context, it should be noted that in the with an edge connecting the current center node to the blank node, computing, by a graph neural network new Knowledge Graph the attributes of the entities and objects may have changed and that it might have new edges and nodes, while preexisting nodes and edges can vanish.; [0067] Multi-Modality: The recommender system according to embodiments of the invention is able incorporate multi-modal data through an embedding in latent space for each node or at least for the current center node and neighbors embedding the information for getting a vector representation. This enables the use of all available data sources (e.g. videos, images, recordings) and thus increases adaptiveness to different embodiments.);
determining, by a classifier, for each node embedding a preliminary confidence score for each class ([0045] According to an embodiment, the GA module 110 may further compute the difference between the respective two graphs via a weight matrix (w|KG_(t)−KG_(t+x)|), which reflects the actual changes between the two graphs. As a result, the GA module 110 returns the weight matrix and determining, by a classifier, for each node embedding a preliminary confidence score for each class for each value in the list of interest c∈C (i.e., for each possible class) a confidence score P for the respective KG (i.e. classification result m={(c, k, P(c|k))|∀c∈C, ∀k∈{KGt, KGt+x}}).);
Sztyler fails to teach a graph representing a current state of the engineering project with nodes and edges, with each node representing a module and technical attributes that have been previously selected for the engineering project, with each edge representing a connection between the previously selected modules in the engineering project, calculating a topology-based measure at least for the current center node; assigning the blank node to a bin from a set of bins depending on the topology-based measure that has been computed for the current center; calibrating, by a post-processor, all preliminary confidence scores for the blank node by applying a scaling factor depending on the assigned bin; and outputting, by a user interface, at least the class with the highest calibrated confidence score for the blank node as well as the respective calibrated confidence score.
Marvaniya teaches a graph representing a current state of the engineering project with nodes and edges, with each node representing a module and technical attributes that have been previously selected for the engineering project, with each edge representing a connection between the previously selected modules in the engineering project ([0033] The a graph representing a current state of the engineering project with nodes and edges knowledge graph may include with each node representing a module and technical attributes that have been previously selected for the engineering project entity nodes capturing different scenarios corresponding to at least a portion of the one or more parameters; and with each edge representing a connection between the previously selected modules in the engineering project edges indicating relationships between pairs of the entity nodes, wherein the relationships correspond to at least one of: a predicted change in demand, a model confidence score, and a model uncertainty score. The spatiotemporal query may include a natural language text query, and wherein said analyzing comprises applying one or more natural language processing techniques to identify a context of the spatiotemporal query, the context comprising at least one of: one or more characteristics of the at least one product, a region related to the spatiotemporal query, a particular stage within the supply chain related to the spatiotemporal query.),
Sztyler and Marvaniya are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Sztyler, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Marvaniya to Sztyler before the effective filing date of the claimed invention in order to implement an explainability framework that provides and outputs explanations of demand forecast predictions, useful for understanding the mid-term risks and/or long-term risks at various stages in the supply chain (cf. Marvaniya, [0016] As described herein, an embodiment of the present disclosure includes techniques for auto-discovery of reasoning KGs in supply chains. One or more example embodiments allow a user to specify natural language queries to determine risks in the supply chains. At least one embodiment includes automatically triggering a short conversation to identify a human-computer conversation to identify, for example, the relevant domain ontology, entities, and stages in the supply chains. Further, a reasoning KG may be automatically discovered by interpreting one or more spatiotemporal queries of a user, which may be implemented as part of an explainability framework that provides and outputs explanations of demand forecast predictions. Such explanations are useful for understanding the mid-term risks and/or long-term risks at various stages in the supply chain, as described in more detail herein.).
Wang teaches calculating a topology-based measure at least for the current center node ([3 Confidence Calibration on GCNs, pg. 4] To this end, we find that GCN itself can play the role of calibration function that meets above requirement since GCN is able to propagate node features along the network topology and smooth similar information between neighboured nodes. Therefore, we can employ another l-layer GCN (CaGCN) as our calibration function to propagate the confidence along the network topology. Specifically, given the output V of the classification GCN, the logit v 0 i and calculating a topology-based measure at least for the current center node confidence pˆi for node i after calibration can be obtained by: V 0 = Aσ(···Aσ(AVW(1) )W(2) ···)W(l) = [v 0 1,··· ,v 0 N] T , zi = [σSM(v 0 i,1),··· ,σSM(v 0 i,K)]T , pˆi = max k zi,k , (3) where σSM(v 0 i,·) = exp(v 0 i,·) ∑ K j=1 exp(v 0 i, j) is the softmax operation.);
assigning the blank node to a bin from a set of bins depending on the topology-based measure that has been computed for the current center; calibrating, by a post-processor, all preliminary confidence scores for the blank node by applying a scaling factor depending on the assigned bin; and outputting, by a user interface, at least the class with the highest calibrated confidence score for the blank node as well as the respective calibrated confidence score ([2 Notation and Preliminary Study, pg. 3] Next, we take two representative GNNs (GCN [16] and GAT [33]) as examples to analyze whether they are perfectly calibrated. Specifically, we apply GCN and GAT to four widely used datasets Cora [29], Citeseer [29], Pubmed [29], CoraFull [3], and examine whether their results satisfy Definition 1. To provide more results, we select three label rates for training set (i.e., 20, 40, 60 labeled nodes per class). All the experimental settings follow [16, 33]. Since the true probability p cannot be exactly known, we take an approximate way to evaluate the calibration as in [12]. In particular, we first partition the [0,1] range of confidence into 20 equal bins and then we assigning the blank node to a bin from a set of bins depending on the topology-based measure that has been computed for the current center group the nodes into corresponding bins according to their confidence. After that we calculate the average accuracy of each bin. We expect the average accuracy is equal to the average confidence of each bin, which means the model is approximately perfectly calibrated. For example, if the average confidence of nodes in the bin [0.95, 1.0] is 0.96, and then the classification accuracy in this bin should be 96%.; [4 Self-training with Confidence Calibration, pg. 7] Consequently, we design a self-training model CaGCN-st where calibrating, by a post-processor, all preliminary confidence scores for the blank node by applying a scaling factor depending on the assigned bin confidence is firstly calibrated then employed to outputting, by a user interface, at least the class with the highest calibrated confidence score for the blank node as well as the respective calibrated confidence score generate pseudo labels for unlabeled nodes. Specifically, given an unlabeled dataset DU and a labeled dataset DL which has been divided into three parts Dtrain, Dval and Dtest, we firstly train a classification GCN using Dtrain to get the logit of each node. Then all the logits will be fed into a CaGCN to train and we get a calibrated confidence for each node. It should be noted that instead of Dval, we still employ Dtrain to train our CaGCN. After that, the most confident predictions of DU will be adopted as the pseudo labels according to a threshold th and added to the label set. The Dtrain is enlarged in this way. The process above will be repeated s stages until convergence. Please note that our classification GCN and CaGCN are re-initialized in each stage.; [B.1 Datasets and Environment, pg. 13] We choose the commonly used Cora [29], Citeseer [29], Pubmed [29] and CoraFull [3] for evaluation, where nodes represent papers, edges are the citation relationship between papers, node features are comprised of bag-of-words vector of the papers and labels represent the fields of papers. We choose 500 nodes for validation, 1000 nodes for test and select three label rates for the training set (i.e., 20, 40, 60 labeled nodes per class). The details of these datasets are summarized in Table 5. Our data are public and do not contain personally identifiable information and offensive content. The address of our data is https://docs.dgl.ai/en/latest/api/python/dgl.data.html# node-prediction-datasets and the license is Apache License 2.0.).
Sztyler, Marvaniya, and Wang are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Sztyler and Marvaniya, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Wang to Sztyler before the effective filing date of the claimed invention in order to calibrate GNN to self-training framework, obtaining more trustworthy pseudo labels with the calibrated confidence, further improving performance (cf. Wang, [Abstract] Despite Graph Neural Networks (GNNs) have achieved remarkable accuracy, whether the results are trustworthy is still unexplored. Previous studies suggest that many modern neural networks are over-confident on the predictions, however, surprisingly, we discover that GNNs are primarily in the opposite direction, i.e., GNNs are under-confident. Therefore, the confidence calibration for GNNs is highly desired. In this paper, we propose a novel trustworthy GNN model by designing a topology-aware post-hoc calibration function. Specifically, we first verify that the confidence distribution in a graph has homophily property, and this finding inspires us to design a calibration GNN model (CaGCN) to learn the calibration function. CaGCN is able to obtain a unique transformation from logits of GNNs to the calibrated confidence for each node, meanwhile, such transformation is able to preserve the order between classes, satisfying the accuracypreserving property. Moreover, we apply the calibration GNN to self-training framework, showing that more trustworthy pseudo labels can be obtained with the calibrated confidence and further improve the performance. Extensive experiments demonstrate the effectiveness of our proposed model in terms of both calibration and accuracy.).
Regarding claim 2, Sztyler, as modified by Marvaniya and Wang, teaches The method of claim 1.
Sztyler teaches wherein the outputting operation includes outputting a ranked list of the calibrated confidence scores as well as the respective classes for the blank node ([0053] Further, according to an embodiment, the evaluation module 114 may also compare the KGs across the pairs and compute the respective weight matrix. Based on the differences between the weight matrices and the confidence scores, the evaluation module 114 computes which action is most effective to turn the KG into the State of Interest. The evaluation module 114 may take into account that an action ak might be more effective than another action ah, but action ah might be easier to implement. In this context, it may be provided that the level of complexity of implementing an action is specified by the user 102 through a score, which may be provided together with the list of actions.; [0054] The result, i.e. the output of the evaluation module 114, is a outputting operation includes outputting a ranked list ranked list of actions, the corresponding effect (i.e. how the KG changes), and the outcome (i.e. the State of Interest). By using a Temporal Knowledge Graph (TKG) based concept to model the (time-dependent) entities, relations, and attributes, the computed recommendations are highly reliable.).
Wang teaches wherein the outputting operation includes outputting a ranked list of the calibrated confidence scores as well as the respective classes for the blank node ([A.2 Brier Score (BS), pg. 13] Brier Score (BS) [4] is another commonly used calibration metric, which measures the accuracy of probabilistic predictions. The higher the accuracy of predictions is, the lower BS is. For any given prediction yˆi , BS is the lowest when the prediction probability zi is exactly equal to the true probability that yˆi is correct. Given the one-hot label yi for node i, BS can be represented as follows: BS = 1 N N ∑ i=1 K ∑ k=1 (zi,k −yi,k). (13); [4 Self-training with Confidence Calibration, pg. 7] Consequently, we design a self-training model CaGCN-st where confidence is firstly calibrated then employed to generate calibrated confidence scores as well as the respective classes for the blank node pseudo labels for unlabeled nodes. Specifically, given an unlabeled dataset DU and a labeled dataset DL which has been divided into three parts Dtrain, Dval and Dtest, we firstly train a classification GCN using Dtrain to get the logit of each node. Then all the logits will be fed into a CaGCN to train and we get a calibrated confidence for each node. It should be noted that instead of Dval, we still employ Dtrain to train our CaGCN. After that, the most confident predictions of DU will be adopted as the pseudo labels according to a threshold th and added to the label set. The Dtrain is enlarged in this way. The process above will be repeated s stages until convergence. Please note that our classification GCN and CaGCN are re-initialized in each stage.).
Sztyler, Marvaniya, and Wang are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 3, Sztyler, as modified by Marvaniya and Wang, teaches The method of claim 2.
Wang teaches wherein only calibrated confidence scores above a given threshold are included in the ranked list ([4 Self-training with Confidence Calibration, pg. 7] Consequently, we design a self-training model CaGCN-st where confidence is firstly calibrated then employed to generate pseudo labels for unlabeled nodes. Specifically, given an unlabeled dataset DU and a labeled dataset DL which has been divided into three parts Dtrain, Dval and Dtest, we firstly train a classification GCN using Dtrain to get the logit of each node. Then all the logits will be fed into a CaGCN to train and we get a calibrated confidence for each node. It should be noted that instead of Dval, we still employ Dtrain to train our CaGCN. After that, the most confident predictions of DU will be adopted as the pseudo labels wherein only calibrated confidence scores above a given threshold are included in the ranked list according to a threshold th and added to the label set. The Dtrain is enlarged in this way. The process above will be repeated s stages until convergence. Please note that our classification GCN and CaGCN are re-initialized in each stage.).
Sztyler, Marvaniya, and Wang are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 4, Sztyler, as modified by Marvaniya and Wang, teaches The method of claim 1.
Sztyler teaches with additional operations of recognizing, by the user interface, a user interaction selecting a recommended class or one of the recommended classes; and
automatically completing the engineering project by replacing the blank node with a module of a selected class in the database ([0013] Furthermore, in accordance with an embodiment, the present invention provides a recommender system for providing recommendations to users based on a state of interest, the system comprising an AI unit configured to organize domain of interest information in an initial temporal Knowledge Graph KGt, where t is a timestamp that refers to the present point in time; a prediction component configured to predict, for at least one future point in time t+x, future entities, future links between entities and/or future attributes of entities for the initial Knowledge Graph KGt and to produce at least one new Knowledge Graph KGt+x based on the predictions; a diachronic analyzer configured to simulate situations resulting from the execution of a particular action or a combination of actions at certain points in time and to predict expected temporal Knowledge Graphs for the simulated situations; a graph analyzer configured to classify the Knowledge Graphs produced for the respective points in time and for the simulated situations based on the state of interest; and evaluation module configured to provide, based on the classification result, a ranked list of recommended actions.; [0016] According to embodiments, the system provides a ranked list of those recommended actions and expected outcomes, possibly together with explanations on the actions/outcomes, to a user of the recommender system. According to embodiments, the ranked list of recommended actions may be provided together with their recommended time of execution. Based on the recommended actions (and possibly their recommended time of execution), the with additional operations of recognizing, by the user interface, a user interaction selecting a recommended class or one of the recommended classes user can then instruct the system to take next steps. According to embodiments, it may be provided that the automatically completing the engineering project by replacing the blank node with a module of a selected class in the database system is able to take user feedback into account by updating its decision-making strategy.).
Sztyler, Marvaniya, and Wang are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 7, Sztyler, as modified by Marvaniya and Wang, teaches The method of claim 1.
Wang teaches with initial steps of: using several graphs of completed engineering projects as training data, randomly deleting nodes in the training data, and adjusting trainable parameters of the graph neural network and classifier according to a training objective that maximizes, for blank nodes replacing the deleted nodes, the calibrated confidence scores for the classes of the deleted nodes ([4 Self-training with Confidence Calibration, pg. 7] Consequently, we design a self-training model CaGCN-st where confidence is firstly calibrated then employed to generate pseudo labels for unlabeled nodes. Specifically, given an unlabeled dataset DU and a labeled dataset DL which has been divided into three parts Dtrain, Dval and Dtest, we firstly train a classification GCN using Dtrain to get the logit of each node. Then all the logits will be fed into a CaGCN to train and we get a calibrated confidence for each node. It should be noted that instead of Dval, we still employ Dtrain to train our CaGCN. After that, the most confident predictions of DU will be adopted as the pseudo labels according to a threshold th and added to the label set. The Dtrain is enlarged in this way. The process above will be using several graphs of completed engineering projects as training data repeated s stages until convergence. Please note that our classification GCN and CaGCN are re-initialized in each stage.; [Parameter study., pg. 15] We investigate the effect of the threshold th in CaGCN-st, i.e., the number of unlabeled nodes added to the training set. Generally speaking, with the increase of th, randomly deleting nodes in the training data fewer but more confident nodes will be chosen. Fig. 5 and Fig. 6 show the changing trends of classification accuracy with respect to th, where different colors represent different label rates. Basically, both too high and too low threshold will harm the performance, since a higher value will leave out correct predictions while a lower value will introduce many incorrect predictions to the label set. CaGCN-st obtains the for blank nodes replacing the deleted nodes, the calibrated confidence scores for the classes of the deleted nodes best performance adjusting trainable parameters of the graph neural network and classifier according to a training objective that maximizes when th is in the range [0.8, 0.9].).
Sztyler, Marvaniya, and Wang are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 8, Sztyler, as modified by Marvaniya and Wang, teaches The method of claim 1.
Wang teaches with an initial step of: training the post-processor for temperature scaling for each bin, by determining an individual temperature as the scaling factor for each bin ([1 Introduction, pg. 2] In this paper, we introduce a topology-aware post-hoc calibration method for GNNs. Specifically, for the logits given by the original classification GNNs, we employ another calibration GCN (CaGCN) to propagate confidence, naturally enabling that the confidence of topologically adjacent nodes becomes similar. CaGCN learns a unique training the post-processor for temperature scaling for each bin temperature t for by determining an individual temperature as the scaling factor for each bin each node for temperature scaling, thus preserving the accuracy of the original classification GCN. In addition, based on our finding that large numbers of high-accuracy predictions are distributed in the low-confidence range, we design a calibrated self-training model CaGCN-st in which the confidence is firstly calibrated then used to generate pseudo labels with high confidence.; [4 Self-training with Confidence Calibration, pg. 7] Consequently, we design a self-training model CaGCN-st where confidence is firstly calibrated then employed to generate pseudo labels for unlabeled nodes. Specifically, given an unlabeled dataset DU and a labeled dataset DL which has been divided into three parts Dtrain, Dval and Dtest, we firstly train a classification GCN using Dtrain to get the logit of each node. Then all the logits will be fed into a CaGCN to train and we get a calibrated confidence for each node. It should be noted that instead of Dval, we still employ Dtrain to train our CaGCN. After that, the most confident predictions of DU will be adopted as the pseudo labels according to a threshold th and added to the label set. The Dtrain is enlarged in this way. The process above will be repeated s stages until convergence. Please note that our classification GCN and CaGCN are re-initialized in each stage.).
Sztyler, Marvaniya, and Wang are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 10, Sztyler, as modified by Marvaniya and Wang, teaches A system for recommending modules for an engineering project according to claim 1.
Marvaniya teaches comprising one or more processors, a database, a graph neural network, a classifier, a post-processor, and a user interface, wherein at least some of these components are hardware components, and wherein all of these components are configured for the execution of the respective operations ([0034] The techniques depicted in FIG. 6 can also, as described herein, include providing a system, wherein the system includes distinct software modules, each of the distinct software modules being embodied on a tangible computer-readable recordable storage medium. All of the modules (or any subset thereof) can be on the same medium, or each can be on a different medium, for example. The modules can include any or all of the components shown in the figures and/or described herein. In an embodiment of the invention, the components are hardware components modules can run, for example, on a hardware processor. The method steps can then be carried out using the distinct software modules of the system, as described above, executing on a processors hardware processor. Further, a computer program product can include a tangible computer-readable recordable storage medium with code adapted to be executed to carry out at least one method step described herein, including the provision of the system with the distinct software modules.; [0073] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframes 61; RISC (Reduced Instruction Set Computer) architecture based servers 62; servers 63; blade servers 64; storage devices 65; and networks and networking components 66. In some embodiments, software components include network application server software 67 and database database software 68.; [0028] In at least one example embodiment, a sentence embedding function is applied to the parsed query (e.g., Sent2Vec using Bidirectional Encoder Representations from Transformers (BERT) model), which is then used as input to an intent classifier classification model and a supply chain stage classification model to determine a particular stage of a supply chain associated with the query.; [0024] The KG is in a form that is utilized by a user interface user interface so that a user can analyze and understand the mid-term/long-term risk of such scenarios at various stages in the supply chain, possibly enable intervention.)
Wang teaches comprising one or more processors, a database, a graph neural network, a classifier, a post-processor, and a user interface, wherein at least some of these components are hardware components, and wherein all of these components are configured for the execution of the respective operations ([1 Introduction, pg. 2] In this paper, we introduce a topology-aware post-hoc calibration method for graph neural network GNNs. Specifically, for the logits given by the original classification GNNs, we employ another calibration GCN (CaGCN) to propagate confidence, naturally enabling that the confidence of topologically adjacent nodes becomes similar. CaGCN learns a unique temperature t for each node for temperature scaling, thus preserving the accuracy of the original classification GCN. In addition, based on our finding that large numbers of high-accuracy predictions are distributed in the low-confidence range, we design a calibrated self-training model CaGCN-st in which the confidence is firstly calibrated then used to generate pseudo labels with high confidence.; [Experimental settings., pg. 8] For the base model GCN and GAT, i.e., the uncalibrated model, we follow parameters suggested by [16] and [33] and further carefully tune them to get optimal performance. For the post-processor post-hoc calibration technique, we follow the official implementation [12, 18]. For our CaGCN, we train a two-layer GCN with the hidden layer dimension to be 16. We set λ = 0.5 for all datasets, weight decay to be 5e-3 for Cora, Citeseer, Pubmed and 0.03 for CoraFull. Other parameters of CaGCN follows [16]. We evaluate the performance of confidence calibration by ECE [22], NLL [10] and Brier Score (BS) [4], which we expect are smaller, and we set the bin number M = 20 for ECE (more details can be seen in Appendix A). For all methods, we randomly run 10 times and report the average results. More detailed experimental settings can be seen in Appendix B.).
Sztyler, Marvaniya, and Wang are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 11, Sztyler, as modified by Marvaniya and Wang, teaches A computer program product according to claim 1.
Marvaniya teaches comprising a non-transitory computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor of a computer system to implement a method ([0034] The techniques depicted in FIG. 6 can also, as described herein, include providing a system, wherein the system includes distinct software modules, each of the distinct software modules being embodied on a tangible computer-readable recordable storage medium. All of the modules (or any subset thereof) can be on the same medium, or each can be on a different medium, for example. The modules can include any or all of the components shown in the figures and/or described herein. In an embodiment of the invention, the modules can run, for example, on a hardware processor. The method steps can then be carried out using the distinct software modules of the system, as described above, executing on a hardware processor. Further, a non-transitory computer readable hardware storage device having computer readable program code stored therein computer program product can include a tangible computer-readable recordable storage medium with code adapted to be program code executable by a processor of a computer system to implement a method executed to carry out at least one method step described herein, including the provision of the system with the distinct software modules.).
Sztyler, Marvaniya, and Wang are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 12, Sztyler, as modified by Marvaniya and Wang, teaches A provision device for the computer program product according to claim 11.
Marvaniya teaches wherein the provision device stores and/or provides the computer program product ([0054] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable provision device for the computer program product computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.)
Sztyler, Marvaniya, and Wang are combinable for the same rationale as set forth above with respect to claim 1.
Claim 5, 9 is rejected under 35 U.S.C. 103 as being unpatentable over Sztyler, in view of Marvaniya, Wang, and further in view of Lopera et al. (NPL: "A Survey of Graph Neural Networks for Electronic Design Automation", hereinafter 'Lopera').
Regarding claim 5, Sztyler, as modified by Marvaniya and Wang, teaches The method of claim 4.
Sztyler, as modified by Marvaniya and Wang, fails to teach with additional step of automatically producing the engineering project: by printing the engineering project with a 3D printer, or by automatically configuring autonomous machines by configuring software modules of the autonomous machines, in order to produce the engineering project, or by automatically assigning autonomous machines to perform as the modules of the engineering project by installing and/or activating and/or configuring software modules of the autonomous machines.
Lopera teaches with additional step of automatically producing the engineering project:
by printing the engineering project with a 3D printer, or by automatically configuring autonomous machines by configuring software modules of the autonomous machines, in order to produce the engineering project, or by automatically assigning autonomous machines to perform as the modules of the engineering project by installing and/or activating and/or configuring software modules of the autonomous machines ([A. Electronic Design Automation] The progress in EDA tools and design methods, and the use of different levels of abstractions on the design flow have improved the hardware design productivity. Figure 1 sketches the stages of a modern chip design process. The flow starts with the chip specification modeling the desired application. The architecture analysis and prototype of the design represent the design as a additional step of automatically producing the engineering project: by automatically assigning autonomous machines to perform as the modules of the engineering project collection of interacting modules such as processors, memories, and buses. In the functional and logical design, the behavioral description of those modules is mapped to Register Transfer Level (RTL) blocks using Hardware Description Languagess (HDLs) such as Verilog. Nowadays, the transition from system specification to RTL can be done in different ways. For instance, using installing and/or activating and/or configuring software modules of the autonomous machines High-level Synthesis (HLS), which provides an automatic conversion from C/C++/System-C specifications to HDL, or using hardware design frameworks such as MetaRTL [5].; [D. Graph Neural Networks] The graph embeddings learned by GNNs can be used as inputs to other ML models building an end-to-end framework depicted in Figure 2. There are three levels of tasks for such a framework: Node, edge, and graph [12]. In the node-level tasks, all nodes are labeled so that a regression or classification of the nodes is possible. In edge-level tasks, the goal is to classify edges or predict the link between two nodes. Finally, in graph-level tasks, the entire graph is labeled and a NN, combined with pooling and readout operations, can classify new unseen graphs. Moreover, the learning tasks of GNNs can be transductive or inductive. In the former, the GNN learns the embedding vectors per each node in the training graphs. Thus, during inference, it cannot generalize to new nodes. On the contrary, an inductive GNN learns the aggregation function that combines the node’s neighborhood features to get the embedding vectors [12].).
Sztyler, Marvaniya, Wang, and Lopera are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Sztyler, Marvaniya, and Wang, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Lopera to Sztyler before the effective filing date of the claimed invention in order to review existing works linking the EDA flow for chip design and graph neural networks (cf. Lopera, [Abstract] Driven by Moore’s law, the chip design complexity is steadily increasing. Electronic Design Automation (EDA) has been able to cope with the challenging very large-scale integration process, assuring scalability, reliability, and proper time to-market. However, EDA approaches are time and resource demanding, and they often do not guarantee optimal solutions. To alleviate these, Machine Learning (ML) has been incorporated into many stages of the design flow, such as in placement and routing. Many solutions employ Euclidean data and ML techniques without considering that many EDA objects are represented naturally as graphs. The trending Graph Neural Networks are an opportunity to solve EDA problems directly using graph structures for circuits, intermediate RTLs, and netlists. In this paper, we present a comprehensive review of the existing works linking the EDA flow for chip design and Graph Neural Networks.).
Regarding claim 9, Sztyler, as modified by Marvaniya and Wang, teaches The method of claim 1.
Sztyler, as modified by Marvaniya and Wang, fails to teach wherein the engineering project is an industrial automation system.
Lopera teaches wherein the engineering project is an industrial automation system ([A. Electronic Design Automation] The progress in EDA tools and design methods, and the use of different levels of abstractions on the design flow have improved the hardware design productivity. wherein the engineering project is an industrial automation system Figure 1 sketches the stages of a modern chip design process. The flow starts with the chip specification modeling the desired application. The architecture analysis and prototype of the design represent the design as a collection of interacting modules such as processors, memories, and buses. In the functional and logical design, the behavioral description of those modules is mapped to Register Transfer Level (RTL) blocks using Hardware Description Languagess (HDLs) such as Verilog. Nowadays, the transition from system specification to RTL can be done in different ways. For instance, using High-level Synthesis (HLS), which provides an automatic conversion from C/C++/System-C specifications to HDL, or using hardware design frameworks such as MetaRTL [5].; [D. Graph Neural Networks] The graph embeddings learned by GNNs can be used as inputs to other ML models building an end-to-end framework depicted in Figure 2. There are three levels of tasks for such a framework: Node, edge, and graph [12]. In the node-level tasks, all nodes are labeled so that a regression or classification of the nodes is possible. In edge-level tasks, the goal is to classify edges or predict the link between two nodes. Finally, in graph-level tasks, the entire graph is labeled and a NN, combined with pooling and readout operations, can classify new unseen graphs. Moreover, the learning tasks of GNNs can be transductive or inductive. In the former, the GNN learns the embedding vectors per each node in the training graphs. Thus, during inference, it cannot generalize to new nodes. On the contrary, an inductive GNN learns the aggregation function that combines the node’s neighborhood features to get the embedding vectors [12].).
Sztyler, Marvaniya, Wang, and Lopera are combinable for the same rationale as set forth above with respect to claim 5.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Sztyler, in view of Marvaniya, Wang, and further in view of He et al. (NPL: "Block Modeling-Guided Graph Convolutional Neural Networks", hereinafter 'He').
Regarding claim 6, Sztyler, as modified by Marvaniya and Wang, teaches The method of claim 1.
Sztyler, as modified by Marvaniya and Wang, fails to teach wherein the topology-based measure is a same-class-neighbor ratio, which is computed as a proportion of the neighbors of the current center node for which the class with the highest determined preliminary confidence score is identical to a class with the highest determined preliminary confidence score for the current center node.
He teaches wherein the topology-based measure is a same-class-neighbor ratio, which is computed as a proportion of the neighbors of the current center node for which the class with the highest determined preliminary confidence score is identical to a class with the highest determined preliminary confidence score for the current center node ([Overview] In specific, we use the block matrix to define a block similarity matrix which measures the similarity degree of any two blocks based on connecting patterns among blocks, and contains homophilic and heterophilic information among blocks. Block similarity matrix provides potentially valuable rules for information propagation in graph convolutional layers, which can ignore the homophily or heterophily of a graph and finally achieve a classified aggregation. Then, we use this block similarity matrix and the learned soft labels from MLP to redefine a new aggregation operation in which class labels of nodes and the similarities among blocks can co-guide the attribute information propagating and aggregating on network topology.; [Homophily Ratio.] The homophily ratio (Pei et al. 2020) can measure the overall homophily level in a graph. It counts the wherein the topology-based measure is a same-class-neighbor ratio ratio of same-class neighbor nodes to the total neighbor nodes in a graph, which is computed as a proportion of the neighbors of the current center node for which the class with the highest determined preliminary confidence score is identical to a class with the highest determined preliminary confidence score for the current center node defined as h = 1 |V| X vi∈V |{vj |vj ∈ Ni , Yj = Yi}| |Ni | (1) where Ni is the neighbor set of node vi . In this work, we use homophily ratio h to determine whether a graph is homophilic or heterophilic.; [Block Matrix.] Given the labels Y ∈ R n×c for all nodes and the adjacency matrix A ∈ {0, 1} n×n, the block matrix is defined as H = Y T AY Y T AE (2) where E an all-ones matrix with the same size as Y , and the Hadamard (element-wise) division operation. Block matrix models the linked possibility of nodes in any two blocks. In this work, blocks represent the classes of labels in a graph. From the node-wise level, Hi,j is the probability that a node in the i-th class connects with a node in the j-th class. The notations are summarized in Table 1.).
Sztyler, Marvaniya, Wang, and He are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Sztyler, Marvaniya, and Wang, it would have been obvious for a person of ordinary skill in the art to apply the teachings of He to Sztyler before the effective filing date of the claimed invention in order to make the propagation and aggregation mechanism of GCN suitable for both homophily and heterophily, automatically learn the corresponding aggregation rules for neighbors of different classes (cf. He, [Abstract] Graph Convolutional Network (GCN) has shown remarkable potential of exploring graph representation. However, the GCN aggregating mechanism fails to generalize to networks with heterophily where most nodes have neighbors from different classes, which commonly exists in real-world networks. In order to make the propagation and aggregation mechanism of GCN suitable for both homophily and heterophily (or even their mixture), we introduce block modeling into the framework of GCN so that it can realize “blockguided classified aggregation”, and automatically learn the corresponding aggregation rules for neighbors of different classes. By incorporating block modeling into the aggregation process, GCN is able to aggregate information from homophilic and heterophilic neighbors discriminately according to their homophily degree. We compared our algorithm with state-of-art methods which deal with the heterophily problem. Empirical results demonstrate the superiority of our new approach over existing methods in heterophilic datasets while maintaining a competitive performance in homophilic datasets.).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAGGIE MAIDO whose telephone number is (703) 756-1953. The examiner can normally be reached M-Th: 6am - 4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MM/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights