DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The amendments were received on 7/23/2026. Claims 1, 3-11, and 13-20 are pending where claims 1, 3-11, and 13-20 were previously presented and claims 2 and 12 were cancelled.
35 USC § 112
The applicant amended claim 11 claim to address the 35 USC 112 rejections. In view of the amendments, the respective 35 USC 112 rejection of claim 17 has been withdrawn.
35 USC § 101
The applicant amended the independent claims to incorporate limitations from a claim that was not rejected under 35 USC 101. Therefore, for at least the reasons discussed in the previous Office Action, the claims overcome the 35 USC 101 rejection and the respective rejection has been withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3, 5, 6, 11, 13, 15, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal et al [US 2020/0074246 A1] in view of Dirac et al [US 2015/0379072 A1], Creed et al [US 2021/0081717 A1], and Desai et al [US 2019/0325350 A1].
With regard to claim 1, Goyal teaches a method of operating a computer system to process a continuous-time dynamic graph (CTDG) to perform a specific prediction task, the CTDG comprising a data structure that represents nodes and edges having temporal properties, the edges representing relationships between the nodes (see paragraphs [0025] and [0026]; the system can operate on dynamic graphs that evolve over time with the graph/data structure having nodes and edges that represent relationships between the nodes),
the method comprising: extracting a time window of data from the CTDG to obtain a history graph that represents a sub-set of the CTDG (see paragraphs [0027], [0035], [0059], and [0042]; the system can utilize a time window or lookback parameter to extract history graph data for the various time steps);
generating, using an encoder model configured by a set of learned encoder parameters and implemented by the computer system, a set of embeddings for the history graph (see Figure 3B and paragraphs [0031] and [0048]; the system can utilize a trained encoder to create embeddings of the graphs);
and predicting, using a first decoder model configured by a set of learned first decoder parameters and implemented by the computer system, one or more predictions for the CTDG corresponding to the specific prediction task (see paragraphs [0040] and [005]); the trained decoder model uses the embeddings to make predictions including link prediction of what the temporal/dynamic graph would look like in the future).
Goyal does not appear to explicitly teach:
the method further comprising:
configuring the computer system to perform the specific prediction task, wherein configuring the computer system comprises:
pre-training the encoder model using self-supervised learning to perform a generalized prediction task by:
partitioning the CTDG into a first batch of first disjoint graphs that each correspond to a respective time window of a first defined size;
performing a random transformation on each of the first disjoint graphs to generate, for each first disjoint graph, a respective pair of transformed graphs;
and iteratively, until a pre-training criteria is reached: (i) generating, using the encoder model, respective embeddings for each transformed graph in each pair of transformed graphs; (ii) generating, for each of the respective embeddings, a respective prediction, using a second decoder model that is configured by a set of second decoder parameters; and (iii) updating the encoder parameters and the second decoder parameters based on the respective predictions;
and training the encoder model and the first decoder model to collectively perform the specific prediction task by:
partitioning the CTDG into a second batch of second disjoint graphs that each correspond to a respective time window of a second defined size;
and iteratively, until a training criteria is reached: (i) generating, using the encoder model, respective embeddings for each of the second disjoint graphs; (ii) generating, for each of the respective embeddings of the second disjoint graphs, a respective task specific prediction, using the first decoder model; and (iii) updating at the first decoder parameters based on a comparison of the respective task prediction to actual data included in the CTDG.
Dirac teaches pre-training the has means to have pre-trained models for general tasks that are usable by other users where the training can be via a supervised technique).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify the prediction system of Goyal by including means of having a model service as taught by Dirac in order to allow for experts/users to share and re-use their models so that the usage of the models can be used by a wider audience as a service while providing means to earn money for the service by billing clients to use the respective models and the server resources.
Goyal in view of Dirac teach the method further comprising: configuring the computer system to perform the specific prediction task, comprising: pre-training the encoder model using self-supervised learning to perform a generalized prediction task by (see Dirac, paragraphs [0047], [0079], and [0169]; see Goyal, paragraphs [0031] and [0059]; the system can utilize training methods include supervised training to create a model that can be shared with others):
partitioning the CTDG into a first batch of first disjoint graphs that each correspond to a respective time window of a first defined size (see Goyal, paragraphs [0059] and [0071]; the system can partition the graph into time windows of a first size which can be used for training);
and iteratively, until a pre-training criteria is reached: (i) generating, using the encoder model, respective embeddings for each parameters based on the respective predictions (See Goyal, paragraphs [0035] and [0046]; the system can train the respective encoder and decoder based on criteria including optimizing a loss function).
Goyal in view of Dirac do not appear to explicitly teach:
performing a random transformation on each of the first disjoint graphs to generate, for each first disjoint graph, a respective pair of transformed graphs;
and iteratively, until a pre-training criteria is reached: (i) generating, using the encoder model, respective embeddings for each transformed graph in each pair of transformed graphs; (ii) generating, for each of the respective embeddings, a respective prediction, using a second decoder model that is configured by a set of second decoder parameters; and (iii) updating the encoder parameters and the second decoder parameters based on the respective predictions
and training the encoder model and the first decoder model to collectively perform the specific prediction task by:
partitioning the CTDG into a second batch of second disjoint graphs that each correspond to a respective time window of a second defined size;
and iteratively, until a training criteria is reached: (i) generating, using the encoder model, respective embeddings for each of the second disjoint graphs;(ii) generating, for each of the respective embeddings of the second disjoint graphs, a respective task specific prediction, using the first decoder model; and (iii) updating at the first decoder parameters based on a comparison of the respective task prediction to actual data included in the CTDG.
Creed teaches performing a random transformation on each of the first disjoint graphs to generate, for each first disjoint graph, a respective pair of transformed graphs (see paragraphs [0151] and [0129]; the system can include means of performing random transformations to the graph can be performed and utilized to form various training datasets).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify the prediction system of Goyal in view of Dirac by including means of having multiple different datasets including transformed graphs of a dataset having incomplete data or noise data as taught by Creed in order to help train a more robust model that can have reduced susceptibility to noise in making wrong predictions while also being able to still accurately make predictions even with incomplete (i.e. less) graph data.
Goyal in view of Dirac and Creed teach: iteratively, until a pre-training criteria is reached: (i) generating, using the encoder model, respective embeddings for each transformed graph in each pair of transformed graphs; (ii) generating, for each of the respective embeddings, a respective prediction, using a second decoder model that is configured by a set of second decoder parameters; and (iii) updating the encoder parameters and the second decoder parameters based on the respective predictions (See Goyal, paragraphs [0035] and [0046]; the system can train the respective encoder and decoder based on criteria including optimizing a loss function).
Goyal in view of Dirac and Creed do not appear to explicitly teach:
training the encoder model and the first decoder model to collectively perform the specific prediction task by:
partitioning the CTDG into a second batch of second disjoint graphs that each correspond to a respective time window of a second defined size;
and iteratively, until a training criteria is reached: (i) generating, using the encoder model, respective embeddings for each of the second disjoint graphs;(ii) generating, for each of the respective embeddings of the second disjoint graphs, a respective task specific prediction, using the first decoder model; and (iii) updating at the first decoder parameters based on a comparison of the respective task prediction to actual data included in the CTDG.
Desai teaches training the
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify the prediction system of Goyal in view of Dirac and Creed by including means of being able to customize/personalize a generic/general model as taught by Desai in order to save users time and effort by allowing client users to be able to utilize already established models without having the clients users develop their own proprietary model from scratch while still being able to customize/personalize the model with the dataset(s) that the client user desires instead of generic/public datasets thus helping the model(s) to be fine-tuned and more accurate when dealing/evaluating the client users’ respective data.
Goyal in view of Dirac, Creed, and Desai teach training the encoder model and the first decoder model to collectively perform the specific prediction task by: partitioning the CTDG into a second batch of second disjoint graphs that each correspond to a respective time window of a second defined size (see Goyal, paragraphs [0059] and [0071]; the system can partition the graph into time windows of a determined size which can be used for training);
and iteratively, until a training criteria is reached: (i) generating, using the encoder model, respective embeddings for each of the second disjoint graphs;(ii) generating, for each of the respective embeddings of the second disjoint graphs, a respective task specific prediction, using the first decoder model; and (iii) updating at the first decoder parameters based on a comparison of the respective task prediction to actual data included in the CTDG (see Desai, paragraph [0035]; See Goyal, paragraphs [0035] and [0046]; the system can train the respective encoder and decoder based on criteria including optimizing a loss function).
With regard to claim 3, Goyal in view of Dirac, Creed, and Desai teach wherein, during the pre-training, updating the encoder parameters and the second decoder parameters based on the respective predictions comprises comparing, for each pair of transformed graphs, the respective predictions made therefore (see Creed, paragraphs [0151] and [0129] and [0106]-[0107]; see Dirac, paragraphs [0047] and [0169]; see Goyal, paragraphs [0031] and [0059]; the system can utilize training methods include supervised training to create a model that can be shared with others based on the respective training datasets used to train the model including updating the model based on the respective loss function).
With regard to claim 5, Goyal in view of Dirac, Creed, and Desai teach wherein the first defined size and the second defined size are hyperparameters (see Goyal, paragraphs [0027] and [0059]; the defined size of the time window is a model parameter).
With regard to claim 6, Goyal in view of Dirac, Creed, and Desai teach wherein performing the random transformation on each of the first disjoint graphs comprises randomly performing edge dropouts and edge feature masking (see Creed, paragraphs [0129] and [0171]; dropout transformation can be used).
With regard to claims 11 and 20, these claims are substantially similar to claim 1 and are rejected for similar reasons as discussed above.
With regard to claims 13, 15, and 16, these claims are substantially similar to claims 3, 5, and 6 respectively and are rejected for similar reasons as discussed above.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal et al [US 2020/0074246 A1] in view of Dirac et al [US 2015/0379072 A1], Creed et al [US 2021/0081717 A1], and Desai et al [US 2019/0325350 A1] in further view of Meyerson et al [US 2019/0244108 A1].
With regard to claim 4, Goyal in view of Dirac, Creed, and Desai teach all the claim limitations of claims 1 and 2 as discussed above.
Goyal in view of Dirac, Creed, and Desai do not appear to explicitly teach:
wherein during training the encoder model and the first decoder model to collectively perform the specific prediction task, the encoder parameters are frozen and only the first decoder parameters are updated.
Meyerson teaches wherein during training the encoder model and the first decoder model to collectively perform the specific prediction task, the encoder parameters are frozen and only the first decoder parameters are updated (see paragraphs [0060]-[0061], [0052], [0023], and [0121]; the system can utilize multiple decoders which can be trained for different tasks while using the same encoder where the training can involve freezing other models).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify the prediction system of Goyal in view of Dirac, Creed, and Desai by being able to freeze various models during training as taught by Meyerson in order to prevent all the models from being adjusted to the same task when multiple different tasks are desired by the system with means to allow the same input methodology to be used but having different task prediction models without having to form a single ginormous model to handle all classification tasks thus helping to keep smaller, robust, and fine-tuned models on specific task features.
With regard to claim 14, this claim is substantially similar to claim 4 and is rejected for similar reason as discussed above.
Claims 7, 8, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal et al [US 2020/0074246 A1] in view of Dirac et al [US 2015/0379072 A1], Creed et al [US 2021/0081717 A1], and Desai et al [US 2019/0325350 A1] in further view of Wu et al [US 2022/0027707 A1].
With regard to claim 7, Goyal in view of Dirac, Creed, and Desai teach all the claim limitations of claims 1 and 2 as discussed above.
Goyal in view of Dirac, Creed, and Desai teach a variety of neural networks but do not appear to explicitly teach:
wherein the encoder model is an attention-based Message-Passing (AMP) neural network.
Wu teaches wherein the encoder model is an attention-based Message-Passing (AMP) neural network (see paragraph [0036]; the encoder is a neural network that can perform message passing).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify the prediction system of Goyal in view of Dirac, Creed, and Desai by being able to substitute one neural network based encoder for a message-passing neural network based encoder various models during training as taught by Wu in order to be able to exchange and use widely-known and utilized encoder neural network methodologies/algorithms/models so that the system can be designed according to the preferred model that the client users prefer using.
With regard to claim 8, Goyal in view of Dirac, Creed, and Desai wherein the first decoder model and second decoder model comprise respective multi-layer perception neural networks (see Goyal, paragraph [0044]-[0045]; the decoders can comprise multiple layers).
With regard to claim 17, this claim is substantially similar to claim 7 and is rejected for similar reason as discussed above.
Claims 9, 10, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal et al [US 2020/0074246 A1] in view of Chen et al [US 2021/0256355 A1].
With regard to claim 9, Goyal teaches all the claim limitations of claim 1 as discussed above.
Goyal teaches link/edge prediction but does not appear to explicitly teach:
wherein the specific prediction task is predicting a probability of an edge between two nodes of the CTDG at a future time, the method comprising outputting the prediction.
Chen teaches wherein the specific prediction task is predicting a probability of an edge between two nodes of the CTDG at a future time, the method comprising outputting the prediction (see paragraphs [0062]-[0063]; the system can do an edge prediction task that outputs a prediction/probability).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify the prediction system of Goyal by including means for the edge/link prediction to output a probability as taught by Chen in order to expand the functionality of the system to incorporate different related tasks that relate to link/edge prediction but can have different output such as providing an output of a single probability of a link existing at a future time, thus making an easier to understand output format for the users of the system.
With regard to claim 10, Goyal teaches all the claim limitations of claim 1 as discussed above.
Goyal teaches link/edge prediction but does not appear to explicitly teach:
wherein the specific prediction task is predicting node classifications for one or more nodes of the CTDG at a future time, the method comprising outputting the predicted node classifications.
Chen teaches wherein the specific prediction task is predicting node classifications for one or more nodes of the CTDG at a future time, the method comprising outputting the predicted node classifications (see paragraph [0061]; the system can make a prediction of classification of nodes).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify the prediction system of Goyal by including means for the additional tasks such as node classifications as taught by Chen in order to expand the functionality of the system to be able to determine or predict whether a node belongs to a particular class/classification thus helping to not only make a prediction about future time steps but also to provide an analysis/classification of the data to aide users of the system in their understanding of what the future time step data represents.
With regard to claims 18 and 19, these claims are substantially similar to claims 8 and 9 and are rejected for similar reason as discussed above.
Response to Arguments
Applicant’s arguments (see the fourth from last paragraph on page 9) with respect to the 35 USC 112 rejections have been fully considered and are persuasive. The 35 USC 112 rejections of the claims have been withdrawn. The applicant amended claim 11 claim to address the 35 USC 112 rejections. In view of the amendments, the respective 35 USC 112 rejection of claim 17 has been withdrawn.
Applicant’s arguments (see the third from last paragraph on page 9 through the second paragraph on page 10) with respect to the 35 USC 101 rejections have been fully considered and are persuasive. The 35 USC 101 rejections of the claims haves been withdrawn. The applicant amended the independent claims to incorporate limitations from a claim that was not rejected under 35 USC 101. Therefore, for at least the reasons discussed in the previous Office Action, the claims overcome the 35 USC 101 rejection and the respective rejection has been withdrawn.
Applicant's arguments (see the third paragraph on page 10 through the last paragraph on page 20) have been fully considered but they are not persuasive. The applicant argues that the cited prior art do not teach all the claim limitations including four features that are further discussed below. The Examiner respectfully disagrees.
With regard to the arguments regarding feature (1) (see last two paragraphs on page 11 through second paragraph on page 14), the applicant argues that Dirac do not teach anything related to having “pre-trained models” and that training can be done via supervised learning technique. The Examiner notes that the claim limitation is taught by a combination of references and not Dirac alone. Thus, in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Additionally, as shown in Goyal reference, the system can use a machine learning model that is trainable in an external cloud environment (i.e. remote system) where Dirac illustrates that the remote/external machine learning service can have models created/generated (i.e. pre-trained) for use by others (see paragraph 47) with paragraph 169 discussing the learning techniques. Therefore, as can be seen, the references teach the claim limitation as recited.
With regard to the arguments regarding feature (2) (see third paragraph on page 14 through fourth paragraph on page 16), the applicant argues that the citations of the Goyal reference do not teach training until a pre-trained criteria is reached via generating embeddings, generating predictions, and updating encoder/decoder parameters based on the predictions including “nowhere does Goyal disclose anything related to “… the system can train the respective encoder and decoder based on criteria including optimizing a loss function”. As shown in the 35 USC 103 rejections, the Goyal reference discloses the usage of a loss function to be optimized where as discussed in paragraph 46, the decoding process is achieved by ‘optimizing a loss function’ (i.e. training and evaluation of the result/prediction) where the system utilizes the loss function as means to be able to apply gradients (i.e. update the respective models’ parameters). As for the criteria argument, although different terms are used, the Examiner notes that the requirements for patentability is not an ipsissimis verbis test, i.e., identify of terminology is not required. As shown above in the 35 USC 103 rejections, the prior art reference(s) perform(s) the same functionality of the claim limitations in that iteratively until pre-training criteria is reached is broad and can relate to a number of iterations as discussed in Goyal in paragraph [0046] which discusses the example algorithm in Figure 4. Therefore, applicant’s arguments are not persuasive.
With regard to the arguments regarding feature (3) (see last paragraph on page 16 through first two paragraphs on page 17), the applicant argues that the Office Action does not provide grounds regarding how the training encoder model and first decoder model are disclosed and thus Examiner’s rejection is defective as “no discussion in connection of the feature…is made”. The Examiner respectfully disagrees. Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. As shown in the 35 USC 103 rejections, different paragraph taught the limitation, e.g. paragraph 38 which is based on the combination of references. Therefore, applicant’s arguments are not persuasive.
With regard to the arguments regarding feature (4) (see third paragraph on page 17 through top of page 19), the applicant argues that respective rejection fails to discuss and disclose the until training criteria is reached limitation and updating the decoder parameters based on a comparison of the task prediction to actual data in the CTDG. The Examiner respectfully disagrees. Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. As illustrated in the 35 USC 103 rejections, the Goyal reference discloses in paragraph [0046] the usage of the decoder to form output/predictions for its task as part of the training with the respective model being trained/updated accordingly so that the system can optimize the respective loss function. As for the criteria argument, although different terms are used, the Examiner notes that the requirements for patentability is not an ipsissimis verbis test, i.e., identify of terminology is not required. As shown above in the 35 USC 103 rejections, the prior art reference(s) perform(s) the same functionality of the claim limitations in that iteratively until pre-training criteria is reached is broad and can relate to a number of iterations as discussed in Goyal in paragraph [0046] which discusses the example algorithm in Figure 4. Therefore, applicant’s arguments are not persuasive.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARC S SOMERS whose telephone number is (571)270-3567. The examiner can normally be reached M-F 11-8 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann Lo can be reached at 5712729767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARC S SOMERS/Primary Examiner, Art Unit 2159 9/16/2026
Prosecution Insights