Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on August 1, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
Paragraph 0050 begins with a “.” that shouldn’t be there.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1, 11, 20:
Step 1: Recites an apparatus (Claim 1), a process (Claim 11) and a machine (Claim 20). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite:
access, at the processor, an input graph including trajectory observations for a plurality of agents over a plurality of previous timesteps of a plurality of timesteps
This limitation encompasses the mental process of accessing data.
generate, at the processor and by application of the input graph as input to an encoder, a spatiotemporal embedding for the plurality of agents for a current timestep of the plurality of timesteps
This limitation encompasses the mental process of classification.
generate, at the processor and by application of the spatiotemporal embedding as input to a decoder, a trajectory prediction for the plurality of agents for one or more future timesteps of the plurality of timesteps
This limitation encompasses the mental process of making prediction of where an object will go.
decoder including a Bayesian Neural Network operable for inferring an uncertainty of the trajectory prediction for the plurality of agents for the one or more future timesteps
This limitation recites inferring, which encompasses a mental process.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. There are no other limitations in the claim besides those that recite judicial exceptions.
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 2, 12:
Step 1: Recites an apparatus (Claim 2), a process (Claim 12). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite the abstract limitation they inherit from the claims they depend on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as additional components of the abstract idea (“access…a training set…”, “infer… a posterior probability distribution”), or simply applying the judicial exception (“sample…parameter values…”).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 3, 13:
Step 1: Recites an apparatus (Claim 3), a process (Claim 13). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite the abstract limitation they inherit from the claims they depend on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as additional components of the abstract idea (“…including application of a variational free technique…”).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 4
Step 1: Recites an apparatus (Claim 4)
Step 2A Prong 1: The claim recites the abstract limitation they inherit from the claims they depend.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as additional components of the abstract idea (“…including an output layer, the output layer being deterministic”).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 5, 14:
Step 1: Recites an apparatus (Claim 5), a process (Claim 14). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite the abstract limitation they inherit from the claims they depend on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as additional components of the abstract idea (“construct the input graph…”, “…incorporating haversine distance…”, “…first agent being a first aircraft…”).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 6, 15:
Step 1: Recites an apparatus (Claim 6), a process (Claim 15). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite the abstract limitation they inherit from the claims they depend on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as additional components of the abstract idea (“generate a preliminary spatiotemporal embedding…”, “generate the spatiotemporal embedding …”), and read in light of the specification, don’t represent an improvement to the technology.
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 7, 16:
Step 1: Recites an apparatus (Claim 7), a process (Claim 16). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite the abstract limitation they inherit from the claims they depend on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as additional components of the abstract idea (“generate by a first temporal transformer… a first updated temporal embedding”, “generate by a first spatial transformer… a first updated spatial embedding…”, “generate the preliminary spatiotemporal embedding…”), and read in light of the specification, don’t represent an improvement to the technology.
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 8, 17:
Step 1: Recites an apparatus (Claim 8), a process (Claim 17). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite the abstract limitation they inherit from the claims they depend on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as additional components of the abstract idea (“generate…a second updated temporal embedding…”, “generate…a second updated spatiotemporal embedding…”), and read in light of the specification, don’t represent an improvement to the technology.
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 9, 18:
Step 1: Recites an apparatus (Claim 9), a process (Claim 18). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite the abstract limitation they inherit from the claims they depend on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as generally linking the judicial exception (“apply the second updated temporal embedding…”), and read in light of the specification, don’t represent an improvement to the technology.
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim 10, 19:
Step 1: Recites an apparatus (Claim 10), a process (Claim 19). Therefore, they are directed to the statutory category of inventions.
Step 2A Prong 1: The claims recite the abstract limitation they inherit from the claims they depend on.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. The additional limitations of the claim are understood as applying the judicial exception (“generate a graphical user interface…”).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis mirrors what written in Step 2A Prong 2.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6-7, 11. 15-16, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yu et al. (“Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction”, 2020) in view of Casas et al. (US 20210276595).
Regarding claim 1, Yu discloses a processor in communication with a memory, the memory including instructions executable by the processor to:
access, at the processor, an input graph including trajectory observations1 (Figure 5a) for a plurality of agents (Page 13, Figure 5b-5d2) over a plurality of previous timesteps of a plurality of timesteps3 (Figure 4, Figure 5a);
generate, at the processor and by application of the input graph as input to an encoder (Figure 4), a spatiotemporal embedding for the plurality of agents for a current timestep of the plurality of timesteps (3.5, Page 9, “spatial and temporal features are then merged by a fully connected layer, which gives a set of new features with spatio-temporal encodings”);
generate, at the processor and by application of the spatiotemporal embedding as input to a decoder, a trajectory prediction for the plurality of agents for one or more future timesteps of the plurality of timesteps (3.5, Page 9, “The prediction is added to the history for the next step prediction”)
Yu fails to disclose the further limitations of the claim. However, Casas discloses a decoder including a Bayesian Neural Network (Paragraph 220, 227) operable for inferring an uncertainty (Paragraph 24) of the trajectory prediction for the plurality of agents (Paragraph 7) for the one or more future timesteps of the plurality of timesteps (Paragraph 24).
Yu and Casas are both considered analogous to the invention because all are in the field of trajectory prediction. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Yu to incorporate the teachings of Casas, and include a Bayesian Neural Network within the decoder. Doing so provides efficient parallel sampling, high expressivity, and more consistent trajectory samples (see paragraph 24 of Casas).
Regarding claim 6, the rejection of claim 1 is incorporated. Yu discloses the memory further including instructions executable by the processor to:
generate a preliminary spatiotemporal embedding for the input graph at a parallel stage of the encoder and for a current timestep of the plurality of timesteps (Figure 4);
generate, by a first temporal transformer (Figure 3a) of the parallel stage of the encoder, a first updated temporal embedding for the input graph (Figure 4, Section 3.3).
Regarding claim 7, the rejection of claim 6 is incorporated. Yu discloses the memory further including instructions executable by the processor to:
generate, by a first spatial transformer (Figure 3b) of the parallel stage of the encoder, a first updated spatial embedding for the input graph (Figure 4, Section 3.4);
generate the preliminary spatiotemporal embedding by combination of the first updated temporal embedding and the first updated spatial embedding at a multilayer perceptron of the encoder (Figure 4).
Claim 11 is a method claim corresponding to system claim 1 and is rejected for the same reasons as given in the rejection of that claim.
Claim 15 is a method claim corresponding to system claim 6 and is rejected for the same reasons as given in the rejection of that claim.
Claim 16 is a method claim corresponding to system claim 7 and is rejected for the same reasons as given in the rejection of that claim.
Claim 20 is a non-transitory computer readable medium claim corresponding to system claim 1 and is rejected for the same reasons as given in the rejection of that claim.
Claims 2-3, 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Yu et al. (“Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction”, 2020) in view of Casas et al. (US 20210276595) and in further view of Dalgaty et al. (US 20210350218).
Regarding claim 2, the rejection of claim 1 is incorporated. Yu and Casas fail to disclose the further limitations of the claim. However, Dalgaty discloses the Bayesian Neural Network (Paragraph 1) being trained by a processor in communication with a memory including instructions executable by the processor to:
access, at the processor, a training set including an input sequence and an output sequence that corresponds with the input sequence (Paragraph 32);
infer, at the processor, a posterior probability distribution of a set of parameters of the Bayesian Neural Network based on the input sequence and the output sequence of the training set (Paragraph 129);
sample (Paragraph 131), at the processor and based on the posterior probability distribution of the set of parameters, parameter values of the set of parameters of the Bayesian Neural Network (Paragraph 129).
Yu, Casas, and Dalgaty are considered analogous to the invention because all concern the field of machine learning, and more specifically, Bayesian Neural Networks. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Yu to incorporate the teachings of Casas and Dalgaty, and include a posterior probability distribution of the set of parameters. Doing so provides a model that is more robust to overfitting and achieves higher accuracy (see paragraph 4 of Dalgaty).
Regarding claim 3, the rejection of claim 2 is incorporated. Yu and Casas fail to disclose the further limitations of the claim. However, Dalgaty discloses inference of the posterior probability distribution of the set of parameters of the Bayesian Neural Network including application of a variational inference technique based on variational free energy (Paragraph 130).
Yu, Casas, and Dalgaty are considered analogous to the invention because all concern the field of machine learning, and more specifically, Bayesian Neural Networks. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Yu to incorporate the teachings of Casas and Dalgaty, and include application of variational inference technique based on variational free energy. Doing so provides a more computationally inexpensive way to train the BNN and approximate the posterior distribution (see Paragraphs 9-10 of Dalgaty).
Claim 12 is a method claim corresponding to system claim 2 and is rejected for the same reasons as given in the rejection of that claim.
Claim 13 is a method claim corresponding to system claim 3 and is rejected for the same reasons as given in the rejection of that claim.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Yu et al. (“Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction”, 2020) in view of Casas et al. (US 20210276595) and in further view of Sharma et al (“Do Bayesian Neural Networks Need To Be Fully Stochastic?”, 2022).
The rejection of claim 1 is incorporated. Yu and Casas fail to disclose the further limitations of the claim. However, Sharma discloses inference of the posterior probability distribution of the set of parameters of the Bayesian Neural Network of the decoder including an output layer, the output layer being deterministic (Table 1, Page 8).
Yu, Casas, and Sharma are considered analogous to the invention because all concern the field of machine learning, and more specifically, Bayesian Neural Networks. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Yu to incorporate the teachings of Casas and Sharma, and incorporate a deterministic layer within the architecture of the Bayesian Neural Network. Doing so provides a way to reduce computational complexity whilst maintaining performance.
Claim 5, 14 are rejected under 35 U.S.C. 103 as being unpatentable over Yu et al. (“Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction”, 2020) in view of Casas et al. (US 20210276595) and in further view of Shahbazi et al. (“Density-Based Clustering and Performance Enhancement of Aeronautical Ad Hoc Networks”, 2022).
The rejection of claim 1 is incorporated. Yu discloses the construction of the input graph including trajectory of agents over the plurality of agents for one or more timesteps of the plurality of timesteps (Figure 4, Figure 5A). Yu and Casas to disclose the further limitations of the claim. However, Shahbazi discloses the incorporation of a Haversine distance between a first agent and a second agent of the plurality of agents for one or more timesteps of the plurality of timesteps (Section III.A), the first agent being a first aircraft and the second agent being a second aircraft (Page 2, first second bullet).
Yu, Casas, and Shahbazi are considered analogous to the invention because all concern the field of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Yu to incorporate the teachings of Casas and Shahbazi, and incorporate haversine distance calculations to the architecture and apply it to aircraft. Doing so provides more stability and accuracy in the management of in-flight aircraft (see abstract of Shahbazi).
Claim 14 is a method claim corresponding to system claim 5 and is rejected for the same reasons as given in the rejection of that claim.
Claims 8-10, 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Yu et al. in view of Casas et al. (US 20210276595) and in further view of Gao et al. (US 20210103744).
Regarding claim 8, the rejection of claim 6 is incorporated. Yu further discloses the application of the preliminary spatiotemporal embedding as an input (Figure 4), generating a second updated temporal embedding (Figure 4), as well as generating the spatiotemporal embedding for the input graph (Figure 4).
Yu fails to disclose that the input of the second temporal transformer is the preliminary spatiotemporal embedding, as well as the input to the second spatial transformer being the second updated temporal embedding. Gao, however, teaches a system of spatial and temporal embeddings being fed to one another (Figure 3A), wherein the order of said transformers is not withheld to a specific embodiment (Paragraph 173-174).
Yu, Casas, and Gao are considered analogous to the invention because all are in the field of machine learning, and specifically with the creation of spatiotemporal embeddings. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Yu to incorporate the teachings of Casas and Gao, and allow for flexibility in the architecture of the system. Doing so allows for increased computation and time efficiency when using spatiotemporal embedding for downstream tasks, such as input to another transformer at a sequential stage (see paragraphs 15-16 of Gao).
Regarding claim 9, the rejection of claim 8 is incorporated. Yu discloses the parallel stage of the encoder including a first temporal transformer in communication with a graph memory, and the memory further including instructions executable by the processor to:
apply the second updated temporal embedding to the first temporal transformer of the parallel stage of the encoder through the graph memory (Figure 4).
Regarding claim 10, the rejection of claim 1 is incorporated. Yu and Casas fail to disclose the further limitations of the claim. However, Gao discloses the memory further including instructions executable by the processor to:
generate a graphical representation of a user interface for display at a display device in communication with the processor, the graphical representation representing the trajectory prediction for the plurality of agents for the one or more future timesteps (Paragraph 149).
Yu, Casas, and Gao are considered analogous to the invention because all are in the field of machine learning, and specifically with the creation of spatiotemporal embeddings. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified Yu to incorporate the teachings of Casas and Gao and include a means for display. Doing so allows for increased computation and time efficiency when using spatiotemporal embedding for downstream tasks, such as graphical representation of a user interface (see paragraphs 15-16 of Gao).
Claim 17 is a method claim corresponding to system claim 8 and is rejected for the same reasons as given in the rejection of that claim.
Claim 18 is a method claim corresponding to system claim 9 and is rejected for the same reasons as given in the rejection of that claim.
Claim 19 is a method claim corresponding to system claim 10 and is rejected for the same reasons as given in the rejection of that claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEWOS MESFIN whose telephone number is (571)270-0782. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEWOS MESFIN/Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
1 The yellow line/dots indicating the trajectory observation.
2 “STAR is able to model crowd interaction and spatio-temporal interactions”, Page 13. The crowd being the plurality of agents.
3 Timesteps indicated by each dot in graph.
Prosecution Insights