Computer-implemented system and method for predicting future developments of a traffic scene

§101 §102 §103 §112

Detailed Action Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claim Rejections - 35 USC § 112 The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 1 recites the limitation "…the different nodes…" in “a plurality of prediction modules, each of the plurality of prediction modules associated with a respective one of the different nodes…”. There is insufficient antecedent basis for this limitation in the claim. For the sake of compact prosecution, the examiner will be interpreting this limitation herein to mean “…the different modes…”, which would not lack antecedent basis. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process) without significantly more. Regarding claim 1, in Step 1 of the 101-analysis set forth in MPEP 2106, the claim recites “A computer-implemented system”. A computer-implemented system is within one of the four statutory categories of invention. In Step 2a Prong 1 of the 101-analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a process that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components: “…predict future developments of a traffic scene” (A person can mentally evaluate a traffic scene and make a judgement to predict future developments (MPEP 2106).) “…aggregate scene-specific information of an input scene” (A person can mentally evaluate the information of a given traffic scene and make a judgement to aggregate it (MPEP 2106).) “…generate a feature set of latent features based on the scene-specific information” (A person can mentally evaluate the information of the traffic scene and make a judgement to “generate” a feature set by identifying features based on that information (MPEP 2106).) “…evaluate a specified number of different modes for the future developments of the input scene based on the feature set” (A person can mentally evaluate a specific number of modes for the previously predicted future developments (MPEP 2106).) “…generate a respective prediction for the future development of the input scene…” (A person can mentally evaluate the scene using several “nodes/neurons” and using at least one of those, make a judgement to generate respective predictions for future developments of the scene (MPEP 2106).) If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. According, the claim “recites” an abstract idea. In Step 2a Prong 2 of the 101-analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application: “A computer-implemented system configured to…” (Uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).) “a perception level configured to…” (Mere instructions to apply the judicial exception (MPEP 2106.05(f)).) “a backbone network configured to…” (Mere instructions to apply the judicial exception (MPEP 2106.05(f)).) “a classifier configured to…” (Mere instructions to apply the judicial exception (MPEP 2106.05(f)).) “a plurality of prediction modules, each of the plurality of prediction modules associated with a respective one of the different nodes, and configured… wherein at least one prediction module of the plurality of prediction modules is optionally activated” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)).) Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea. In Step 2b of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, additional elements (vi) recites use of a computer as a tool to perform the abstract idea, which is not indicative of significantly more. Additional elements (vii) (viii), & (ix) recite mere instructions to apply the judicial exception, which is not indicative of significantly more. Additional element (x) recites generally linking the use of a judicial exception to a particular technological environment or field of use, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible. Regarding claim 2, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 2 recites “wherein the optional activation of the at least one prediction module is dependent on the evaluation of the associated mode carried out by the classifier” (In step 2a, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h).) In step 2B, generally linking the use of the judicial exception to a particular technological environment or field of use is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 3, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 3 recites the following additional mental process: “generate a prediction for the future development of the input scene based on the feature set” (A person can mentally evaluate the features of a traffic scene and make a judgement to generate a prediction based on that (MPEP 2106).) Further, claim 3 recites “wherein at least a first prediction module of the plurality of prediction modules is a scene anchor network (SAN), configured to…” (In step2A, prong 2, this recites mere instructions to apply the judicial exception (MPEP 2106.05(f).) In step 2B, mere instructions to apply the judicial exception is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 4, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 4 recites the following additional mental process: “generate a prediction for the future development of the input scene based on the scene-specific information” (A person can mentally evaluate the information of a traffic scene and make a judgement to generate a prediction based on that (MPEP 2106).) Further, claim 4 recites “wherein at least a first prediction module of the plurality of prediction modules is a pre-trained prediction network or in the form of a model-based prediction module, configured to…” (In step2A, prong 2, this recites mere instructions to apply the judicial exception (MPEP 2106.05(f).) In step 2B, mere instructions to apply the judicial exception is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 5, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 5 recites the following additional mental process: “aggregate semantic information about the input scene, in the form of map information and/or information about traffic participants in the input scene in the form of information about the current state of movement and/or the traveled trajectory of the traffic participants, as scene-specific information” (A person can mentally evaluate the information of a traffic scene and make a judgement to aggregate the information in the form of map information and/or about participants’ movement and/or trajectory (MPEP 2106).) Further, claim 5 recites “the perception level is configured to…” (In step2A, prong 2, this recites mere instructions to apply the judicial exception (MPEP 2106.05(f).) In step 2B, mere instructions to apply the judicial exception is not indicative of significantly more.) Further, claim 5 recites ““convert the scene-specific information into a data representation processable by the backbone network and/or into a data representation processable by a pre-trained prediction network” (In step 2A, prong 2, this recites insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g).) In step 2B, the courts have found steps that store and retrieve information in memory to be a well-understood, routine, and conventional activity, which is not indicative of significantly more (Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)).) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 6, it is dependent upon claim 5, and thereby incorporates the limitations of, and corresponding analysis applied to claim 5. Further, claim 6 recites the following additional mental process: “convert the scene-specific information into one of a graph representation, a grid representation, and a voxel grid representation” (A person can mentally evaluate the information of a traffic scene and make a judgement to convert it into a grid representation (MPEP 2106).) Further, claim 6 recites “the perception level is configured to…” (In step2A, prong 2, this recites mere instructions to apply the judicial exception (MPEP 2106.05(f).) In step 2B, mere instructions to apply the judicial exception is not indicative of significantly more.) Further, claim 6 recites “the backbone network and/or the pre-trained prediction network is correspondingly realized in the form of a graph neural network (GNN) or in the form of a convolutional neural network (CNN)” (In step 2a, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h).) In step 2B, generally linking the use of the judicial exception to a particular technological environment or field of use is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 7, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 7 recites “wherein the classifier is realized in the form of a neural network, whose type depends on a data representation of the feature set” (In step 2a, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h).) In step 2B, generally linking the use of the judicial exception to a particular technological environment or field of use is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 8, it is dependent upon claim 7, and thereby incorporates the limitations of, and corresponding analysis applied to claim 7. Further, claim 8 recites the following additional mental process: “generate a feature set in the form of a feature vector” (A person can mentally evaluate the features of a traffic scene and make a judgement to generate a vector based on that (MPEP 2106).) Further, claim 8 recites “the backbone network is configured to…” (In step2A, prong 2, this recites mere instructions to apply the judicial exception (MPEP 2106.05(f).) In step 2B, mere instructions to apply the judicial exception is not indicative of significantly more.) Further, claim 8 recites “the classifier is realized in the form of a feed forward neural network” (In step 2a, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h).) In step 2B, generally linking the use of the judicial exception to a particular technological environment or field of use is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 9, in Step 1 of the 101-analysis set forth in MPEP 2106, the claim recites “A computer-implemented method”. A method is within one of the four statutory categories of invention. In Step 2a Prong 1 of the 101-analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a process that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components: “…predicting future developments of a traffic scene” (A person can mentally evaluate a traffic scene and make a judgement to predict future developments (MPEP 2106).) “aggregating scene-specific information of an input scene…” (A person can mentally evaluate the information of a given traffic scene and make a judgement to aggregate it (MPEP 2106).) “generating at least one feature set of latent features based on the scene-specific information…” (A person can mentally evaluate the information of the traffic scene and make a judgement to “generate” a feature set by identifying features based on that information (MPEP 2106).) “evaluating a specified number of different modes for the future developments of the input scene based on the feature set…” (A person can mentally evaluate a specific number of modes for the previously predicted future developments (MPEP 2106).) “selecting at least one mode based on the evaluation by the classifier…” (A person can mentally evaluate several modes and make a judgement to select one based on that evaluation (MPEP 2106).) “generating a prediction for the future development of the input scene…” (A person can mentally evaluate the scene using several “nodes/neurons” and using at least one of those, make a judgement to generate respective predictions for future developments of the scene (MPEP 2106).) If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. According, the claim “recites” an abstract idea. In Step 2a Prong 2 of the 101-analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application: “A computer-implemented method for…” (Uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).) “…with the aid of a backbone network” (Mere instructions to apply the judicial exception (MPEP 2106.05(f)).) “…with the aid of a classifier” (Mere instructions to apply the judicial exception (MPEP 2106.05(f)).) “a classifier configured to…” (Mere instructions to apply the judicial exception (MPEP 2106.05(f)).) “…activating at least one prediction module associated with the selected mode” (Mere instructions to apply the judicial exception (MPEP 2106.05(f)).) “…with the aid of the at least one activated prediction module” (Mere instructions to apply the judicial exception (MPEP 2106.05(f)).) Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea. In Step 2b of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, additional elements (vii) recites use of a computer as a tool to perform the abstract idea, which is not indicative of significantly more. Additional elements (viii) (ix), (x), (xi) & (xii) recite mere instructions to apply the judicial exception, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible. Regarding claims 10-12, they are dependent upon claim 9, and thereby incorporate the limitations of, and corresponding analysis applied to claim 9. Further, claims 10-12 comprise similar additional limitations as claims 5, 3, & 4, respectively, and are rejected under the same rationale. Regarding claim 13, it is dependent upon claim 9, and thereby incorporates the limitations of, and corresponding analysis applied to claim 9. Further, claim 13 recites the following additional mental process: “a deterministic or probabilistic parametric or non-parametric trajectory is generated for each traffic participant in the input scene as a future development of the input scene” (A person can mentally evaluate the participants of a traffic scene and make a judgement to “generate” trajectories for them as predictions of future developments of that scene (MPEP 2106).) Further, claim 13 recites “…with the aid of the at least one activated prediction module…” (In step2A, prong 2, this recites mere instructions to apply the judicial exception (MPEP 2106.05(f).) In step 2B, mere instructions to apply the judicial exception is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 14, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 14 recites “a monitoring system” (In step 2a, prong 2, this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h).) In step 2B, generally linking the use of the judicial exception to a particular technological environment or field of use is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Regarding claim 15, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 14 recites the following additional mental process: “…plan trajectory and/or maneuvering of a vehicle” (A person can mentally evaluate the aforementioned prediction of the traffic scene and make a judgement to plan the trajectory or maneuvering of a vehicle based on that (MPEP 2106).) Further, claim 15 recites “A vehicle module… configured to…” (In step2A, prong 2, this recites mere instructions to apply the judicial exception (MPEP 2106.05(f).) In step 2B, mere instructions to apply the judicial exception is not indicative of significantly more.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Claim Rejections - 35 USC § 102 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. Claims 1-7 & 9-13 are rejected under 35 U.S.C. 102(a)(1) as being clearly anticipated by Chai, Y. et al. “MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction.” Obtained from the IDS (hereafter, CHAI) Regarding claim 1, CHAI teaches “A computer-implemented system configured to predict future developments of a traffic scene”: ([Introduction, Paragraph 1] “…it is important to know whether to yield to a vehicle if they are going to cut in front of our robot or when would be the best time to merge into traffic. Such future prediction requires an understanding of the static and dynamic world context…”) This citation explicitly describes predicting future developments of a traffic scene. Further, CHAI teaches “a perception level configured to aggregate scene-specific information of an input scene; a backbone network configured to generate a feature set of latent features based on the scene-specific information”: ([Figure 1] PNG media_image1.png 423 702 media_image1.png Greyscale “As shown in the figure, MultiPath receives an “input scene” which is used to aggregate scene-specific information as the “Scene features” are identified and extracted to “encode the state of individual agents and their interactions” in the scene.”) Further, CHAI teaches “a classifier configured to evaluate a specified number of different modes for the future developments of the input scene based on the feature set”: ([Introduction, Paragraphs 4-7] “…Models which try to achieve diversity and coverage often suffer from mode collapse during training [4, 5, 6], while tractable probabilistic inference is difficult due to the space of possible trajectories growing exponentially over time. Our MultiPath model (a classifier) addresses these issues with a key insight: it employs a fixed set of trajectory anchors as the basis of our modeling… Our trajectory anchors are modes found in our training data in state-sequence space via unsupervised learning… Our complete model predicts a Gaussian mixture model (GMM) at each time step, with the mixture weights (intent distribution) fixed over time. Given such a parametric distribution model, we can directly evaluate the likelihood of any future trajectory and also have a simple way to obtain a compact, diverse weighted set of trajectory samples: the MAP sample from each anchor-intent.”) The “trajectory anchors” are different “modes” and a “fixed set of trajectory anchors” would then equivalate to a “specified number of different modes”. Further, it is shown to be used for the evaluation of the future developments of the input scene, which is based on the feature set. Further, CHAI teaches “a plurality of prediction modules, each of the plurality of prediction modules associated with a respective one of the different nodes, and configured to generate a respective prediction for the future development of the input scene, wherein at least one prediction module of the plurality of prediction modules is optionally activated”: ([Introduction, Paragraphs 5-7] “Our MultiPath model addresses these issues with a key insight: it employs a fixed set of trajectory anchors (modes) as the basis of our modeling. This lets us factor stochastic uncertainty hierarchically: First, intent uncertainty captures the uncertainty of what an agent intends to do and is encoded as a distribution over the set of anchor trajectories (modes). Second, given an intent, control uncertainty represents our uncertainty over how they might achieve it. We assume control uncertainty is normally distributed at each future time step [7], parameterized such that the mean corresponds to a context-specific offset from the anchor state, with the associated covariance capturing the unimodal aleatoric uncertainty [8]... (a plurality of prediction modules are presented, including the “intent uncertainty”, the “control uncertainty”, and the “unimodal aleatoric uncertainty”, all of which are associated with respective “anchors/modes”) Our trajectory anchors are modes found in our training data in state-sequence space via unsupervised learning. These anchors provide templates for coarse-granularity futures for an agent and might correspond to semantic concepts like “change lanes”, or “slow down” (although to be clear, we don’t use any semantic concepts in our modeling). (generating respective predictions of future traffic developments) Our complete model predicts a Gaussian mixture model (GMM) at each time step, with the mixture weights (intent distribution) fixed over time. Given such a parametric distribution model, we can directly evaluate the likelihood of any future trajectory and also have a simple way to obtain a compact, diverse weighted set of trajectory samples: the MAP sample from each anchor-intent.”) At least one of the prediction modules is activated over the course of the process, which activates them in association with different modes to generate respective future developments of a traffic scene. Dependent on the mode/anchor trajectory of the classifier, different associated prediction modules will be used, making them optionally activated. Regarding claim 2, CHAI teaches the limitations of claim 1. Further, CHAI teaches “wherein the optional activation of the at least one prediction module is dependent on the evaluation of the associated mode carried out by the classifier”: Our complete model predicts a Gaussian mixture model (GMM) at each time step, with the mixture weights (intent distribution) fixed over time. Given such a parametric distribution model, we can directly evaluate the likelihood of any future trajectory and also have a simple way to obtain a compact, diverse weighted set of trajectory samples: the MAP sample from each anchor-intent.”) At least one of the prediction modules is activated over the course of the process, which activates them in association with different modes to generate respective future developments of a traffic scene. Dependent on the mode/anchor trajectory of the classifier, different associated prediction modules will be used, making them optionally activated. Regarding claim 3, CHAI teaches the limitations of claim 1. Further, CHAI teaches “wherein at least a first prediction module of the plurality of prediction modules is a scene anchor network (SAN), configured to generate a prediction for the future development of the input scene based on the feature set”: ([Introduction, Paragraphs 5-7] “Our MultiPath model addresses these issues with a key insight: it employs a fixed set of trajectory anchors (modes) as the basis of our modeling. This lets us factor stochastic uncertainty hierarchically: First, intent uncertainty captures the uncertainty of what an agent intends to do and is encoded as a distribution over the set of anchor trajectories (the set of anchor trajectories being the modes means that the network is a “scene anchor network”). Second, given an intent, control uncertainty represents our uncertainty over how they might achieve it. We assume control uncertainty is normally distributed at each future time step [7], parameterized such that the mean corresponds to a context-specific offset from the anchor state, with the associated covariance capturing the unimodal aleatoric uncertainty [8]... (a plurality of prediction modules are presented, including the “intent uncertainty”, the “control uncertainty”, and the “unimodal aleatoric uncertainty”, all of which are associated with respective “anchors/modes”) Regarding claim 4, CHAI teaches the limitations of claim 1. Further, CHAI teaches “wherein at least a first prediction module of the plurality of prediction modules is a pre-trained prediction network or in the form of a model-based prediction module, configured to generate a prediction for the future development of the input scene based on the scene-specific information”: ([Figure 1] “As shown in the figure above, the “Agent-centric CNN” (a first prediction module, which is a pre-trained prediction network) extracts features from the input scene to be used in the rest of the process to generate a prediction for the future development of that scene based on the information extracted.”) Regarding claim 5, CHAI teaches the limitations of claim 1. Further, CHAI teaches “wherein: the perception level is configured to aggregate semantic information about the input scene, in the form of map information and/or information about traffic participants in the input scene in the form of information about the current state of movement and/or the traveled trajectory of the traffic participants, as scene-specific information”: ([Figure 1] “As shown in the figure above, the three bars below the input scene represent the semantic information of the input scene which are used to extract features using the CNN. As shown in the figure description, it recognizes “agents/traffic participants” in the scene, their current trajectories (current state of movement).”) And further: ([2. Related Work, Final Paragraph] “Our method is influenced heavily by the concept of predefined anchors, which have a rich history in machine learning applications to handle multi-modal problems, starting with classic semi parametric methods such as locally-weighted logistic regression, radial basis SVM and Gaussian Mixture Models [5]. In the computer vision literature, they have been used effectively for detection [21] and human-pose estimation [22]. Like ours, these effective approaches predict the likelihood of anchors and also predict continuous refinements of state conditioned on these anchors (e.g. box corners, joint locations or vehicle positions).”) Further, CHAI teaches “the perception level is configured to convert the scene-specific information into a data representation processable by the backbone network and/or into a data representation processable by a pre-trained prediction network”: ([Figure 1] “As shown in the figure above, the scene specific information taken from the input scene is converted into a scene features extracted into a data representation processable by a pre-trained prediction network (the agent-centric CNN).”) Regarding claim 6, CHAI teaches the limitations of claim 5. Further, CHAI teaches “wherein: the perception level is configured to convert the scene-specific information into one of a graph representation, a grid representation, and a voxel grid representation”: ([Figure 1] “As shown in the figure above, the scene specific information taken from the input scene is converted into a scene features extracted into a data representation processable by a pre-trained prediction network (the agent-centric CNN), which is shown to take the form of a grid representation.”) Further, CHAI teaches “the backbone network and/or the pre-trained prediction network is correspondingly realized in the form of a graph neural network (GNN) or in the form of a convolutional neural network (CNN)”: ([Figure 1] “The agent-centric CNN is a convolutional neural network, which uses a graph representation, which also makes it a graph neural network.”) Regarding claim 7, CHAI teaches the limitations of claim 1. Further, CHAI teaches “wherein the classifier is realized in the form of a neural network, whose type depends on a data representation of the feature set”: ([Figure 1] “The agent-centric CNN is a convolutional neural network, which uses a graph representation, which also makes it a graph neural network, because of the graph features analyzed.”) Regarding claim 9, CHAI teaches “A computer-implemented method for predicting future developments of a traffic scene”: ([Abstract] “Predicting human behavior is a difficult and crucial task required for motion planning. It is challenging in large part due to the highly uncertain and multimodal set of possible outcomes in real-world domains such as autonomous driving (“predicting human behavior” in the domain of “autonomous driving” is equivalent to “predicting movement/ future developments” in “a traffic scene”, which is the typical scenario of “autonomous driving”). Beyond single MAP trajectory prediction [1, 2], obtaining an accurate probability distribution of the future is an area of active interest [3, 4]. We present MultiPath (A computer-implemented system), which leverages a fixed set of future state-sequence anchors that correspond to modes of the trajectory distribution. At inference, our model predicts a discrete distribution over the anchors and, for each anchor, regresses offsets from anchor waypoints along with uncertainties, yielding a Gaussian mixture at each time step. Our model is efficient, requiring only one forward inference pass to obtain multi-modal future distributions, and the output is parametric, allowing compact communication and analytical probabilistic queries. We show on several datasets that our model achieves more accurate predictions, and compared to sampling baselines, does so with an order of magnitude fewer trajectories.”) And further: ([Introduction, Paragraph 1] “…it is important to know whether to yield to a vehicle if they are going to cut in front of our robot or when would be the best time to merge into traffic. Such future prediction requires an understanding of the static and dynamic world context…”) This citation explicitly describes predicting future developments of a traffic scene. Further, CHAI teaches “aggregating scene-specific information of an input scene; generating at least one feature set of latent features based on the scene-specific information with the aid of a backbone network”: ([Figure 1] “As shown in the figure above, MultiPath receives an “input scene” which is used to aggregate scene-specific information as the “Scene features” are identified and extracted to “encode the state of individual agents and their interactions” in the scene.”) Further, CHAI teaches “evaluating a specified number of different modes for the future developments of the input scene based on the feature set with the aid of a classifier”: ([Introduction, Paragraphs 4-7] “…Models which try to achieve diversity and coverage often suffer from mode collapse during training [4, 5, 6], while tractable probabilistic inference is difficult due to the space of possible trajectories growing exponentially over time. Our MultiPath model (a classifier) addresses these issues with a key insight: it employs a fixed set of trajectory anchors as the basis of our modeling… Our trajectory anchors are modes found in our training data in state-sequence space via unsupervised learning… Our complete model predicts a Gaussian mixture model (GMM) at each time step, with the mixture weights (intent distribution) fixed over time. Given such a parametric distribution model, we can directly evaluate the likelihood of any future trajectory and also have a simple way to obtain a compact, diverse weighted set of trajectory samples: the MAP sample from each anchor-intent.”) The “trajectory anchors” are different “modes” and a “fixed set of trajectory anchors” would then equivalate to a “specified number of different modes”. Further, it is shown to be used for the evaluation of the future developments of the input scene, which is based on the feature set. Further, CHAI teaches “selecting at least one mode based on the evaluation by the classifier and activating at least one prediction module associated with the selected mode; and generating a prediction for the future development of the input scene with the aid of the at least one activated prediction module”: ([Introduction, Paragraphs 5-7] “Our MultiPath model addresses these issues with a key insight: it employs a fixed set of trajectory anchors (modes) as the basis of our modeling. This lets us factor stochastic uncertainty hierarchically: First, intent uncertainty captures the uncertainty of what an agent intends to do and is encoded as a distribution over the set of anchor trajectories (modes). Second, given an intent, control uncertainty represents our uncertainty over how they might achieve it. We assume control uncertainty is normally distributed at each future time step [7], parameterized such that the mean corresponds to a context-specific offset from the anchor state, with the associated covariance capturing the unimodal aleatoric uncertainty [8]... (a plurality of prediction modules are presented, including the “intent uncertainty”, the “control uncertainty”, and the “unimodal aleatoric uncertainty”, all of which are associated with respective “anchors/modes”) Our trajectory anchors are modes found in our training data in state-sequence space via unsupervised learning. These anchors provide templates for coarse-granularity futures for an agent and might correspond to semantic concepts like “change lanes”, or “slow down” (although to be clear, we don’t use any semantic concepts in our modeling). (generating respective predictions of future traffic developments) Our complete model predicts a Gaussian mixture model (GMM) at each time step, with the mixture weights (intent distribution) fixed over time. Given such a parametric distribution model, we can directly evaluate the likelihood of any future trajectory and also have a simple way to obtain a compact, diverse weighted set of trajectory samples: the MAP sample from each anchor-intent.”) At least one of the prediction modules is activated over the course of the process, which activates them in association with different modes to generate respective future developments of a traffic scene. Dependent on the mode/anchor trajectory of the classifier, different associated prediction modules will be used, making them optionally activated. Regarding claims 10-12, CHAI teaches the limitations of claim 9. Further, claims 10-12 comprise similar additional limitations as claims 5, 3, & 4, respectively, and are rejected under the same rationale. Regarding claim 13, CHAI teaches the limitations of claim 9. Further, CHAI teaches “wherein, with the aid of the at least one activated prediction module, a deterministic or probabilistic parametric or non-parametric trajectory is generated for each traffic participant in the input scene as a future development of the input scene”: ([Conclusion] “We have introduced MultiPath, a model which predicts parametric distributions of future trajectories for agents in real-world settings (predicting parametric trajectories for agents/traffic participants in the input scene). Through synthetic and real-world datasets, we have shown the benefits of MultiPath over previous single-trajectory and stochastic models in achieving likelihood and trajectory-set metrics and needing only 1 feed-forward inference pass.”) Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over CHAI, as applied to claims above, and further in view of Gao, J. et al. “VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation.” Available at https://arxiv.org/abs/2005.04259 on 8 May 2020 (hereafter, GAO) Regarding claim 8, CHAI teaches the limitations of claim 7. Further, CHAI teaches “the classifier is realized in the form of a feed forward neural network”: ([Conclusion] “We have introduced MultiPath, a model which predicts parametric distributions of future trajectories for agents in real-world settings. Through synthetic and real-world datasets, we have shown the benefits of MultiPath over previous single-trajectory and stochastic models in achieving likelihood and trajectory-set metrics and needing only 1 feed-forward inference pass.”) Further, CHAI fails to explicitly teach “wherein: the backbone network is configured to generate a feature set in the form of a feature vector.” However, analogous art, GAO, does teach this: ([Abstract] “In contrast to most recent approaches, which render trajectories of moving agents and road context information as bird-eye images and encode them with convolutional neural networks (ConvNets), our approach operates on a vector representation. By operating on the vectorized high definition (HD) maps and agent trajectories, we avoid lossy rendering and computationally intensive ConvNet encoding steps.”) And further: ([Figure 1] “ PNG media_image2.png 322 431 media_image2.png Greyscale It can be seen in the above figure that the features of an input traffic scene are converted into a vector representation”) It would be obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to combine the base reference of CHAI with the teachings of GAO because both explore effective means of predicting future developments of a traffic scene. One of ordinary skill in the art would be motivated to do so because, as pointed out by GAO in the Abstract, “By operating on the vectorized high definition (HD) maps and agent trajectories, we avoid lossy rendering and computationally intensive ConvNet encoding steps” and “Our method achieves on par or better performance than the competitive rendering approach on both benchmarks while saving over 70% of the model parameters with an order of magnitude reduction in FLOPs.” Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over CHAI, as applied to claims above, and further in view of Sadat, A. et al. “Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehicles.” Available at https://arxiv.org/abs/1910.04586 on 10 October 2019 (hereafter, SADAT) Regarding claim 14, CHAI teaches the limitations of claim 1. Further, CHAI fails to explicitly teach “A monitoring system.” However, analogous art, SADAT, does teach this: ([Abstract] “The motion planners used in self-driving vehicles need to generate trajectories that are safe, comfortable, and obey the traffic rules... In this paper, we propose a jointly learnable behavior and trajectory planner… our approach features an interpretable cost function on top of perception (monitoring), prediction and vehicle dynamics, and a joint learning algorithm that learns a shared cost function employed by our behavior and trajectory components…”) And further: ([VII. Experiments, Section c) Spatiotemporal overlap] “A good planning should avoid obstacles. We measure the spatiotemporal overlap with obstacles within f1.0, 2.0, 3.0g seconds into the future, in terms of the percentage of scenarios. We use a perception and prediction (P&P) module (a monitoring system) as input to the planner…”) It would be obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to combine the base reference of CHAI with the teachings of SADAT because both references teach methods of planning the motion of automated vehicles. One of ordinary skill in the art would be motivated to do so because, as pointed out by the citation above, “A good planning should avoid obstacles” and an active monitoring system will help to keep actively updated scene input information. Regarding claim 15, CHAI teaches the limitations of claim 1. Further, CHAI fails to explicitly teach “A vehicle module… configured to plan trajectory and/or maneuvering of a vehicle.” However, analogous art, SADAT does teach this: ([Abstract] “The motion planners used in self-driving vehicles need to generate trajectories that are safe, comfortable, and obey the traffic rules... In this paper, we propose a jointly learnable behavior and trajectory planner (A vehicle module configured to plan trajectory of the vehicle)… our approach features an interpretable cost function on top of perception, prediction and vehicle dynamics, and a joint learning algorithm that learns a shared cost function employed by our behavior and trajectory components…”) It would be obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to combine the base reference of CHAI with the teachings of SADAT because both references teach methods of planning the motion of automated vehicles. One of ordinary skill in the art would be motivated to do so because, as pointed out by the citation above, “A good planning should avoid obstacles” and an active monitoring system will help to keep actively updated scene input information. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW LEE LEWIS whose telephone number is (571)272-1906. The examiner can normally be reached Monday: 9:30AM - 3:30PM and Tuesday - Friday: 9:30AM - 6PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at (571)272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Matthew Lee Lewis/Examiner, Art Unit 2144 /TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144