DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a Non-Final Action on the Merits. Claims 11-19 are currently pending and are addressed below.
Preliminary Amendment
The Preliminary Amendment filed on August 5th, 2025 has been considered and entered. Accordingly, claims 1-10 have been cancelled. Claims 11-19 have been newly added.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on August 5th, 2025 has been considered and entered.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 11-19 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without significantly more.
In sum, claims 11-19 are rejected under 35 U.S.C. §101 because the claimed invention is directed to a judicial exception to patentability (i.e., a law of nature, a natural phenomenon, or an abstract idea) and do not include an inventive concept that is something “significantly more” than the judicial exception under the January 2019 patentable subject matter eligibility guidance (2019 PEG) analysis which follows.
Under the 2019 PEG step 1 analysis, it must first be determined whether the claims are directed to one of the four statutory categories of invention (i.e., process, machine, manufacture, or composition of matter). Applying step 1 of the analysis for patentable subject matter to the claims, it is determined that the claims are directed to the statutory category of a process. Therefore, we proceed to step 2A, Prong 1.
Revised Guidance Step 2A – Prong 1
Under the 2019 PEG step 2A, Prong 1 analysis, it must be determined whether the claims recite an abstract idea that falls within one or more designated categories of patent ineligible subject matter (i.e., organizing human activity, mathematical concepts, and mental processes) that amount to a judicial exception to patentability.
Here, with respect to independent claims 11, 18, and 19, the claims recite the abstract idea of generating and identifying vehicle test data, and mentally determine “training a variable autoencoder of the electronic computing device with the extracted training trajectories; generating, using the trained variable autoencoder, potential test data used to simulate the assistance system; comparing, by the electronic computing device, the real recorded trajectory data with the potential test data; identifying, based on the comparing of the real recorded trajectory data with the potential test data, test data”, where these claims fall within one or more of the three enumerated 2019 PEG categories of patent ineligible subject matter, specifically, a mental process, that can be performed in the human mind since each of the above steps could alternatively be performed in the human mind or with the aid of pen and paper. This conclusion follows from CyberSource Corp. v. Retail Decisions, Inc., where our reviewing court held that section 101 did not embrace a process defined simply as using a computer to perform a series of mental steps that people, aware of each step, can and regularly do perform in their heads. 654 F.3d 1366, 1373 (Fed. Cir. 2011); see also In re Grams, 888 F.2d 835, 840–41 (Fed. Cir. 1989); In re Meyer, 688 F.2d 789, 794–95 (CCPA 1982); Elec. Power Group, LLC v. Alstom S.A., 830 F. 3d 1350, 1354–1354 (Fed. Cir. 2016) (“we have treated analyzing information by steps people go through in their minds, or by mathematical algorithms, without more, as essentially mental processes within the abstract-idea category”).
Additionally, mental processes remain unpatentable even when automated to reduce the burden on the user of what once could have been done with pen and paper. See CyberSource, 654 F.3d at 1375 (“That purely mental processes can be unpatentable, even when performed by a computer, was precisely the holding of the Supreme Court in Gottschalk v. Benson.”). These limitations, as drafted, are a simple process that under their broadest reasonable interpretation, covers the performance of the limitations of the mind. For example, the claim limitation encompasses mentally gen generating and identifying vehicle test data based off of the information provided by the car’s sensors while traveling, or alternatively, mentally generating and identifying vehicle test data based on observations by a human.
For example, a human could mentally and with the aid of pen and paper generate and identify vehicle test data.
Revised Guidance Step 2A – Prong 2
Under the 2019 PEG step 2A, Prong 2 analysis, the identified abstract idea to which the claim is directed does not include limitations that integrate the abstract idea into a practical application, since the additional elements of an electronic computing device, memory, and an autoencoder are merely generic components used as a tool (“apply it”) to implement the abstract idea. (See, e.g., MPEP §2106.05(f)). See Alice, 573 U.S. at 223 (“[T]he mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.”)
In addition, the limitation “specifying real recorded trajectory data of at least one driving situation for an at least partially assisted driving operation of a motor vehicle having an assistance system; extracting, by an electronic computing device, training trajectories from the real recorded trajectory data” constitutes insignificant presolution activity that merely gathers data and, therefore, do not integrate the exception into a practical application. See In re Bilski, 545 F.3d 943, 963 (Fed. Cir. 2008) (en banc), aff' d on other grounds, 561 U.S. 593 (2010) (characterizing data gathering steps as insignificant extra-solution activity); see also CyberSource, 654 F.3d at 1371–72 (noting that even if some physical steps are required to obtain information from a database (e.g., entering a query via a keyboard, clicking a mouse), such data-gathering steps cannot alone confer patentability); OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015) (presenting offers and gathering statistics amounted to mere data gathering). Accord Guidance, 84 Fed. Reg. at 55 (citing MPEP § 2106.05(g)).
In addition, merely “[u]sing a computer to accelerate an ineligible mental process does not make that process patent-eligible.” Bancorp Servs., L.L.C. v. Sun Life Assur. Co. of Canada (U.S.), 687 F.3d 1266, 1279 (Fed. Cir. 2012); see also CLS Bank Int’l v. Alice Corp. Pty. Ltd., 717 F.3d 1269, 1286 (Fed. Cir. 2013) (en banc) (“simply appending generic computer functionality to lend speed or efficiency to the performance of an otherwise abstract concept does not meaningfully limit claim scope for purposes of patent eligibility.”), aff’d, 573 U.S. 208 (2014). Accordingly, the additional element of a processor does not transform the abstract idea into a practical application of the abstract idea.
Revised Guidance Step 2B
Under the 2019 PEG step 2B analysis, the additional elements are evaluated to determine whether they amount to something “significantly more” than the recited abstract idea. (i.e., an innovative concept). Here, the additional elements, such as: an electronic computing device, memory, and an autoencoder does not amount to an innovative concept since, as stated above in the step 2A, Prong 2 analysis, the claims are simply using the additional elements as a tool to carry out the abstract idea (i.e., “apply it”) on a computer or computing device and/or via software programming. (See, e.g., MPEP §2106.05(f)). The additional elements are specified at a high level of generality to simply implement the abstract idea and are not themselves being technologically improved. (See, e.g., MPEP §2106.05 I.A.). See Alice, 573 U.S. at 223 (“[T]he mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.”). Thus, these elements, taken individually or together, do not amount to “significantly more” than the abstract ideas themselves.
The additional elements of the dependent claims 12-17 merely refine and further limit the abstract idea of the independent claims and do not add any feature that is an “inventive concept” which cures the deficiencies of their respective parent claim under the 2019 PEG analysis. None of the dependent claims considered individually, including their respective limitations, include an “inventive concept” of some additional element or combination of elements sufficient to ensure that the claims in practice amount to something “significantly more” than patent-ineligible subject matter to which the claims are directed.
The elements of the instant claimed invention, when taken in combination do not offer substantially more than the sum of the functions of the elements when each is taken alone. The claims as a whole, do not amount to significantly more than the abstract idea itself because the claims do not effect an improvement to another technology or technical field; the claims do not amount to an improvement to the functioning of an electronic device itself which implements the abstract idea (e.g., the general purpose computer and/or the computer system which implements the process are not made more efficient or technologically improved); the claims do not perform a transformation or reduction of a particular article to a different state or thing (i.e., the claims do not use the abstract idea in the claimed process to bring about a physical change. See, e.g., Diamond v. Diehr, 450 U.S. 175 (1981), where a physical change, and thus patentability, was imparted by the claimed process; contrast, Parker v. Flook, 437 U.S. 584 (1978), where a physical change, and thus patentability, was not imparted by the claimed process); and the claims do not move beyond a general link of the use of the abstract idea to a particular technological environment
Accordingly, claims 11-19 are rejected under 35 USC 101 as being drawn to an abstract idea without significantly more, and thus are ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 11-13 and 17-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Demetriou (Generation of Driving Scenario Trajectories with Generative Adversiral Networks) (“Demetriou”) (Attached).
With respect to claim 1, Demetriou teaches a method comprising:
specifying real recorded trajectory data of at least one driving situation for an at least partially assisted driving operation of a motor vehicle having an assistance system (See at least Demetriou Page 1 Col. 2 “Scenarios are obtained through time-series (sequence of the ego-vehicle states) which in turn are the processed data collected by sensors of the AD vehicle. To the best of our knowledge, scenario extraction can, in general, be addressed with two approaches: an explicit rule-based approach [17] (that requires expert domain knowledge) and a clustering approach … In this work, we are provided with the data collected by Volvo Cars Corporation. This data consists of information about the ego vehicle and its surroundings such as detected objects, information about the road, etc. We focus on generating realistic scenario trajectories, in particular, the cut in trajectories for a specific tracked vehicle. To describe a trajectory, we consider two features: the relative latitude and longitude positions of the vehicle with respect to the ego vehicle.”);
extracting, by an electronic computing device, training trajectories from the real recorded trajectory data (See at least Demetriou Page 1 Col. 2 – Page 2 Col. 1 “To achieve this goal, our framework performs the following steps Extracting scenarios from the logged data- this is done with explicit rules defined by an expert; Building the models for generating trajectories similar to the extracted ones; Evaluating the obtained results- this is done by visual inspection and the metrics to be introduced … The objects’ trajectories are extracted from the raw-data (sensor measurements) and fused sensor data. These trajectories can vary in length from 1 second up to 1 hour. The length depends on how long the object was tracked by the ego vehicle, in the field of view (FoV).”);
training a variable autoencoder of the electronic computing device with the extracted training trajectories (See at least Demetriou FIG. 2 and Page 2 Col. 2 – Page 3 Col. 1 “This solution is based on the architecture proposed for text generation by Donahue et al. [4]. It consists of an autoencoder for time-series as shown in Fig. 2 and GAN for latent space representation and data generation. We adapt this architecture to deal with variable length in put/output. It is essential to know the length of the sequence in order to bring it back from the latent-space representation. We address this issue by training a separate feed-forward neural network to estimate the lengths based on the latent space representation. The reconstruction process is presented in Fig. 3. Hence, once the autoencoder is trained, all trajectories are encoded to the latent space using the encoder. During this process, the length of each trajectory is stored. Thus, two sets are created: X- set of latent representations, and Y set of the lengths for each trajectory. With these sets, the task of length estimation from latent space can be considered as a supervised regression task which can be solved using a feed-forward NN as discussed above”);
generating, using the trained variable autoencoder, potential test data used to simulate the assistance system (See at least Demetriou Page 3 Col. 1 “At this stage, GANs are used to generate new latent space representations. Even though it seems reasonable to implement both the generator and the discriminator as standard fully-connected NNs, in [4] the authors use the ResNet architecture [7] to mitigate the problems related to gradient instability. With that said, we investigate both approaches to experimentally illustrate which one works better. For the training phase, we consider two alternatives: original GANs and Wasserstein GANs with Gradient Penalty (WGAN-GP)”);
comparing, by the electronic computing device, the real recorded trajectory data with the potential test data (See at least Demetriou Page 3 Col. 2 “An important and challenging task is to choose proper metrics to evaluate the quality of the generated trajectories [12]. It is reasonable to first evaluate the results via visual ization to see whether they do make sense. However, as the results improve it becomes harder to determine precisely how good they are. Thus, it is important to consider quantitative evaluation metrics as well, such that one can objectively quantify the similarity of the generated trajectories with the original ones. One commonly used method to measure similarities be tween time series is Dynamic Time Warping (DTW). Thus, to compare sets of time series, we build a matrix of pairwise (DTW) distances between the samples from the two sets as shown in Table I.”); and
identifying, based on the comparing of the real recorded trajectory data with the potential test data, test data (See at least Demetriou Page 3 Col. 2 – Page 4 Col. 2 “In addition, we analyze this pairwise matrix using the following two metrics. 1) Matching + Coverage: In this approach, each sample from the generated set (called GSM with M samples) is matched with the closest sample from the real set (called RSN with N samples). The matching criterion is defined by the following formula: Even if reasonable results are achieved with this metric, it does not necessarily indicate that the model performs well, since many generated samples can be ’mapped’ to the same real sample. In this case, the coverage of the model is low. Thus we also measure a coverage metric as follows … We start by examining the results from the autoencoder. In Fig. 5 real and reconstructed trajectories are presented. The main difference between the trajectories is the smoothness of the reconstructed ones which is a typical feature of an autoencoder. We perform two experiments: a first experiment with the trajectories from 3 to 5 seconds and the second experiment with the 3 to 7 seconds trajectories. In both cases, a two layer LSTM cell is used. The loss values with respect to different sizes of hidden states are shown in Table II. For the first experiment, a hidden size of 32 is sufficient and produces meaningful results from a visual inspection point of view. However, for the second experiment, we choose the size of the hidden state to be 64 as it decreases the loss drastically. Note that close loss values for trajectories with different lengths does not necessarily imply the same level of performance for the autoencoder, since mean is calculated with respect to a different number of samples … the generated trajectories can be distinguished from the real ones. The samples from RC-GAN are noisier compared to the real ones, while samples generated with AE-GAN are more smooth. From a visual inspection, both models seem to capture the distribution of the trajectories. Similar to the real dataset, there are more trajectories generated close to the ego-vehicle and less further away. Both models also generate accelerating and decelerating cut-ins.”).
With respect to claim 12, Demetriou teaches that the potential test data or the test data is generated in such a way that the potential test data or the test data enable a functional device of the assistance system to be controlled (See at least Demetriou Page 1 Col. 1 “To address this issue, in this paper, we study an alterna tive scenario-based verification approach. It uses a scenario database created by extracting driving scenarios (e.g. cut in, overtaking, etc.) that the AD vehicle is exposed to in naturalistic driving situations. Once such a scenario database is developed, it can be used for test case generation and verification of the AD functionality in a virtual environment”).
With respect to claim 13, Demetriou teaches controlling the functional device of the assistance system using the test data (See at least Demetriou Page 1 Col. 1 “To address this issue, in this paper, we study an alternative scenario-based verification approach. It uses a scenario database created by extracting driving scenarios (e.g. cut in, overtaking, etc.) that the AD vehicle is exposed to in naturalistic driving situations. Once such a scenario database is developed, it can be used for test case generation and verification of the AD functionality in a virtual environment”).
With respect to claim 17, Demetriou teaches that the real recorded trajectory data are qualitatively and quantitatively compared with the potential test data (See at least Demetriou Page 3 Col. 2 – Page 4 Col. 1 “An important and challenging task is to choose proper metrics to evaluate the quality of the generated trajectories [12]. It is reasonable to first evaluate the results via visualization to see whether they do make sense. However, as the results improve it becomes harder to determine precisely how good they are. Thus, it is important to consider quantitative evaluation metrics as well, such that one can objectively quantify the similarity of the generated trajectories with the original ones. One commonly used method to measure similarities be tween time series is Dynamic Time Warping (DTW). Thus, to compare sets of time series, we build a matrix of pairwise (DTW) distances between the samples from the two sets as shown in Table I. Such a matrix can be used to find the most similar samples from the two investigated sets. In addition, we analyze this pairwise matrix using the following two metrics. 1) Matching + Coverage: In this approach, each sample from the generated set (called GSM with M samples) is matched with the closest sample from the real set (called RSN with N samples). The matching criterion is defined by the following formula: Even if reasonable results are achieved with this metric, it does not necessarily indicate that the model performs well, since many generated samples can be ’mapped’ to the same real sample. In this case, the coverage of the model is low. Thus we also measure a coverage metric as follows: However, even the combination of these metrics has still shortcomings. For instance, if there are two (or more) similar samples in the real set and many generated samples are ’mapped’ to one of the real samples, then the coverage decreases. However, this does not mean that the model performs poorly. Since the sets are very diverse, we consider GSM and RSN with M > N. In our experiments, we use M=4∗N 2) One-to-One Matching with Hungarian Method: The Hungarian algorithm is a matching method for one-to-one matching. Applying it to Table I gives the mapping for each sample from the generated set to the real set. This mapping ensures that the sum of distances of the paired samples is minimal. The main disadvantage of this approach is that the sample distributions in the real and generated sets may not be identical. For example, the last 10% of the matched samples can be outliers that are from irrelevant parts of the distribution. Once the aforementioned metrics are defined it is still an open question which ground truth should be used as a reference to compare the results with. To address this issue we spilt the real dataset into different subsets and apply these metrics among the different subsets. We use the results as a baseline when analyzing the trajectories obtained from the generation models. Fig. 5a shows 100 real trajectories wherein a cut-in occurs. It is clear that the distribution is not even as there are a lot more samples in the 20-60 meters longitudinal region while only a few are seen past 100 meters. Another observation that can be made is that the majority of trajectories have a trend to increase in the longitudinal distance through time. However, there are several samples where the longitudinal distance decrease instead. This can be interpreted as the cut ins wherein the tracked vehicle accelerates or decelerates respectively. It seems worth to check if the proposed models capture these different trends and outliers”).
With respect to claim 18, Demetriou teaches a non-transitory computer-readable storage medium having a computer program product containing program code, which when executed by an electronic computing device cause the electronic computing device to:
specify real recorded trajectory data of at least one driving situation for an at least partially assisted driving operation of a motor vehicle having an assistance system (See at least Demetriou Page 1 Col. 2 “Scenarios are obtained through time-series (sequence of the ego-vehicle states) which in turn are the processed data collected by sensors of the AD vehicle. To the best of our knowledge, scenario extraction can, in general, be addressed with two approaches: an explicit rule-based approach [17] (that requires expert domain knowledge) and a clustering approach … In this work, we are provided with the data collected by Volvo Cars Corporation. This data consists of information about the ego vehicle and its surroundings such as detected objects, information about the road, etc. We focus on generating realistic scenario trajectories, in particular, the cut in trajectories for a specific tracked vehicle. To describe a trajectory, we consider two features: the relative latitude and longitude positions of the vehicle with respect to the ego vehicle.”);
extract, by an electronic computing device, training trajectories from the real recorded trajectory data (See at least Demetriou Page 1 Col. 2 – Page 2 Col. 1 “To achieve this goal, our framework performs the following steps Extracting scenarios from the logged data- this is done with explicit rules defined by an expert; Building the models for generating trajectories similar to the extracted ones; Evaluating the obtained results- this is done by visual inspection and the metrics to be introduced … The objects’ trajectories are extracted from the raw-data (sensor measurements) and fused sensor data. These trajectories can vary in length from 1 second up to 1 hour. The length depends on how long the object was tracked by the ego vehicle, in the field of view (FoV).”);
train a variable autoencoder of the electronic computing device with the extracted training trajectories (See at least Demetriou FIG. 2 and Page 2 Col. 2 – Page 3 Col. 1 “This solution is based on the architecture proposed for text generation by Donahue et al. [4]. It consists of an autoencoder for time-series as shown in Fig. 2 and GAN for latent space representation and data generation. We adapt this architecture to deal with variable length in put/output. It is essential to know the length of the sequence in order to bring it back from the latent-space representation. We address this issue by training a separate feed-forward neural network to estimate the lengths based on the latent space representation. The reconstruction process is presented in Fig. 3. Hence, once the autoencoder is trained, all trajectories are encoded to the latent space using the encoder. During this process, the length of each trajectory is stored. Thus, two sets are created: X- set of latent representations, and Y set of the lengths for each trajectory. With these sets, the task of length estimation from latent space can be considered as a supervised regression task which can be solved using a feed-forward NN as discussed above”);
generate, using the trained variable autoencoder, potential test data used to simulate the assistance system (See at least Demetriou Page 3 Col. 1 “At this stage, GANs are used to generate new latent space representations. Even though it seems reasonable to implement both the generator and the discriminator as standard fully-connected NNs, in [4] the authors use the ResNet architecture [7] to mitigate the problems related to gradient instability. With that said, we investigate both approaches to experimentally illustrate which one works better. For the training phase, we consider two alternatives: original GANs and Wasserstein GANs with Gradient Penalty (WGAN-GP)”);
compare, by the electronic computing device, the real recorded trajectory data with the potential test data (See at least Demetriou Page 3 Col. 2 “An important and challenging task is to choose proper metrics to evaluate the quality of the generated trajectories [12]. It is reasonable to first evaluate the results via visual ization to see whether they do make sense. However, as the results improve it becomes harder to determine precisely how good they are. Thus, it is important to consider quantitative evaluation metrics as well, such that one can objectively quantify the similarity of the generated trajectories with the original ones. One commonly used method to measure similarities be tween time series is Dynamic Time Warping (DTW). Thus, to compare sets of time series, we build a matrix of pairwise (DTW) distances between the samples from the two sets as shown in Table I.”); and
identify, based on the comparing of the real recorded trajectory data with the potential test data, test data (See at least Demetriou Page 3 Col. 2 – Page 4 Col. 2 “In addition, we analyze this pairwise matrix using the following two metrics. 1) Matching + Coverage: In this approach, each sample from the generated set (called GSM with M samples) is matched with the closest sample from the real set (called RSN with N samples). The matching criterion is defined by the following formula: Even if reasonable results are achieved with this metric, it does not necessarily indicate that the model performs well, since many generated samples can be ’mapped’ to the same real sample. In this case, the coverage of the model is low. Thus we also measure a coverage metric as follows … We start by examining the results from the autoencoder. In Fig. 5 real and reconstructed trajectories are presented. The main difference between the trajectories is the smoothness of the reconstructed ones which is a typical feature of an autoencoder. We perform two experiments: a first experiment with the trajectories from 3 to 5 seconds and the second experiment with the 3 to 7 seconds trajectories. In both cases, a two layer LSTM cell is used. The loss values with respect to different sizes of hidden states are shown in Table II. For the first experiment, a hidden size of 32 is sufficient and produces meaningful results from a visual inspection point of view. However, for the second experiment, we choose the size of the hidden state to be 64 as it decreases the loss drastically. Note that close loss values for trajectories with different lengths does not necessarily imply the same level of performance for the autoencoder, since mean is calculated with respect to a different number of samples … the generated trajectories can be distinguished from the real ones. The samples from RC-GAN are noisier compared to the real ones, while samples generated with AE-GAN are more smooth. From a visual inspection, both models seem to capture the distribution of the trajectories. Similar to the real dataset, there are more trajectories generated close to the ego-vehicle and less further away. Both models also generate accelerating and decelerating cut-ins.”).
With respect to claim 19, Demetriou teaches an electronic computing device configured to:
specify real recorded trajectory data of at least one driving situation for an at least partially assisted driving operation of a motor vehicle having an assistance system (See at least Demetriou Page 1 Col. 2 “Scenarios are obtained through time-series (sequence of the ego-vehicle states) which in turn are the processed data collected by sensors of the AD vehicle. To the best of our knowledge, scenario extraction can, in general, be addressed with two approaches: an explicit rule-based approach [17] (that requires expert domain knowledge) and a clustering approach … In this work, we are provided with the data collected by Volvo Cars Corporation. This data consists of information about the ego vehicle and its surroundings such as detected objects, information about the road, etc. We focus on generating realistic scenario trajectories, in particular, the cut in trajectories for a specific tracked vehicle. To describe a trajectory, we consider two features: the relative latitude and longitude positions of the vehicle with respect to the ego vehicle.”);
extract, by an electronic computing device, training trajectories from the real recorded trajectory data (See at least Demetriou Page 1 Col. 2 – Page 2 Col. 1 “To achieve this goal, our framework performs the following steps Extracting scenarios from the logged data- this is done with explicit rules defined by an expert; Building the models for generating trajectories similar to the extracted ones; Evaluating the obtained results- this is done by visual inspection and the metrics to be introduced … The objects’ trajectories are extracted from the raw-data (sensor measurements) and fused sensor data. These trajectories can vary in length from 1 second up to 1 hour. The length depends on how long the object was tracked by the ego vehicle, in the field of view (FoV).”);
train a variable autoencoder of the electronic computing device with the extracted training trajectories (See at least Demetriou FIG. 2 and Page 2 Col. 2 – Page 3 Col. 1 “This solution is based on the architecture proposed for text generation by Donahue et al. [4]. It consists of an autoencoder for time-series as shown in Fig. 2 and GAN for latent space representation and data generation. We adapt this architecture to deal with variable length in put/output. It is essential to know the length of the sequence in order to bring it back from the latent-space representation. We address this issue by training a separate feed-forward neural network to estimate the lengths based on the latent space representation. The reconstruction process is presented in Fig. 3. Hence, once the autoencoder is trained, all trajectories are encoded to the latent space using the encoder. During this process, the length of each trajectory is stored. Thus, two sets are created: X- set of latent representations, and Y set of the lengths for each trajectory. With these sets, the task of length estimation from latent space can be considered as a supervised regression task which can be solved using a feed-forward NN as discussed above”);
generate, using the trained variable autoencoder, potential test data used to simulate the assistance system (See at least Demetriou Page 3 Col. 1 “At this stage, GANs are used to generate new latent space representations. Even though it seems reasonable to implement both the generator and the discriminator as standard fully-connected NNs, in [4] the authors use the ResNet architecture [7] to mitigate the problems related to gradient instability. With that said, we investigate both approaches to experimentally illustrate which one works better. For the training phase, we consider two alternatives: original GANs and Wasserstein GANs with Gradient Penalty (WGAN-GP)”);
compare, by the electronic computing device, the real recorded trajectory data with the potential test data (See at least Demetriou Page 3 Col. 2 “An important and challenging task is to choose proper metrics to evaluate the quality of the generated trajectories [12]. It is reasonable to first evaluate the results via visual ization to see whether they do make sense. However, as the results improve it becomes harder to determine precisely how good they are. Thus, it is important to consider quantitative evaluation metrics as well, such that one can objectively quantify the similarity of the generated trajectories with the original ones. One commonly used method to measure similarities be tween time series is Dynamic Time Warping (DTW). Thus, to compare sets of time series, we build a matrix of pairwise (DTW) distances between the samples from the two sets as shown in Table I.”); and
identify, based on the comparing of the real recorded trajectory data with the potential test data, test data (See at least Demetriou Page 3 Col. 2 – Page 4 Col. 2 “In addition, we analyze this pairwise matrix using the following two metrics. 1) Matching + Coverage: In this approach, each sample from the generated set (called GSM with M samples) is matched with the closest sample from the real set (called RSN with N samples). The matching criterion is defined by the following formula: Even if reasonable results are achieved with this metric, it does not necessarily indicate that the model performs well, since many generated samples can be ’mapped’ to the same real sample. In this case, the coverage of the model is low. Thus we also measure a coverage metric as follows … We start by examining the results from the autoencoder. In Fig. 5 real and reconstructed trajectories are presented. The main difference between the trajectories is the smoothness of the reconstructed ones which is a typical feature of an autoencoder. We perform two experiments: a first experiment with the trajectories from 3 to 5 seconds and the second experiment with the 3 to 7 seconds trajectories. In both cases, a two layer LSTM cell is used. The loss values with respect to different sizes of hidden states are shown in Table II. For the first experiment, a hidden size of 32 is sufficient and produces meaningful results from a visual inspection point of view. However, for the second experiment, we choose the size of the hidden state to be 64 as it decreases the loss drastically. Note that close loss values for trajectories with different lengths does not necessarily imply the same level of performance for the autoencoder, since mean is calculated with respect to a different number of samples … the generated trajectories can be distinguished from the real ones. The samples from RC-GAN are noisier compared to the real ones, while samples generated with AE-GAN are more smooth. From a visual inspection, both models seem to capture the distribution of the trajectories. Similar to the real dataset, there are more trajectories generated close to the ego-vehicle and less further away. Both models also generate accelerating and decelerating cut-ins.”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Demetriou (Generation of Driving Scenario Trajectories with Generative Adversiral Networks) (“Demetriou”) (Attached) in view of Hoseini (Vehicle Motion Trajectories Clustering via Embedding Transitive Relations) (“Hoseini”) (Attached”).
With respect to claim 14, Demetriou fails to explicitly disclose that the training trajectories are extracted by a classifier of the electronic computing device.
Hoseini, however, teaches that the training trajectories are extracted by a classifier of the electronic computing device (See at least Hoseini Page 1314-1315 “This can be done via different approaches [3] depending on the use cases. For example, for finding known/standard scenario classes (e.g. ”cut-in” scenario, where the target car cuts into the ego car lane), knowledge-based approaches use a prior specified rules for a cut-in scenario description with some threshold that needs to be set in advance. The main advantages with this approach are having insight and control over all steps, and the possibility to incorporate domain/expert knowledge about driving scenarios in scenario annotation and extraction. However, rule-based approaches can be subjected to errors such as miss-classification of some trajectories that lie on the border of two standard scenario classes and prone to the risk of missing unknown driving events (since a pre-specified known scenario class is not assigned for them). Therefore, as a complement approach we investigate using exploratory data management and machine learning tools in particular clustering approach to accelerate and verify the obtained scenario labels … We propose a framework for unsupervised clustering of vehicle trajectories with varying lengths, called DTMM. We first extract the temporal relations, and then, we embed them into a low-dimensional vector space, wherein the underlying structures and clusters can be detected. Since such clusters have elongated and complex shapes and boundaries, thus at the next step, we compute the transitive relations in order to make the cluster boundaries well separated. In the next step, since several clustering methods such as Gaussian Mixture Models (GMMs) require feature or vector representations of the data, we embed the transitive-based distances into a new vector space. Finally, we find the optimal number of clusters and apply a clustering method such as GMM. So, briefly stated, the proposed framework can are summarized into the following five steps: (1) aligning trajectories and quantifying their pairwise temporal dissimilarities (2) embedding the trajectory-based dissimilarities into a vector space (3) extracting transitive relations (4) embedding the transitive relations into a new vector space (5) clustering the trajectories with a determined number of clusters.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Demetriou to include that the training trajectories are extracted by a classifier of the electronic computing device, as taught by Hoseini as disclosed above, in order to ensure accurate training trajectories (Hoseini “To achieve this goal, we need to build a scenario database containing both real-world collected data and synthesized scenarios that are consistent with the real-world driving behaviour.”).
Claims 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Demetriou (Generation of Driving Scenario Trajectories with Generative Adversiral Networks) (“Demetriou”) (Attached) in view of Semple (US 20220266859 A1) (“Semple”).
With respect to claim 15, Demetriou fails to explicitly disclose filtering the extracted training trajectories such that a non-physical behavior of at least one object is removed in the real recorded trajectory data.
Semple, however, teaches filtering the extracted training trajectories such that a non-physical behavior of at least one object is removed in the real recorded trajectory data (See at least Semple Paragraphs 79-81 “In some instances, the simulator 540 can use filters to remove objects represented in the log data from a simulated scenario based on attributes associated with the objects. In some instances, the simulator 540 can filter objects based on an object/classification type (car, pedestrian, motorcycle, bicyclist, etc.), an object size (e.g., length, width, height, and/or volume), a confidence level, track length, an amount of interaction between the object and a vehicle generating the log data, and/or a time period. By way of example and without limitation, the log data can include objects of varying sizes such as mailboxes and buildings. The log-based simulation generator 538 can use a volume-based filter such that objects that are associated with a volume greater equal to or greater than a threshold volume of three cubic meters, such as buildings, are represented in the simulated scenario and objects that are associated with a volume less than three cubic meters are not represented in the simulated scenario, such as the mailboxes. In some instances, the log-based simulation generator 538 can use a track length filter where objects that have track lengths (e.g., data associated with a physical distance or a period of time) that do not meet or exceed a track length threshold are filtered from the simulated scenario. This can result in a simulation that omits objects associated with poor detections during the time of data capture. In some instances, the log-based simulation generator 538 can use a motion-based filter such that objects associated with motion or a trajectory according to the log data are represented in the simulation. In some instances, the filters can be applied in combination or mutually exclusively. In some instances, the log-based simulation generator 538 can filter objects that do not meet or exceed a confidence threshold. By way of example and without limitation, the log data can indicate that an object is associated with a classification attribute of a pedestrian and a confidence value of associated with the classification of 5%. The log-based simulation generator 538 can have a confidence value threshold of 75% and filter the object based on the confidence value not meeting or exceeding the confidence value threshold. In some instances, a user can provide a user-generated filter that includes one or more attribute thresholds such that the log-based simulation generator 538 can filter objects that do not meet or exceed the one or more attribute thresholds indicated by the user-generated filter.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Demetriou to include filtering the extracted training trajectories such that a non-physical behavior of at least one object is removed in the real recorded trajectory data, as taught by Semple as disclosed above, in order to ensure accurate training trajectories (Semple Paragraph 10 “Techniques are discussed herein for executing log-based driving simulations to evaluate the functionalities and performance of vehicle control systems.”).
With respect to claim 16, Demetriou in view of Semple teach that objects in the trajectory data with only a short period of existence compared to other objects are identified as ghost objects and filtered out (See at least Semple Paragraphs 79-81 “In some instances, the simulator 540 can use filters to remove objects represented in the log data from a simulated scenario based on attributes associated with the objects. In some instances, the simulator 540 can filter objects based on an object/classification type (car, pedestrian, motorcycle, bicyclist, etc.), an object size (e.g., length, width, height, and/or volume), a confidence level, track length, an amount of interaction between the object and a vehicle generating the log data, and/or a time period. By way of example and without limitation, the log data can include objects of varying sizes such as mailboxes and buildings. The log-based simulation generator 538 can use a volume-based filter such that objects that are associated with a volume greater equal to or greater than a threshold volume of three cubic meters, such as buildings, are represented in the simulated scenario and objects that are associated with a volume less than three cubic meters are not represented in the simulated scenario, such as the mailboxes. In some instances, the log-based simulation generator 538 can use a track length filter where objects that have track lengths (e.g., data associated with a physical distance or a period of time) that do not meet or exceed a track length threshold are filtered from the simulated scenario. This can result in a simulation that omits objects associated with poor detections during the time of data capture. In some instances, the log-based simulation generator 538 can use a motion-based filter such that objects associated with motion or a trajectory according to the log data are represented in the simulation. In some instances, the filters can be applied in combination or mutually exclusively. In some instances, the log-based simulation generator 538 can filter objects that do not meet or exceed a confidence threshold. By way of example and without limitation, the log data can indicate that an object is associated with a classification attribute of a pedestrian and a confidence value of associated with the classification of 5%. The log-based simulation generator 538 can have a confidence value threshold of 75% and filter the object based on the confidence value not meeting or exceeding the confidence value threshold. In some instances, a user can provide a user-generated filter that includes one or more attribute thresholds such that the log-based simulation generator 538 can filter objects that do not meet or exceed the one or more attribute thresholds indicated by the user-generated filter.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM ABDOALATIF ALSOMAIRY whose telephone number is (571)272-5653. The examiner can normally be reached M-F 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Faris Almatrahi can be reached at 313-446-4821. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IBRAHIM ABDOALATIF ALSOMAIRY/Examiner, Art Unit 3667
/FARIS S ALMATRAHI/Supervisory Patent Examiner, Art Unit 3667
Prosecution Insights