DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-15 of U.S. Application No. 18/817,819 filed on 08/28/2024 have been examined.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claims 11 and 12 of this application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because each of the claim limitations uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function. Such claim limitations are:
“an end point generator configured to determine” in claim 11
“a trajectory generator configured to determine” in claims 11 and 12
“a discriminator configured to evaluate” in claims 11 and 12
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. In the specification, applicant discloses that, “As such, the processor 702, the memory 704 and the non-transitory data storage 706 may represent input data receiving circuit 602, the extraction circuit 604, the end point determination circuit 606, the trajectory determination circuit 608, the trajectory evaluation circuit 610, as described above.” (See at least [0109] in applicant’s specification). It will therefore be appreciated that the functions disclosed are executed in the form of hardware and/or software executed by one or more processors. This is adequate structure to perform the claimed functions, so no 112(b) rejections are given based on this claim interpretation.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function) or make an argument as to why the claim limitations already recite sufficient structure as written. However, since the specification discloses adequate structure to perform the claimed functions, applicant does not need to take any action in response to this claim interpretation.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. The claimed invention is directed to the concept of tracking and predicting the movement of unclassified object in a scene. This judicial exception is not integrated into a practical application. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The Examiner will further explain in view of the Revised Patent Subject Matter Eligibility Guidance:
Claims 1 is directed to a computer implemented method (i.e., a process). Therefore, claim 1 is within at least one of the four statutory categories.
101 Analysis – Step 2A, Prong I
Regarding Prong I of the Step 2A analysis, the claims are to be analyzed to determine whether they recite subject matter that falls within one of the follow groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes.
Independent claims 1, 10, 13 and 15 include limitations that recite an abstract idea (emphasized below) and will be used as a representative claim for the remainder of the 101 rejection.
Claims 1, 10, 13 and 15 recites: A computer implemented method for determining and evaluating a trajectory of a road user, the method comprising:
receiving input data associated with a state of movement and with an environment of the road user,
extracting characteristics related to the road user from the input data,
determining one or more trajectory end points the road user by using the extracted characteristics,
determining, for each of the trajectory end points, a respective trajectory associated with one of the trajectory end points by using the associated trajectory end point and the extracted characteristics,
evaluating the respective trajectory of the road user by using a classification which relies on the extracted characteristics to provide a confidence score each trajectory associated with one of the trajectory end points.
The examiner submits that the foregoing bolded limitation(s) constitute a “mental process” because under its broadest reasonable interpretation, the claim covers performance of the limitation in the human mind. For example, “determine, extract, evaluate …” in the context of this claim encompasses a person looking at data collected and forming a simple judgement. Accordingly, the claim recites at least one abstract idea.
101 Analysis – Step 2A, Prong II
Regarding Prong II of the Step 2A analysis, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract into a practical application. As noted, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”):
A computer implemented method for determining and evaluating a trajectory of a road user, the method comprising:
receiving input data associated with a state of movement and with an environment of the road user,
extracting characteristics related to the road user from the input data,
determining one or more trajectory end points the road user by using the extracted characteristics,
determining, for each of the trajectory end points, a respective trajectory associated with one of the trajectory end points by using the associated trajectory end point and the extracted characteristics,
evaluating the respective trajectory of the road user by using a classification which relies on the extracted characteristics to provide a confidence score each trajectory associated with one of the trajectory end points.
For the following reason(s), the examiner submits that the above identified additional limitations do not integrate the above-noted abstract idea into a practical application.
Regarding the additional limitations of “computer” the examiner submits that these limitations are an attempt to generally link additional elements to a technological environment. In particular, the receive, determine, extract, evaluate by a computer processor is recited at a high level of generality and merely automates the determining steps, therefore acting as a generic computer to perform the abstract idea. The computer is claimed generically and is operating in its ordinary capacity and does not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the exception. The additional limitation is no more than mere instructions to apply the exception using a computer processor.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception (MPEP § 2106.05). Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
101 Analysis – Step 2B
Regarding Step 2B of the Revised Guidance, representative independent claim 1 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “computer” amounts to nothing more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Hence, the claim is not patent eligible.
Dependent claims 2-9, 11-12 and 14 do not recite any further limitations that cause the claim(s) to be patent eligible. Rather, the limitations of dependent claims are directed toward additional aspects of the judicial exception and/or well-understood, routine and conventional additional elements that do not integrate the judicial exception into a practical application. Therefore, dependent claims 2-9, 11-12 and 14 are not patent eligible under the same rationale as provided for in the rejection of Claims 1, 10, 13 and 15.
Therefore, claims 2-9, 11-12 and 14 are ineligible under 35 USC §101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 8, 10 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta et al. (“Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks,” CVPR 2018, arXiv:1803.10892, hereinafter “Gupta”) in view of Lee et al. (“DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents,” CVPR 2017, hereinafter “Lee.
Regarding claims 1, 10 and 15, Gupta discloses A computer implemented method for determining and evaluating a trajectory of a road user, the method comprising:
receiving input data associated with a state of movement and with an environment of the road user,(“The input trajectory of a person i is defined as X_i = (x_t^i, y_t^i) from time steps t = 1, …, t_obs and the future trajectory (ground truth) can be defined similarly as Y_i = (x_t^i, y_t^i) from time steps t = t_obs + 1, …, t_pred.” - Gupta, Section 3.1);
determining one or more trajectory end points for the road user by using the extracted characteristics (“The generator G takes as input X_i and outputs predicted trajectory Ŷ_i.” - Gupta, Section 3.3; note: predictions include end points as part of full trajectory);
determining, for each of the trajectory end points, a respective trajectory associated with one of the trajectory end points by using the associated trajectory end point and the extracted characteristics (“After initializing the decoder states as described above we can obtain predictions as follows: […] (x̂_t^i, ŷ_t^i) = γ(h_t^{di})” - Gupta, Section 3.3);
evaluating the respective trajectory of the road user by using a classification which relies on the extracted characteristics to provide a confidence score for each trajectory associated with one of the trajectory end points (“The discriminator consists of a separate encoder. Specifically, it takes as input T_real = [X_i, Y_i] or T_fake = [X_i, Ŷ_i] and classifies them as real/fake. We apply a MLP on the encoder’s last hidden state to obtain a classification score.” - Gupta, Section 3.3).
Gupta does not explicitly teach extracting characteristics related to the road user from the input data
determining one or more trajectory end points for the road user by using the extracted characteristics, including sampling latent variables.
However, Lee does teach extracting characteristics related to the road user from the input data (“Note that we directly use raw images to extract visual features, rather than semantically labeled feature maps.” - Lee, Sec. 4.1) ,
determining one or more trajectory end points for the road user by using the extracted characteristics, including sampling latent variables (“The distribution of z_i is modeled as a Gaussian distribution (i.e., z_i ∼ Q_φ(z_i | X_i, Y_i) = N(μ_{z_i}, σ_{z_i})) and is regularized by the KL divergence against a prior distribution P_ν(z_i) := N(0, I) during the training.” - Lee, Sec. 3.1; “At test time z_i is sampled randomly from the prior distribution and decoded through the decoder network to produce a prediction hypothesis.” - Lee, Sec. 3.1). Both Gupta and Lee teach methods for determining and evaluating a trajectory of a road user. However, Lee explicitly teaches extracting characteristics related to the road user from the input data and determining one or more trajectory end points for the road user by using the extracted characteristics, including sampling latent variables.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include extracting characteristics related to the road user from the input data and determining one or more trajectory end points for the road user by using the extracted characteristics, including sampling latent variables, as taught by Lee, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [Lee, 3.1, 4.1]).
Regarding claim 2, Gupta discloses The method according to claim 1, wherein the method relies on at least one machine learning algorithm, the input data includes embedded features which are provided (“We first embed the location of each person using a single layer MLP to get a fixed length vector e_t^i.” - Gupta, Section 3.3; “Our proposed GAN is a RNN Encoder-Decoder generator and a RNN based encoder discriminator” - Gupta, Abstract).
by a prediction algorithm and which include information regarding other road users and regarding a static environment of the road user, (Our goal is to jointly reason and predict the future trajectories of all the agents involved in a scene. We assume that we receive as input all the trajectories for people in a scene as X = X1, X2, ..., Xn and predict the future trajectories Yˆ = Yˆ 1, Yˆ 2, ..., Yˆ n of all the people simultaneously.” -Gupta, 3.1).
Gupta does not explicitly teach a feature extraction algorithm is applied to the embedded features in order to provide the extracted characteristics.
However, Lee does teach a feature extraction algorithm is applied to the embedded features in order to provide the extracted characteristics(“Note that we directly use raw images to extract visual features, rather than semantically labeled feature maps.” - Lee, Sec. 4.1). Both Gupta and Lee teach methods for determining and evaluating a trajectory of a road user. However, Lee explicitly teaches a feature extraction algorithm is applied to the embedded features in order to provide the extracted characteristics.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include a feature extraction algorithm is applied to the embedded features in order to provide the extracted characteristics, as taught by Lee, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [Lee, 3.1, 4.1]).
Regarding claim 3, Gupta discloses The method according to claim 1,
Gupta does not explicitly teach wherein; determining one or more trajectory end points for the road user includes applying a machine learning algorithm configured to learn encoding the extracted characteristics and ground truth data of real trajectories into at least one latent variable
However, Lee does teach wherein; determining one or more trajectory end points for the road user includes applying a machine learning algorithm configured to learn encoding the extracted characteristics and ground truth data of real trajectories into at least one latent variable (“The CVAE introduces a latent variable to account for the ambiguity of the future, which is combined with a recurrent neural network (RNN) encoding of past trajectories, to generate hypotheses using another RNN.” - Lee, Sec. 3.1) and (“The distribution of z_i is modeled as a Gaussian distribution (i.e., z_i ∼ Q_φ(z_i | X_i, Y_i) = N(μ_{z_i}, σ_{z_i})) and is regularized by the KL divergence against a prior distribution P_ν(z_i) := N(0, I) during the training.” - Lee, Sec. 3.1; “At test time z_i is sampled randomly from the prior distribution and decoded through the decoder network to produce a prediction hypothesis.” - Lee, Sec. 3.1). Both Gupta and Lee teach methods for determining and evaluating a trajectory of a road user. However, Lee explicitly teaches determining one or more trajectory end points for the road user includes applying a machine learning algorithm configured to learn encoding the extracted characteristics and ground truth data of real trajectories into at least one latent variable.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include determining one or more trajectory end points for the road user includes applying a machine learning algorithm configured to learn encoding the extracted characteristics and ground truth data of real trajectories into at least one latent variable, as taught by Lee, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [Lee, 3.1, 4.1]).
Regarding claim 4, Gupta discloses The method according to claim 3,
Gupta does not explicitly teach wherein; determining one or more trajectory end points for the road user by using the extracted characteristics further includes: sampling the at least one latent variable from a prior distribution provided by the machine learning algorithm when learning to encode the extracted features and the ground truth data of real trajectories, and decoding the at least one latent variable into one or more trajectory end points.
However, Lee does teach wherein; determining one or more trajectory end points for the road user by using the extracted characteristics further includes: sampling the at least one latent variable from a prior distribution provided by the machine learning algorithm when learning to encode the extracted features and the ground truth data of real trajectories, and decoding the at least one latent variable into one or more trajectory end points (“At test time, the encodings of future trajectories H_Y_i are not available, thus the encodings of past trajectories H_X_i are combined with multiple random samples of latent variable z^(k)_i drawn from the prior z^(k)_i ∼ P_ν(z_i).” - Lee, Sec. 3.1; “Finally, the following RNN decoder (i.e., RNN Decoder1 in Fig. 2) takes the output of the previous step, H_X_i ⊠ β(z^(k)_i), and generates K number of future prediction samples, i.e., ˆY_i^(1), ˆY_i^(2), .., ˆY_i^(K).” - Lee, Sec. 3.1). Both Gupta and Lee teach methods for determining and evaluating a trajectory of a road user. However, Lee explicitly teaches determining one or more trajectory end points for the road user by using the extracted characteristics further includes: sampling the at least one latent variable from a prior distribution provided by the machine learning algorithm when learning to encode the extracted features and the ground truth data of real trajectories, and decoding the at least one latent variable into one or more trajectory end points.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include determining one or more trajectory end points for the road user by using the extracted characteristics further includes: sampling the at least one latent variable from a prior distribution provided by the machine learning algorithm when learning to encode the extracted features and the ground truth data of real trajectories, and decoding the at least one latent variable into one or more trajectory end points, as taught by Lee, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [Lee, 3.1, 4.1]).
Regarding claim 5, Gupta discloses The method according to claim 2,
Gupta does not explicitly teach wherein: determining the respective trajectory of the road user includes determining positions of the road user starting from the current position of the road user up to the associated trajectory end point and determining dynamic parameters of the road user for each of the positions.
However, Lee does teach wherein: determining the respective trajectory of the road user includes determining positions of the road user starting from the current position of the road user up to the associated trajectory end point and determining dynamic parameters of the road user for each of the positions (“The regression vector for each agent i is obtained with the regression function η defined as follows, △Yˆ (k) i = η(Yˆ (k) i ; I, X, Yˆ (∀) j\i ). (2) Represented as parameters of a neural network, the regression function η accumulates both scene contexts and all other agents dynamics from the past to entire future frames, and estimates the best displacement vector △Yˆ (k) i over entire time-horizon T. Similarly to the score s, it accounts for what happens in the future both in terms of scene context and interactions among dynamic agents to produce the output. “ -Lee 3.2). Both Gupta and Lee teach methods for determining and evaluating a trajectory of a road user. However, Lee explicitly teaches determining the respective trajectory of the road user includes determining positions of the road user starting from the current position of the road user up to the associated trajectory end point and determining dynamic parameters of the road user for each of the positions.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include determining the respective trajectory of the road user includes determining positions of the road user starting from the current position of the road user up to the associated trajectory end point and determining dynamic parameters of the road user for each of the positions, as taught by Lee, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [Lee, 3.2, 4.1]).
Regarding claim 8, Gupta discloses The method according to any one of claim 2, wherein; the machine learning algorithm includes a generative adversarial network which is utilized to provide the classification for evaluating the respective trajectory of the road user (“Our proposed GAN is a RNN Encoder-Decoder generator and a RNN based encoder discriminator […] We predict socially plausible futures by training adversarially against a recurrent discriminator, and encourage diverse predictions with a novel variety loss.” - Gupta, Abstract).
Claims 6 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta in view of Lee in further in view of Saxena et al. (US 11,465,650 B2) (hereinafter “SAXENA”).
Regarding claim 6, Gupta discloses The method according to claim 5,
Gupta does not explicitly teach wherein; determining the respective trajectory of the road user includes applying a machine learning algorithm which is configured to learn parameters of a bicycle model which are associated with the positions of the road user along the trajectory.
However, SAXENA does teach wherein; determining the respective trajectory of the road user includes applying a machine learning algorithm which is configured to learn parameters of a bicycle model which are associated with the positions of the road user along the trajectory (“Each kinematic bicycle model for each corresponding vehicle or traffic participant may be associated with a set of spatial coordinates, a heading, a velocity, a local frame angle of velocity vector, an angle of tires, and an acceleration.” - SAXENA, Col.2 ln 46-59) and (“The nonlinear equations of motion for this model may be written as: [{dot over (x)} = \nu \cos(\phi + \beta)] [{dot over (y)} = \nu \sin(\phi + \beta)] [\phi . = \frac{v}{l_r} \sin(\beta)] [v . = a] [\beta = \arctan\left(\frac{l_r}{l_f + l_r} \tan(\delta_f)\right)]” - SAXENA, Col.8 ln 37-58). Both Gupta and SAXENA teach methods for determining and evaluating a trajectory of a road user. However, SAXENA explicitly teaches determining the respective trajectory of the road user includes applying a machine learning algorithm which is configured to learn parameters of a bicycle model which are associated with the positions of the road user along the trajectory.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include determining the respective trajectory of the road user includes applying a machine learning algorithm which is configured to learn parameters of a bicycle model which are associated with the positions of the road user along the trajectory, as taught by SAXENA, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [SAXENA, Col.8 ln 37-58]).
Regarding claim 13, Gupta discloses a computer system ,
Gupta does not explicitly teach A vehicle including a perception system
However, SAXENA does teach A vehicle including a perception system and the computer system of claim 10 (“The policy may be implemented on an autonomous vehicle.” - SAXENA, Col.1 ln 38-41). Both Gupta and SAXENA teach methods for determining and evaluating a trajectory of a road user. However, SAXENA explicitly teaches A vehicle including a perception system.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include A vehicle including a perception system, as taught by SAXENA, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [SAXENA, Col.1 ln 38-41]).
Claims 7, 9, 12 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Gupta in view of Lee in further in view of Song et al. (US 11,392,132 B2) (hereinafter “SONG”).
Regarding claim 7, Gupta discloses The method according to claim 2, wherein; the machine learning algorithm is trained regarding the step of evaluating of the trajectory by: providing ground truth trajectories and predicted trajectories for a plurality of road users (“The discriminator will ideally learn subtle social interaction rules and classify trajectories which are not socially acceptable as ‘fake’.” - Gupta, Section 3.30) and (“Current approaches have focused on predicting the average future trajectory which minimizes the L2 distance from the ground truth future trajectory whereas we want to predict multiple “good” trajectories”- Gupta, Section 3),
and learning the classification of the road user by applying a generative adversarial network to the ground truth trajectories and the predicted trajectories (“A Generative Adversarial Network (GAN) consists of two neural networks trained in opposition to each other [14]. The two adversarially trained models are: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The generator G takes a latent variable z as input, and outputs sample G(z). The discriminator D takes a sample x as input and outputs D(x) which represents the probability that it is real.” - Gupta, Section 3.2) and (“In addition to adversarial loss, we also apply L2 loss on the predicted trajectory which measures how far the generated samples are from the actual ground truth.” - Gupta, Section 3.3).
Gupta does not explicitly teach wherein the predicted trajectories are determined by applying extracted characteristics derived from training input data associated with a respective state of movement and a respective environment of each of the plurality of road users
However, SONG does teach wherein the predicted trajectories are determined by applying extracted characteristics derived from training input data associated with a respective state of movement and a respective environment of each of the plurality of road users (“providing the generated simulated time-dependent 3D traffic environmental data for creating time-dependent 3D GAN discriminator training data to be used by the time-dependent 3D GAN discriminator sub-model” - SONG, [Col.21 ln 40-54]). Both Gupta and SONG teach methods for determining and evaluating a trajectory of a road user. However, SONG explicitly teaches wherein the predicted trajectories are determined by applying extracted characteristics derived from training input data associated with a respective state of movement and a respective environment of each of the plurality of road users.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include wherein the predicted trajectories are determined by applying extracted characteristics derived from training input data associated with a respective state of movement and a respective environment of each of the plurality of road users, as taught by SONG, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [SONG, Col.21 ln 40-54]).
Regarding claim 9, Gupta discloses The method according to any one of claim 2,
Gupta does not explicitly teach wherein; a common training procedure of the machine learning algorithm is performed for the steps of determining and evaluating the respective trajectory of the road user
However, SONG does teach wherein; a common training procedure of the machine learning algorithm is performed for the steps of determining and evaluating the respective trajectory of the road user (“creating the time-dependent 3D generative adversarial network (GAN) model through an adversarial machine learning process of a time-dependent 3D GAN discriminator sub-model and a time-dependent 3D GAN generator sub-model of the time-dependent 3D GAN model.” - SONG, Col.1 ln 61-67). Both Gupta and SONG teach methods for determining and evaluating a trajectory of a road user. However, SONG explicitly teaches a common training procedure of the machine learning algorithm is performed for the steps of determining and evaluating the respective trajectory of the road user.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include a common training procedure of the machine learning algorithm is performed for the steps of determining and evaluating the respective trajectory of the road user, as taught by SONG, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [SONG, Col.1 ln 61-67]).
Regarding claim 12, Gupta discloses The computer system according to claim 11,
Gupta does not explicitly teach wherein; the end point generator, the trajectory generator and the discriminator are implemented as a generative adversarial network, the discriminator is configured to evaluate each trajectory by using a classification provided by the generative adversarial network.
However, SONG does teach wherein; the end point generator, the trajectory generator and the discriminator are implemented as a generative adversarial network, the discriminator is configured to evaluate each trajectory by using a classification provided by the generative adversarial network (“performing, using the time-dependent 3D GAN discriminator sub-model, discrimination analysis of the received time-dependent 3D GAN discriminator training data to generate a discrimination result indicating whether the time-dependent 3D GAN discriminator sub-model determined that the time-dependent 3D GAN discriminator training data represents real-world time-dependent 3D traffic environmental data or simulated time-dependent 3D traffic environmental data” - SONG, Col.21 ln 20-40). Both Gupta and SONG teach methods for determining and evaluating a trajectory of a road user. However, SONG explicitly teaches the end point generator, the trajectory generator and the discriminator are implemented as a generative adversarial network, the discriminator is configured to evaluate each trajectory by using a classification provided by the generative adversarial network.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include the end point generator, the trajectory generator and the discriminator are implemented as a generative adversarial network, the discriminator is configured to evaluate each trajectory by using a classification provided by the generative adversarial network, as taught by SONG, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [SONG, Col.21 ln 20-40]).
Regarding claim 14, Gupta discloses The vehicle according to claim 13,
Gupta does not explicitly teach further including a control system being configured to control the actual trajectory of the vehicle, wherein computer system is configured to transfer the at least one evaluated trajectory of the road user to the control system in order to enable the control system to apply the evaluated trajectory of the road user when controlling the actual trajectory of the vehicle
However, SONG does teach further including a control system being configured to control the actual trajectory of the vehicle (“carrying out, using the computer-based autonomous driving model, a virtual autonomous driving operation in the generated virtual photorealistic time-dependent 3D traffic environment” - SONG, Col.22 ln 9-12).
wherein computer system is configured to transfer the at least one evaluated trajectory of the road user to the control system in order to enable the control system to apply the evaluated trajectory of the road user when controlling the actual trajectory of the vehicle (“Further depending on a specific implementation, the certain route may be determined based on a selected one of driving modes, which may include an economic mode (e.g., lowest cost), a fastest mode, and so on. Moreover, the certain route may be dynamically changes as the virtual autonomous driving operation proceeds. In some embodiments, during the virtual autonomous driving operation, a virtual vehicle that performs the virtual autonomous driving operation is controlled according to the computer-based autonomous driving model so as to be safe to passengers and people and animals outside the virtual vehicle.” - SONG, Col.12 ln 2-13). Both Gupta and SONG teach methods for determining and evaluating a trajectory of a road user. However, SONG explicitly teaches a control system being configured to control the actual trajectory of the vehicle, wherein computer system is configured to transfer the at least one evaluated trajectory of the road user to the control system in order to enable the control system to apply the evaluated trajectory of the road user when controlling the actual trajectory of the vehicle.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include a control system being configured to control the actual trajectory of the vehicle, wherein computer system is configured to transfer the at least one evaluated trajectory of the road user to the control system in order to enable the control system to apply the evaluated trajectory of the road user when controlling the actual trajectory of the vehicle, as taught by SONG, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [SONG, Col.12 ln 2-13, Col.22 ln 9-12]).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Gupta in view of Lee in further in view of Jeon et al. (US 2022/0326042 A1) (hereinafter “JEON”).
Regarding claim 11, Gupta discloses The computer system according to claim 10, comprising:
a trajectory generator configured to determine, for each of the trajectory end points, a respective trajectory associated with one of the trajectory end points (“we propose to use them to generate multiple socially-acceptable trajectories given an observed past. One network (the generator) generates candidates and the other (the discriminator) evaluates them. The adversarial loss enables our forecasting model to go beyond the limitation of L2 loss and potentially learn the distribution of “good behaviors” that can fool the discriminator. In our work, these behaviors are referred to as socially-accepted motion trajectories in crowded scenes.” - Gupta, Section 3.3),
and a discriminator configured to evaluate each trajectory associated with one of the trajectory end points(“The discriminator consists of a separate encoder. Specifically, it takes as input T_real = [X_i, Y_i] or T_fake = [X_i, Ŷ_i] and classifies them as real/fake. We apply a MLP on the encoder’s last hidden state to obtain a classification score.” - Gupta, Section 3.3).
Gupta does not explicitly teach an end point generator configured to determine one or more trajectory end points
However, JEON does teach an end point generator configured to determine one or more trajectory end points (“The waypoint learning unit predicts a plurality of waypoints, and adds the plurality of waypoints predicted in time series to provide a predicted destination.” - JEON, Claim 6). Both Gupta and JEON teach methods for determining and evaluating a trajectory of a road user. However, JEON explicitly teaches an end point generator configured to determine one or more trajectory end points.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the road user trajectory determining and evaluating method of Gupta to also include an end point generator configured to determine one or more trajectory end points, as taught by JEON, with a reasonable expectation of success. Doing so improves methods for determining and evaluating a trajectory of a road users (With regard to this reasoning, see at least [JEON, Claim 6]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED ALKIRSH whose telephone number is (703) 756-4503. The examiner can normally be reached M-F 9:00 am-5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, FADEY JABR can be reached on (571) 272-1516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AA/Examiner, Art Unit 3668
/Fadey S. Jabr/Supervisory Patent Examiner, Art Unit 3668
Prosecution Insights