DETAILED ACTION
This Office action is drafted in response to amendments and arguments dated 05/11/2026. Claims 1-3, and 5-15 are now pending. Claim 4 is cancelled by the applicant, claims 5-14 are withdrawn, and claims 1-3 and 15 are rejected as cited below. This action is made FINAL.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Specification Objection
Examiner withdraws the title objection in view of Applicant’s amendments.
Response to Claim Objections
Examiner withdraws the objection to claim 15 in view of Applicant’s amendments.
Response to Claim Rejections - 35 USC § 101
Examiner withdraws the 35 USC 101 rejection of claims 1-4 and 15 in view of Applicant’s amendments. Examiner finds that controlling the host vehicle to follow a trajectory based on at least one road user trajectory incorporates the judicial exception into the practical application of driving the vehicle autonomously.
Response to Claim Rejections - 35 USC § 112
Examiner withdraws the 35 USC 112 rejection of claims 1-4 and 15 in view of Applicant’s amendments.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-4 and 15 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Pronovost (US Pub. 2024/0101157 A1; hereafter Pronovost) in view of ARNDT et al. (US Pub. 2016/0012301 A1; hereafter ARNDT).
Pronovost was cited in the NF Office action dated 02/10/2026.
Regarding claim 1, Pronovost teaches:
A computer implemented method for predicting respective trajectories of a plurality of road users, the method comprising:
transforming, for each of the road users, the set of characteristics (input data 302) to a respective set of input data (token sequence 316) for a prediction algorithm (Decoder 318) via a processing unit of the host vehicle, wherein each set of input data comprises the same predefined number of data elements that represent the set of input data (At least ¶ [0062] “The encoder 304 can represent a machine learned model such as a GNN, RNN, CNN, and the like, and output one or more feature vectors 306 which can be sent to a codebook 308 and a quantizer 310.” And ¶ [0063] “the quantizer 310 can receive the feature vectors 306 output by the encoder 304, and discretize the feature vectors 306 to output the discretized feature vectors 312 “ and ¶ [0064] “the codebook 308 can receive the discretized feature vectors 312 and determine the token sequence 316.” And ¶ [0067] “a decoder 318 can receive the token sequence 316 and determine the output data 320.” In this case, the token sequence 316, which is derived from the input data 302, is used itself as input data for the decoder 318 (i.e. prediction algorithm).);
determining, via the processing unit, at least one respective trajectory for each of the road users by applying the prediction algorithm to the input data (At least ¶ [0067] “a decoder 318 can receive the token sequence 316 and determine the output data 320. The decoder 318 can represent a machine learned model such as a GNN, a GAN, an RNN, another Transformer model, etc. The output data 320 can, for example, be similar to the output data 106 or the output data 210 and represent an object trajectory, scene data, simulation data, and so on.”); and
controlling, with a control system (system controllers 826), an actual trajectory of the host vehicle based on the at least one respective trajectory determined for each of the road users (At least ¶ [0147] “At operation 912, the process may include causing the vehicle to be controlled in a real-world environment based at least in part on the object trajectory.”).
Pronovost does not teach:
determining, for each road user, a respective set of characteristics detected by a perception system of a host vehicle, the set of characteristics including specific characteristics associated with a predefined class of road users …
wherein a number and a type of the specific characteristics is different for different classes of road users.
However, ARNDT, within the same field of endeavor, teaches:
determining, for each road user (non-motorized road users 3), a respective set of characteristics (At least ¶ [0055] “individual features …”) detected by a perception system (one or more cameras 1) of a host vehicle (motor vehicle 2), the set of characteristics including specific characteristics associated with a predefined class of road users (At least ¶ [0062] “Colors and color patterns such as those characteristic of the uniforms of traffic policemen, firemen, road workers, etc., can also be taken into account for the identification and interpretation …” Specific colors and color patterns (i.e. specific characteristics) are associated with traffic policemen (i.e. predefined class of road user). and at least ¶ [0063] “Since specific features are often associated with one another, e.g. a traffic policeman wears typical clothing and head covering or a cyclist often wears a helmet …”.) …
wherein a number and a type of the specific characteristics is different for different classes of road users (At least ¶ [0063] “Since specific features are often associated with one another, e.g. a traffic policeman wears typical clothing and head covering or a cyclist often wears a helmet …”. Typical policeman head coverings and helmets are types of head equipment (i.e. specific characteristics) which are different for different classes of road users (e.g. Head Equipment: cyclist wears a helmet; policeman wears a typical head covering). Additionally, see FIG. 7 , which shows “Pedestrian on the road …” (thin dashed line) having 8 different specific characteristics, while the “Pedestrian on the sidewalk …” (thick dashed line) only has 6. This is analogous to a number of the specific characteristics being different for different classes of road users.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Pronovost with ARNDT. This modification would have been obvious as both Pronovost and ARNDT contain subject matter within the same field of endeavor (vehicle environment recognition) and Pronovost ¶ [0002] notes that “…Accurately predicting future object trajectories may be necessary to safely operate the vehicle in the vicinity of the object...”. Introducing ARNDT to Pronovost may help accomplish the goal of more accurately predicting future object trajectories. ARNDT ¶ [0007] recites “for example, a cyclist can indicate an intention to turn off with a hand signal so that following or oncoming motor vehicle drivers can react accordingly.” One of ordinary skill in the art would recognize that a vehicle system which is able to recognize cyclist hand signals would lead to an improvement in the accuracy of the Pronovost system, thus increasing safety of the vehicle occupants and those cyclists proximate to the vehicle.
Claim 15 recites a non-transitory computer-readable medium comprising instructions which perform the method detailed in claim 1, thus is rejected on the same basis. Additionally, Pronovost teaches a non-transitory computer-readable medium (At least ¶ [0139] “Memory 818 and memory 838 are examples of non-transitory computer-readable media.”).
Claims 2-4 are rejected under 35 U.S.C. 103 as being unpatentable over Pronovost in view of ARNDT, in further view of H. Bi (“Joint Prediction for Kinematic Trajectories in Vehicle-Pedestrian-Mixed Scenes”, hereafter Bi).
Bi was cited in the IDS dated 05/15/2024.
Regarding claim 2:
The combination of Pronovost and ARNDT teaches the method according to claim 1.
The combination of Pronovost and ARNDT does not teach:
each set of input data includes a set of latent features;
transforming the respective set of characteristics for each of the road users to the respective set of input data comprises applying an embedding algorithm to the respective set of characteristics in order to generate the corresponding set of latent features;
the latent features are allocated to a dynamic grid map of a predefined region of interest in the environment of the host vehicle, the dynamic grid map including a predefined number of pixels;
the prediction algorithm is applied to the allocated latent features for determining respective occupancy information for each class of the road users for each pixel of the grid map; and
the at least one respective trajectory for each of the road users is determined by using the respective occupancy information for each pixel.
However, Bi, within the same filed of endeavor, teaches:
each set of input data includes a set of latent features (At least section 3.2 “We adopt
this pooling scheme in our network to collect the latent motion representations of vehicles and pedestrians in the neighborhood.”);
transforming the respective set of characteristics for each of the road users to the respective set of input data comprises applying an embedding algorithm to the respective set of characteristics in order to generate the corresponding set of latent features (At least section 3.2 “For any pedestrian p 1 and any vehicle vj, we first use separate embedding functions ¢ ( ·) with Re LU nonlinearity to embed x!, P{ as follows:) …”);
the latent features are allocated to a dynamic grid map of a predefined region of interest in the environment of the host vehicle (At least section 3.2 “The positions of all the neighbors, including pedestrians and vehicles, are pooled on the occupancy map.”), the dynamic grid map including a predefined number of pixels (At least section 3.2 “We use a similar grid of N-0 x N0 cells in [2], called occupancy map, which is centered at the position of a pedestrian or vehicle.” A grid of N0 X N0 is a predefined size.);
the prediction algorithm is applied to the allocated latent features for determining respective occupancy information for each class of the road users for each pixel of the grid map (At least section 3.2 “The hidden states of pi and vj, denoted as ht (p,i) and ht (v,j) respectively, carry their latent representations. Through the occupancy map, pedestrians and vehicles share the latent representations with hidden states. As shown in Fig. 3, the occupancy map VO and PO are built respectively for both vehicles and pedestrians.”. Building an occupancy map is analogous to determining respective occupancy information.); and
the at least one respective trajectory for each of the road users is determined by using the respective occupancy information for each pixel (At least section 3.4 “Through the occupancy maps respectively for pedestrians and vehicles, frame-by-frame heterogeneous interactions are pooled. The predicted kinematic trajectories of pedestrians and vehicles at t are respectively given …”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Pronovost and ARNDT with Bi. This modification would have been obvious as both the Pronovost/ARNDT combination and Bi contain subject matter within the same field of endeavor (vehicle environment recognition) and Pronovost ¶ [0002] notes that “…Accurately predicting future object trajectories may be necessary to safely operate the vehicle in the vicinity of the object...”. Introducing Bi to the Pronovost/ARNDT combination may help accomplish the goal of more accurately predicting future object trajectories. Bi notes that predicting the orientation together with the position simultaneously will produce more accurate kinematic trajectories. One of ordinary skill in the art would recognize that the method described by Bi would help create a more accurate dataset. This increased accuracy may lead to increased safety for those proximate to the host vehicle.
Regarding claim 3, the combination of Pronovost, ARNDT and Bi teaches The method according to claim 2. Bi further teaches wherein transforming the set of characteristics to the respective set of input data is performed separately for each class of road users by applying a separated embedding algorithm being defined for the respective class to the respective set of characteristics (At least section 3.2 “For any pedestrian p 1 and any vehicle vj, we first use separate embedding functions ¢ ( ·) with Re LU nonlinearity to embed x!, P{ as follows:) …”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jonathan E Reinert whose telephone number is (571)272-1260. The examiner can normally be reached Mon - Thurs 7AM - 5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James J Lee can be reached at (571) 270-5965. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.E.R./Examiner, Art Unit 3668
/BRIAN P SWEENEY/Primary Examiner, Art Unit 3668
Prosecution Insights