DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/22/20226 has been entered.
Claims 1-20 remain pending in this application. Claims 1, 19, and 20 have been amended.
Claim Objections
Claim 19 is objected to because of the following informalities:
On line 1, replace … vehicle; comprising … with … vehicle comprising …
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 8, 11, 12, 15, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al., US PGPUB 2019/0367021 Al (hereinafter as Zhao) in view of Shin et al., US PGPUB 2016/0297436 Al (hereinafter as Shin).
Regarding independent claim 1, Zhao teaches a method [see [0002]] for generating a control signal for controlling motion of a vehicle [see e.g. [0045] and [0074]; see also [0052]], the method comprising:
obtaining a representation of a trajectory of a single object in the same environment as the vehicle [again, see [0045] and note sensing other objects such as users of the vehicle’s transportation network such as dynamic objects or road users; note e.g. from [0257] receiving a predicted path of an object; see also [0047] indicating receiving anticipated trajectories of other users of the vehicle’s transportation network; see also [0276]];
updating the representation of the single-object trajectory based on data received by one or more sensors of the vehicle within a framework of multi-object and multi-hypothesis tracker [see e.g. [0309] indicating the updating of the predictions for the objects based on sensor observations; note from [0045] and [0047] the possible presence of multiple objects; note from [0199] and [0252] the multi-hypothesis tracking; see also [0052]];
generating the control signal for controlling motion of the vehicle based on the updated trajectory of the object [again, see e.g. [0045] indicating controlling the vehicle including stopping, accelerating, merging, etc. and [0074] indicating generating a signal to control the vehicle to follow a certain trajectory; see also [0052]]; and
controlling motion of the vehicle based on the generated control signal, wherein controlling motion of the vehicle comprises at least one of maintaining speed, accelerating, decelerating, or changing direction [note in [0045] controlling the vehicle in the form of accelerating and decelerating as exemplary aspects of control].
Zhao teaches the use of past trajectory data of an object for trajectory updating [see e.g. [0276] and process 2000 in fig. 20 indicating the use of previous observations for predicting a trajectory and future states as in [0289]].
However, Zhao does not explicitly teach that the trajectory representation is a parametric representation. Neither does it teach that the obtaining of the parametric representation is using the object’s past trajectory over a number of time steps as a single state of the object, such that the parametric representation comprises a fixed number of parameters, independent of the number of time steps and/or a length of the object's past trajectory.
Shin teaches a parametric representation of a trajectory of an object [see e.g. [0027] indicating a trajectory of a vehicle that is represented as a polynomial of a third degree].
Shin further teaches using the object’s past trajectory over a number of time steps as a single state of the object, such that the parametric representation comprises a fixed number of parameters, independent of the number of time steps and/or a length of the object's past trajectory [see again, [0027] and note utilizing coordinate history of the vehicle and that the third-degree-polynomial has 4 coefficients; note that the number of coefficients is four, independent of the number of points used from the coordinate history or the distance covered by the past trajectory, as can be seen in exemplary points and paths in figs. 4-6].
It would have been obvious to one of ordinary skill in the art having the teachings of Zhao and Shin before the effective filing date of the claimed invention to modify the obtained and updated representations of trajectories taught by Zhao by explicitly specifying the use of a parametric representation, as per the teachings of Shin, and to further explicitly specify using the object’s past trajectory over a number of time steps as a single state of the object, such that the parametric representation comprises a fixed number of parameters, independent of the number of time steps and/or a length of the object's past trajectory, as per the combined teachings of Zhao and Shin. The motivation for this obvious combination would be to enable direct comparison of trajectory representations by comparing the fixed parameters, as suggested by Shin [see e.g. [0030]-[0031]].
Regarding claim 2, the rejection of claim 1 is incorporated. Zhao further teaches that the control signal is generated based on the updated trajectory of the object and at least one other object in the same environment as the vehicle [see e.g. fig. 9, especially views 950 and 960 indicating controlling vehicle 912 based on updated trajectory of dynamic object 918 and static object 942, as described in [0171]-[0173]].
Regarding claim 3, the rejection of claim 1 is incorporated. Zhao further teaches providing the control signal to the vehicle for controlling motion of the vehicle [again, see e.g. [0045] indicating controlling the motion of the vehicle (see e.g. stopping, acceleration, deceleration, merging) and [0074] indicating generating a signal to control the vehicle to follow a certain trajectory].
Regarding claim 4, the rejection of claim 1 is incorporated. Zhao further teaches determining an intent associated with the object based on the updated trajectory, wherein the control signal is determined based on the intent [see e.g. [0052] indicating determining intentions associated with road users based on the predicted trajectories; see also [0047] indicating determining intentions of the objects].
Regarding claim 5, the rejection of claim 4 is incorporated. Zhao further teaches that the intent comprises exiting a road, entering a road, changing lanes, crossing street, making a turn, or any combination thereof [again, see [0047] indicating an intent to turn right or left which also includes exiting/entering a road].
Regarding claim 8, the rejection of claim 1 is incorporated. Zhao further teaches that obtaining the representation of the trajectory comprises retrieving, from a memory, data related to the representation [see e.g. [0276] and note the data store 2014 which can be stored in a memory and keeps hypotheses and states related to dynamic objects in the vehicle’s environment; note in [0301] the reading of the hypotheses (which are associated with trajectories) from the data store].
Again, Shin teaches a parametric representation of trajectories [see e.g. [0027] indicating a trajectory of a vehicle that is represented as a polynomial of a third degree], wherein obtaining the parametric representation of the trajectory comprises retrieving a plurality of control points [see [0026]-[0037] and note retrieving coordinate history to obtain the parametric representation].
See the rejection of claim 1 for motivations to combine Zhao and Shin.
Regarding claim 11, the rejection of claim 1 is incorporated. Zhao further teaches that updating the representation comprises:
predicting an expected representation based on the obtained representation and a motion model [see e.g. [0302] describing the use of a motion model based on the classification included in state information (which is part of the representation of the object) to predict the trajectory; see also in [0346] the use of the motion model for predicting an expected representation];
comparing the expected representation with the data received by the one or more sensors of the vehicle [see e.g. in [0309] the comparison of the sensor observations to states of objects]; and
updating the representation based on the comparison [again, see [0309] and note the updating of the predicted representations based on the comparison].
Again, Shin teaches a parametric representation of trajectories [see e.g. [0027] indicating a trajectory of a vehicle that is represented as a polynomial of a third degree].
See the rejection of claim 1 for motivations to combine Zhao and Shin.
Regarding claim 12, the rejection of claim 11 is incorporated. Shin further teaches that predicting the expected parametric representation comprises determining a plurality of control points of the expected parametric representation [see [0026]-[0037] and note determining points of coordinate history to obtain/get the parametric representation].
See the rejection of claim 1 for motivations to combine Zhao and Shin.
Regarding claim 15, the rejection of claim 11 is incorporated. Zhao further teaches that the representation is updated based on a Kalman filter algorithm [see e.g. [0359]].
Again, Shin teaches a parametric representation of trajectories [see e.g. [0027] indicating a trajectory of a vehicle that is represented as a polynomial of a third degree].
See the rejection of claim 1 for motivations to combine Zhao and Shin.
Regarding claim 17, the rejection of claim 1 is incorporated. Zhao further teaches that the data is first data, and that the method further comprises:
updating the obtained representation of the trajectory based on second data received by the one or more sensors of the vehicle to obtain a second updated representation [see [0391] indicating updating hypotheses when new sensor measurements are available; notice also in [0406] having a second hypothesis for the object; note from [0276] maintaining hypotheses as new sensor observations are received]; and
storing the first updated representation and the second updated representation as hypotheses associated with the object [see e.g. [0276] describing storing one or more hypotheses].
Again, Shin teaches a parametric representation of trajectories [see e.g. [0027] indicating a trajectory of a vehicle that is represented as a polynomial of a third degree].
See the rejection of claim 1 for motivations to combine Zhao and Shin.
Regarding claim 18, the rejection of claim 1 is incorporated. Zhao further teaches that the object is a traffic participant [again, see e[0045] and note that the objects are road users or dynamic objects; see also [0042] indicating a pedestrian, remote vehicle, motorcycle, bicycle, etc.].
Independent claims 19 and 20 rejected analogous to the rejection of independent claim 1.
Regarding independent claim 19, Zhao also teaches a vehicle [see e.g. fig. 1 which is a vehicle based on [0060]] comprising: one or more processors [see fig. 1, 120]; a memory [see fig. 1, 122]; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors [see e.g. [0428]], the one or more programs including instructions for performing the operations of the method of claim 1 [See the full rejection of claim 1].
Regarding independent claim 20, Zhao also teaches a system [see [0002]] for generating a control signal for controlling a vehicle [see e.g. [0045] and [0074]; see also [0052]], the system comprising: one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors [see e.g. [0428]], the one or more programs including instructions for performing the operations of the method of claim 1 [See the full rejection of claim 1].
Claims 6 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Shin, as applied to claim 1 above, and further in view of Aragon, US PGPUB 2021/0213977 Al (hereinafter as Aragon).
Regarding claim 6, the rejection of claim 1 is incorporated. The previously combined art does not explicitly teach inputting the updated trajectory into a trained machine-learning model to obtain an output, wherein the control signal is determined based on the output of the trained machine-learning model.
Aragon teaches inputting a trajectory (of an object in the environment of a vehicle) into a trained machine-learning model to obtain an output, wherein a control signal (for controlling a vehicle) is determined based on the output of the trained machine-learning model [see e.g. fig. 8 and note the input, into a machine learning model, of a current state of a second vehicle in 802; note the output in 806 that includes a control action for a vehicle; note from [0056] that the current state may comprise a trajectory of the second vehicle; note from [0060] different control actions that can be communicated to the vehicle based on the machine learning output; note the training of the neural networks, e.g. in [0046]].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Aragon, before the effective filing date of the claimed invention, to apply the machine-learning model taught by Aragon to the method taught by Zhao that generates the control signal for controlling the vehicle based on the updated trajectory of the object. The motivation for this obvious combination would be to enable autonomous driving systems to emulate driving actions of a human driver based on determined intent of a human driver of a nearby vehicle, by training a neural network, as suggested by Aragon [see e.g. [0002]-[0003] and [0004]-[0007]].
Regarding claim 7, the rejection of claim 6 is incorporated. Aragon further teaches that the machine-learning model is a neural network [see e.g. [0007], [0017], and [0046]]. See the rejection of claim 6 for motivations to combine the cited art.
Claims 9, 14, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Shin, and further in view of Wankhede et al., US Patent No. 12,030,518 B2 (hereinafter as Wankhede).
Regarding claim 9, the rejection of claim 8 is incorporated.
The previously combined art does not explicitly teach transforming the obtained parametric representation to a new coordinate system based on movement of the vehicle.
Wankhede teaches transforming obtained parametric representation to a new coordinate system based on movement of a vehicle [see fig. 1 and note the transformation between frames based on motion; see col. 6, lines 43-57 and note the transformation between frames (or coordinate systems); see also fig. 3 and col. 9, lines 26-34].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Wankhede before the effective filing date of the claimed invention to specify transforming the obtained parametric representation to a new coordinate system based on movement of the vehicle, as per the teachings of Wankhede. The motivation for this obvious combination would be to enable direct predictions of transformed representations, which would simplify updating the trajectory in different frames as needed, as suggested by Wankhede [see col. 9, line 26- 41; see also figs. 1, 3, and 7].
Regarding claim 14, the rejection of claim 11 is incorporated. Zhao further teaches that the motion model is a model configured to shift the obtained representation forward by a time period [see e.g. [0330] and [0346] indicating a motion model shifting representations in time for generating predictions; see also [0302]].
The previously combined art does not explicitly teach that the motion model is a linear model.
Wankhede further teaches a motion model that is a linear model [note the transformation matrix in col. 9, lines 26-41 which is a linear transformation that can be used for transformations such as those described in col. 6, lines 43-57; see also figs. 1 and 2; see also col. 9, lines 62-66].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Wankhede before the effective filing date of the claimed invention to specify that the motion model for shifting the obtained representation forward by a time period is a linear model as that taught by Wankhede. The motivation for this obvious combination would be to enable direct predictions of transformed target waypoints, which would simplify updating the trajectory as time passes and thus the comparison to indicate a need for a newer path, as suggested by Wankhede [see col. 9, line 26- col. 10, line 31; see also figs. 3 and 7].
Regarding claim 16, the rejection of claim 11 is incorporated. Zhao further teaches determining whether the object behavior is abnormal based on the comparison [see e.g. [0344] indicating the determination that a car is aggressively driven].
The previously combined art does not explicitly teach that the abnormal behavior comprises a deviation of an observed trajectory of the object from the updated trajectory exceeding a predetermine threshold.
Wankhede further teaches an abnormal behavior that comprises a deviation of an observed trajectory of an object from an updated trajectory exceeding a predetermine threshold [note example 704 shown in fig. 7 and described in col. 13, lines 43-66 where the vehicle diverged from the global path 708 (trajectory) more than a certain extent resulting is a comparison result exceeding a certain threshold difference].
Again, it would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Wankhede before the effective filing date of the claimed invention to specify that the abnormal behavior comprises a deviation of an observed trajectory of the object from the updated trajectory exceeding a predetermine threshold, as that taught by Wankhede. The motivation for this obvious combination would be to enable generating an alert to perform replanning and the creation of a new global path 714 for the AD system instead of a remainder of an older path, as suggested by Wankhede [see col. 13, lines 60-67] which would enhance the adaptability of the vehicle control, as unexpected incidents related to object behavior arise, which would in turn increase the reliability of the method taught by Zhao.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Shin and Wankhede, as applied to claim 9 above, and further in view of “Choi, Ji-Wung, and Kalevi Huhtala. "Constrained path optimization with Bézier curve primitives." 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2014” (hereinafter as Choi).
Regarding claim 10, the rejection of claim 9 is incorporated. The previously combined art does not explicitly teach that transforming the obtained representation comprises transforming the plurality of control points of the parametric representation to the new coordinate system using an affine transformation.
Choi teaches transforming a certain parametric representation[see abstract] comprises transforming a plurality of control points of the parametric representation [see the 2 lines before equation (1) on p. 247 indicating that a Bezier curve of degree n is defined by n+1 control points] to a new coordinate system using an affine transformation [see the last 4 lines above fig. 6 on p. 249 indicating that affine transformations can be applied to the curve representations by applying the respective transform on the control points of the curves].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Choi, before the effective filing date of the claimed invention to explicitly specify that that transforming the obtained representation comprises transforming the plurality of control points of the parametric representation to the new coordinate system using an affine transformation, as per the teachings of Choi. The motivation for this obvious combination would be to improve the computational efficiency of trajectory planning, as suggested by Choi [see e.g. lines 9-10 under the Introduction Section on p. 246].
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Shin, as applied to claim 12 above, and further in view of Hong et al., US PGPUB 2022/0092983 Al (hereinafter as Hong).
Regarding claim 13, the rejection of claim 12 is incorporated. The previously combined art does not explicitly teach that determining the plurality of control points of the expected parametric representation comprises obtaining a mean and/or a covariance of the plurality of control points of the expected parametric representation.
Hong teaches that determining a plurality of control points of an expected parametric representation comprises obtaining a mean and/or a covariance of the plurality of control points of the expected parametric representation [again, see in [0022] the determination of x-y-points associated with future locations of the object of interest; note that the points are used to calculate the expected trajectory; note in [0024] the predicted control points used to fit a Bezier curve or a polynomial curve (which is a predicted parametric representation of the trajectory); see also steps 304 and 320 of the method in fig. 3; see also from [0142]-[0143] that covariance matrices are determined for 2D positional data].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Hong, before the effective filing date of the claimed invention, to modify the method taught by Zhao and modified by Shin to specify that determining the plurality of control points of the expected parametric representation comprises obtaining a mean and/or a covariance of the plurality of control points of the expected parametric representation, as per the teachings of Hong. The motivation for this obvious combination would be to enable incorporation of state information uncertainty in the predicted trajectories, as suggested by Hong [see e.g. [0143]] which would help with planning systems that use these predictions for controlling autonomous vehicles, as also suggested by Hong [see the last 7 lines of [0025]].
Response to Arguments
Applicant’s arguments with respect to the amended independent claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Examiner specifically notes that the amended limitation reciting “using the object's past trajectory over a number of time steps as a single state of the object” is broadly interpreted, in light of the specifications, as collectively using the data pertaining to several points in past times of the trajectory of the object.
Applicant is referred to the new ground of rejection presented above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Examiner notes from the cited art:
Simba, Kenneth Renny, Naoki Uchiyama, and Shigenori Sano. "Real-time trajectory generation for mobile robots in a corridor-like space using Bézier curves." Proceedings of the 2013 IEEE/SICE International Symposium on System Integration. IEEE, 2013, which teaches a Bezier-Curve trajectory generation, wherein the Bezier curve is represented by a fixed number of parameters.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARIA S AYAD whose telephone number is (571)272-2743. The examiner can normally be reached Monday-Friday, 7:30 am - 4:30 pm. Alt, Friday, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Queler can be reached at (571) 272-4140. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARIA S AYAD/Primary Examiner, Art Unit 2172