DETAILED ACTION
Response to Amendment
The amendment filed 5/11/26 has been accepted and entered. Accordingly, claims 8-9 are amended.
Response to Arguments
With respect to the pending claims, Applicant's arguments filed have been fully considered but they are not persuasive.
With respect to the 35 U.S.C. § 102 rejection of claim 1 as anticipated by Abbot, Applicant asserts Abbot fails to disclose each element of claim 1 (Amend. 6). Applicant asserts this is because Abbot does not disclose "a second relevance of the foreign object in relation to a predicted trajectory of the ego vehicle." (Amend. 5). Applicant reasons this is because:
In rejecting claim 1, the Office Action cites portions of Abbott. The cited portions of Abbott describe that "because lane assignments may be more accurate when an object is closer to a sensor(s) of vehicle 900, as the object is moving further away, the prior lane assignment 120 may be leveraged to provide more accurate predictions of the object location and lane assignment at further distances-e.g., where the lanes appear to merge in image space. In such examples, a prior prediction(s) may be weighted with respect to current predictions of lane assignment 120 (e.g., 90% for current prediction/10% for prior prediction(s), 70% for current prediction/20% for immediately preceding prediction/10% for two predictions prior, etc.). Temporal smoothing may thus leverage prior predictions to improve the accuracy of the lane assignments 120 within the process 100." Abbott at [0062]. Therefore, Abbott describes temporal smoothing for weighting prior lane assignments against current lane assignments to improve the accuracy of object lane assignments over time
Applicant further asserts that Abbot does not disclose “selecting a foreign object...taking into account...a second relevance of the foreign object in relation to a predicted trajectory of the ego vehicle" because:
The second relevance is a "spatial assignment of the foreign object to the trajectory of the ego vehicle," where "the trajectory predicts the future course of the ego vehicle." Application p. 5 at lines 3-25. Abbott does not disclose using a predicted trajectory for the ego vehicle as a relevance for selecting a foreign object as a target object. Furthermore, Abbott does not disclose determining a predicted trajectory for the ego vehicle before selecting an object, rather it describes using the object's lane assignment to make decisions regarding a trajectory for the vehicle.
First, with respect to claim 1, it is important to note the broadest reasonable interpretation (BRI) of the limitation “selecting a foreign object, detected using the image data, as a target object, taking into account at least: a first relevance of the foreign object in relation to a lane of the ego vehicle, and a second relevance of the foreign object in relation to a predicted
trajectory of the ego vehicle”.
Under the plain ordinary meaning of the terms1, a first “relevance” includes the distance of an object to an ego lane, an object being positioned in a lane adjacent lane to the ego lane at a first time, or that the object is captured in the field of view of a camera positioned within the ego lane. This is because the phrase “selecting . . . taking into account at least . . . relevance . . . in relation to a lane of the ego vehicle” and “relevance . . . in relation to a predicted trajectory of the ego vehicle” are broad in that they merely indicate information somehow related to an ego vehicle lane and predicted trajectory.
Abbott clearly discloses an ego vehicle selecting a target object, i.e., including an object for performing a function of a driver assistance system relative to, by taking into consideration information related to an ego vehicle lane and predicted trajectory. A key point here is that Abbot discloses that the ego lane imaged ahead of the vehicle is the ego vehicles’ predicted trajectory. For example, in FIG. 6E boundaries of ego lane 664 including predicted virtual extension lines 684 and 686 used as the boundaries of the predicted ego vehicle trajectory with objects 662 and 668 located in adjacent lanes (¶ 31 “lanes may be extended using a lane extension algorithm . . . virtual lanes may be generated . . . including the ego lane . . . a path (e.g., an ego lane) of the ego vehicle”; ¶ 56 “Virtual lane generation 124 may
be used when there is limited lane data 126 and/or sensor data 102 for lane detection 112 . . . virtual lanes provide a better understanding . . . of locations of objects relative to a path of the ego-vehicle”). Because the boundaries of the ego lane imaged ahead of the vehicle are used as the predicted ego vehicle trajectory, predicted overlaps (i.e., ¶ 62 “temporal smoothing . . . leverage prior lane assignments for previous frames or images for an object to estimate the location and/or lane assignment of the object in subsequent frames or images . . . more accurate predictions of the object location and lane assignment at further distances”, i.e., current, immediately preceding and prior predictions weighted) between the object and lane boundaries ahead provide the basis for path planning and obstacle avoidance of the ego vehicle (¶ 47 “future locations and/or assignments of objects in the environment may be determined between to inform the vehicle 900 of lanes or other portions of the environment where the objects may be located at a future time to aid in trajectory or path planning, obstacle avoidance . . . determination of lane assignments 120 may be similar for future object fence locations as for current object fence locations, and the combination of current and future object fences 110 for a vehicle may provide a more detailed understanding of areas or volumes within the environment
that the object is predicted to occupy over time ( e.g., over the next second, two seconds, etc.).The future ego lane/ predicted object trajectory overlap is particularly helpful in Lane Keep Assist functionality used in Abbott (¶¶ 1 ADAS . . . lane keeping . . . when making decisions,
such as what path or trajectory to follow . . . whether to change a lane; 31; 182 vehicle 900 may include . . . lane keep assist (LKA); 183 Lateral ACC performs distance keeping, and advises the vehicle 900 to change lanes when necessary).
In addition, trajectory as defined by Applicant explicitly includes the future lane width ahead of the vehicle (Spec. ¶ 12 “vehicle movement comprises, for example, a driving trajectory and also (for example, over the lane width) a spatial extent of the required driving space within the width; 20 “The trajectory is ascertained, for example, on the basis of an expected driving trajectory of the ego vehicle (possibly plus the width of the ego vehicle). In an alternative embodiment, the trajectory is ascertained by means of a projection of the driving corridor
of the ego vehicle into the 2D camera plane predicted, for example, with the aid of a vehicle model.”).
Accordingly, Applicants arguments with respect to claim 1 are unpersuasive.
With respect to claim 5, Applicant asserts “Abbott fails to disclose ‘wherein the second relevance of the foreign object is ascertained, taking into account a geometric variable between a bounding box assigned to the foreign object and the predicted trajectory of the ego vehicle’”. Applicant reasons this is because
Abbott describes that a scaling factor, that may be applied to an object fence as the future location of the object, changes with respect to the location of the vehicle. See Abbott at [0041]. However, Abbott's does not disclose "a geometric variable between a bounding box assigned to the foreign object and the predicted trajectory of the ego vehicle." Rather, Abbott's scaling factor is a size adjustment applied to the object fence. The size adjustment does not involve computing any spatial relationship between the object's bounding box and the future path of the ego vehicle. Furthermore, Abbott does not define a predicted trajectory for the ego vehicle, and therefore cannot disclose a geometric variable between a bounding box and such a trajectory.
(Amend. 6-7).
However, as noted above, the future trajectory in the image ahead of the vehicle denoted by future ego lane boundary lines (i.e., middle lane in 6A-6E, 7A-7B and corresponding descriptions) is the predicted trajectory of the ego vehicle. Abbot does disclose this feature as an overlap between the bounding box (object fence 110) assigned to the foreign object and the predicted trajectory of the ego vehicle in the form of the future ego lane in the image ahead of the vehicle. The degree of overlap is also described as a distance (FIG. 8 steps 808-812 and corresponding description; 59 boundary scoring, overlap between pixels of object fence 110 and lane, pixel distance; 61-62; 89-90; overlap between object fence and lane is in the form of a distance; 95-96; claim 11; FIG. 1 overlap determination 118 and corresponding descriptions of overlap determination).
With respect to claim 7, Applicant asserts “Abbott fails to disclose "wherein the second relevance of the foreign object is ascertained, taking into account a temporal change in a horizontal distance between a bounding box assigned to the foreign object and the predicted trajectory of the ego vehicle," as recited in claim 7”. Applicant reasons this is because:
Abbott describes that the "scaling factor may be used as the future location of the object changes with respect to the location of the vehicle 900 (e.g., as an object moves further away, the object fence 110 may be decreased from a current size, as an object moves closer as a result of slowing down, for example, the object fence 110 may be increased in size, and so on)." Abbott at [0047]. Abbott's scaling factor, however, does not disclose "a temporal change in a horizontal distance between a bounding box assigned to the foreign object and the predicted trajectory of the ego vehicle." Abbott's scaling factor describes a one-time size adjustment to the object fence based on the object's future location. It does not disclose any computation of a temporal change of a horizontal distance between a bounding box and a predicted ego trajectory.
However, Abbot explicitly discloses taking into account a temporal change in a horizontal distance between a bounding box assigned to the foreign object and the predicted trajectory of the ego vehicle (¶¶ 59-60, 62, 89, 32, 113, 144, 175; FIG. 7A-7B; 806-812, FIG. 8; esp. ¶¶ 60-62 a ratio of intersection per lane may be determined based on the first sum of distances and the second sum of distances. Knowing the ratio of intersection, and how the ratio changes from frame to frame, may provide an indication of the trajectory of the object (e.g., switching lanes, swerving, lane keeping, etc.). In addition, the ratio of intersection may be used to assign the object to multiple lanes during lane assignment 120 . . . temporal smoothing 142 may be performed to leverage prior lane assignments 120 for previous frame(s) and/or image(s) for an object to estimate the location and/or lane assignment of the object in subsequent frame(s) and/or image(s). For example, because lane assignments may be more accurate when an object is closer to a sensor(s) of vehicle 900, as the object is moving further away, the prior lane assignment 120 may be leveraged to provide more accurate predictions of the object location and lane assignment at further distances—e.g., where the lanes appear to merge in image space. In such examples, a prior prediction(s) may be weighted with respect to current predictions of lane assignment 120 (e.g., 90% for current prediction/10% for prior prediction(s), 70% for current prediction/20% for immediately preceding prediction/10% for two predictions prior, etc.). Temporal smoothing may thus leverage prior predictions to improve the accuracy of the lane assignments 120 within the process 100).
Claim Rejections - 35 USC § 112
The rejection of claims 8-9 under 35 U.S.C. 112(b) have been withdrawn as a result of the amendment.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. 20210042535 to Abbot et al. (Abbot).
With respect to claims 1 and 10-11, Abbot discloses a method for selecting a target object for performing a function of a driver assistance system of an ego vehicle taking into account the target object, the method comprising the following steps:
reading in image data of surroundings of the ego vehicle generated using an image sensor; and
(sensor data 102, FIG. 1 and corresponding description)
selecting a foreign object, detected using the image data, as a target object, taking into account at least:
(¶ 27 object detection input may include a bounding shape, such as a box, that corresponds to an object; i.e., 204, FIG. 2A; 41 “FIG. 2A, a bounding shape 204 may be generated for a vehicle 202—using object detection 104—to identify a portion of an image that corresponds to the vehicle 202”)
a first relevance of the foreign object in relation to a lane of the ego vehicle, and
(FIG. 7A-8 and corresponding description; ¶¶ 30, 32, 46 pixels within the object fence(s) 110 may be determined to correspond to the object for lane assignment. The object fence(s) 110 may be generated prior to lane assignment 120—using any method, such as those described herein—to improve accuracy and reliability of lane assignment predictions by more closely defining the shape or footprint of the object on a driving surface. For example, once the object fences 110 are determined, the object fences 110 may be used—in combination with the lane masks 116—to make an overlap determination 118 for lane assignment 120.; 57-61, 73-74; 82-83; 89-97; claims 1-7, 10,-11, 18)
(FIG. 1, overlap determination 118, boundary scoring 134, lane assignment 120)
a second relevance of the foreign object in relation to a predicted trajectory of the ego vehicle.
(¶¶ 29, 47, 60-62)
(¶¶ 61-62 “For example, because lane assignments may be more accurate when an object is closer to a sensor(s) of vehicle 900, as the object is moving further away, the prior lane assignment 120 may be leveraged to provide more accurate predictions of the object location and lane assignment at further distances—e.g., where the lanes appear to merge in image space. In such examples, a prior prediction(s) may be weighted with respect to current predictions of lane assignment 120 (e.g., 90% for current prediction/10% for prior prediction(s), 70% for current prediction/20% for immediately preceding prediction/10% for two predictions prior, etc.). Temporal smoothing may thus leverage prior predictions to improve the accuracy of the lane assignments 120 within the process 100”; i.e., first relevance is more important and second relevance is less so, and the relative first and second relevance weightings are used to determine predictions of lane assignment; see also US 20190266418, US 20200293064, US 20190286153 incorporated by reference into Abbot at ¶¶ 43, 48)
With respect to claim 2, Abbot discloses the first relevance of the foreign object is ascertained taking into account an overlap of the foreign object and the lane of the ego vehicle.
(FIG. 7A-8 and corresponding description; ¶¶ 30, 32, 46 pixels within the object fence(s) 110 may be determined to correspond to the object for lane assignment. The object fence(s) 110 may be generated prior to lane assignment 120—using any method, such as those described herein—to improve accuracy and reliability of lane assignment predictions by more closely defining the shape or footprint of the object on a driving surface. For example, once the object fences 110 are determined, the object fences 110 may be used—in combination with the lane masks 116—to make an overlap determination 118 for lane assignment 120.; 57-61, 73-74; 82-83; 89-97; claims 1-7, 10,-11, 18)
With respect to claim 3, Abbot discloses image data in a native measurement space of the image sensor are used to ascertain the second relevance.
(FIG. 1, sensor data 102, object detection 104, free space detection 106, fence generation 108, object fence 110; FIG. 5, 502-506; FIG. 3, 302-304; ¶ 47 once the object fence 110 is determined, the sensor data 102-representative of speed, velocity, acceleration, yaw rate, etc.-may be used to determine a future path or trajectory of the objects in the environment to determine one or more future locations (e.g., 0.5 seconds in the future, 1 second in the future, etc.).)
With respect to claim 4, Abbot discloses wherein, to ascertain the second relevance:
a bounding box is assigned to the foreign object in a native measurement space of the image sensor, and/or
the predicted trajectory is defined in a native measurement space of the image sensor.
(¶ 41 bounding shaped output using object detection . . . bounding shape 204 may be generated for a vehicle 202-using object detection 104-to identify a portion of an image that corresponds to the vehicle 202)
(FIG. 1, sensor data 102, object detection 104, free space detection 106, fence generation 108, object fence 110; FIG. 5, 502-506; FIG. 3, 302-304; ¶ 47 once the object fence 110 is determined, the sensor data 102-representative of speed, velocity, acceleration, yaw rate, etc.-may be used to determine a future path or trajectory of the objects in the environment to determine one or more future locations (e.g., 0.5 seconds in the future, 1 second in the future, etc.).)
With respect to claim 5, Abbot discloses wherein the second relevance of the foreign object is ascertained, taking into account a geometric variable between a bounding box assigned to the foreign object and the predicted trajectory of the ego vehicle.
(¶ 41 bounding shaped output using object detection . . . bounding shape 204 may be generated for a vehicle 202-using object detection 104-to identify a portion of an image that corresponds to the vehicle 202 . . . Once the future location(s) are known, the object fence 110 may be generated ( e.g., in image space) using the future location and the object fence 110 information . . . scaling factor may be used as the future location of the object changes with respect to the location of the vehicle 900 (e.g., as an object moves further away, the object fence 110 may be decreased from a current size, as an object moves closer as a result of slowing down, for example, the object fence 110 may be increased in size, and so on). This information may then be used to inform the vehicle 900 of lanes or other portions of the environment where the objects may be located at a future time to aid in trajectory or path planning, obstacle avoidance, and/or other operations of the vehicle; FIG. 8 steps 808-812 and corresponding description; 59 boundary scoring, overlap between pixels of object fence 110 and lane, pixel distance; 61-62; 89-90; overlap between object fence and lane is in the form of a distance; 95-96; claim 11; FIG. 1 overlap determination 118 and corresponding descriptions of overlap determination; see also US 20190266418, US 20200293064, US 20190286153 incorporated by reference into Abbot at ¶¶ 43, 48)
With respect to claim 6, Abbot discloses the second relevance of the foreign object is ascertained, taking into account a horizontal distance between the bounding box assigned to the foreign object and the predicted trajectory of the ego vehicle.
(¶ 41 bounding shaped output using object detection . . . bounding shape 204 may be generated for a vehicle 202-using object detection 104-to identify a portion of an image that corresponds to the vehicle 202 . . . Once the future location(s) are known, the object fence 110 may be generated ( e.g., in image space) using the future location and the object fence 110 information . . . scaling factor may be used as the future location of the object changes with respect to the location of the vehicle 900 (e.g., as an object moves further away, the object fence 110 may be decreased from a current size, as an object moves closer as a result of slowing down, for example, the object fence 110 may be increased in size, and so on). This information may then be used to inform the vehicle 900 of lanes or other portions of the environment where the objects may be located at a future time to aid in trajectory or path planning, obstacle avoidance, and/or other operations of the vehicle; ¶¶ 59-60, 62, 89, 32, 113, 144, 175; FIG. 7A-7B; 806-812, FIG. 8)
With respect to claim 7, Abbot discloses the second relevance of the foreign object is ascertained, taking into account a temporal change in a horizontal distance between a bounding box assigned to the foreign object and the predicted trajectory of the ego vehicle
(¶ 41 bounding shaped output using object detection . . . bounding shape 204 may be generated for a vehicle 202-using object detection 104-to identify a portion of an image that corresponds to the vehicle 202 . . . Once the future location(s) are known, the object fence 110 may be generated ( e.g., in image space) using the future location and the object fence 110 information . . . scaling factor may be used as the future location of the object changes with respect to the location of the vehicle 900 (e.g., as an object moves further away, the object fence 110 may be decreased from a current size, as an object moves closer as a result of slowing down, for example, the object fence 110 may be increased in size, and so on). This information may then be used to inform the vehicle 900 of lanes or other portions of the environment where the objects may be located at a future time to aid in trajectory or path planning, obstacle avoidance, and/or other operations of the vehicle; ¶¶ 59-60, 62, 89, 32, 113, 144, 175; FIG. 7A-7B; 806-812, FIG. 8; esp. ¶¶ 60-62 a ratio of intersection per lane may be determined based on the first sum of distances and the second sum of distances. Knowing the ratio of intersection, and how the ratio changes from frame to frame, may provide an indication of the trajectory of the object (e.g., switching lanes, swerving, lane keeping, etc.). In addition, the ratio of intersection may be used to assign the object to multiple lanes during lane assignment 120 . . . temporal smoothing 142 may be performed to leverage prior lane assignments 120 for previous frame(s) and/or image(s) for an object to estimate the location and/or lane assignment of the object in subsequent frame(s) and/or image(s). For example, because lane assignments may be more accurate when an object is closer to a sensor(s) of vehicle 900, as the object is moving further away, the prior lane assignment 120 may be leveraged to provide more accurate predictions of the object location and lane assignment at further distances—e.g., where the lanes appear to merge in image space. In such examples, a prior prediction(s) may be weighted with respect to current predictions of lane assignment 120 (e.g., 90% for current prediction/10% for prior prediction(s), 70% for current prediction/20% for immediately preceding prediction/10% for two predictions prior, etc.). Temporal smoothing may thus leverage prior predictions to improve the accuracy of the lane assignments 120 within the process 100).
With respect to claim 8, Abbot discloses wherein the function of the driver assistance system of the ego vehicle is performed based on the selected target object taking into account:
(i) a driving situation and/or a driving environment and/or a traffic situation; and/or
(ii) an ascertained object class of the target object
(¶¶ 29, 41, 47, 59-60, 62, 89, 32, 109 advanced driver assistance, 111, 113, 144, 175; 812, Fig. 8)
With respect to claim 9, Abbot discloses the function of the driver assistance system of the ego vehicle is defined based on the selected target object in a native measurement space of the image sensor.
(FIG. 1, sensor data 102, object detection 104, free space detection 106, fence generation 108, object fence 110; FIG. 5, 502-506; FIG. 3, 302-304; ¶ 47 once the object fence 110 is determined, the sensor data 102-representative of speed, velocity, acceleration, yaw rate, etc.-may be used to determine a future path or trajectory of the objects in the environment to determine one or more future locations (e.g., 0.5 seconds in the future, 1 second in the future, etc.).)
(¶ 41 bounding shaped output using object detection . . . bounding shape 204 may be generated for a vehicle 202-using object detection 104-to identify a portion of an image that corresponds to the vehicle 202 . . . Once the future location(s) are known, the object fence 110 may be generated ( e.g., in image space) using the future location and the object fence 110 information . . . scaling factor may be used as the future location of the object changes with respect to the location of the vehicle 900 (e.g., as an object moves further away, the object fence 110 may be decreased from a current size, as an object moves closer as a result of slowing down, for example, the object fence 110 may be increased in size, and so on). This information may then be used to inform the vehicle 900 of lanes or other portions of the environment where the objects may be located at a future time to aid in trajectory or path planning, obstacle avoidance, and/or other operations of the vehicle; ¶¶ 29, 41, 47, 59-60, 62, 89, 32, 109 advanced driver assistance, 111, 113, 144, 175; 812, Fig. 8)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNETH J MALKOWSKI whose telephone number is (313)446-4854. The examiner can normally be reached 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Faris Almatrahi can be reached at 313-446-4821. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNETH J MALKOWSKI/Primary Examiner, Art Unit 3667
1 Applicant asserts the second relevance is limited by the specification as a “spatial assignment” to the “future course” of the vehicle. However, these quoted potions are not claim requirements and the specification does not use them as a limiting definition as it is merely a description of “one possible embodiment”. See published specification (hereinafter “Spec.”) ¶ 17. See MPEP 2111.01.
Prosecution Insights